作者想表达自己的感激之情洪虎先生在执行本文报道的实验援助。这项工作是由中国国家自然科学基金（61473207号）的部分支持。

1.获取脑电信号<ol><li>解释了实验过程的主题，并取得书面知情同意参加实验。 </li><li>测量被检者的头部用卷尺测量的周长，并选择EEG帽大小是接近测量。电极排列是根据"国际:10-20系统"12。 </li><li>测量鼻根和INION之间的距离。用皮肤标记笔标记的距离作为用于对准​​的帽的基准的10％，并标记的距离作为被检者的头皮的顶点的中点。 </li><li>通过与FP1和FP2电极的中点对准10％大关将关于这一主题的脑电图帽。位置电极锆石的顶点帽。 </li><li>确保Fz的，CZ，Pz的和盎司是在头的中线，而每对位于所述中线的任一侧电极是在一个水平线上。 </li><li>涂抹REF1和REF2电极的导电凝胶。放置在使用医用胶带左和右乳突参考电极。拧紧南极。 </li><li>放置一个钝头的注射器在电极座和导电凝胶按以下顺序注入到各电极:第一，在前额和"地"电极秒，在实验中使用的，O 2，O 1，盎司，Pz的五个电极和Cz。 </li><li>座椅舒适的椅子60cm以下的刺激监视器前的对象。指示的主题，以保持他/她的眼睛在同一水平与显示器的中心。 </li><li>电极导线连接到脑电数据采集系统。配置采样率到1千赫。 </li><li>检查的专用DPC脑电图信号质量。如果存在与特定的电极的问题，重新注入凝胶来调整信道的阻抗。 </li><li>使用专用的SPC闪烁4机器人图像在4.615，12，15和20的频率赫兹的用户界面上的视觉刺激，如图2。 </li></ol> 2.离线分析SSVEP特点<ol><li>行为离线训练试验32试验为每个主题并记录脑电图系统，在此过程中获得他们的大脑信号。 注:此过程仅受实验的第一次主题进行，以建立自己的SSVEP特征向量和训练分类。 </li><li>当审判开始时，选择一个刺激作为目标随机显示其上方的黄色箭头。 </li><li> 1秒后，无闪烁的四个视觉刺激的用户界面，持续5秒上不同的频率。 </li><li>要求受试者对着眼于所选择的目标的刺激，同时保持他/她的身体运动到最低限度。 </li><li>每次试验后，给受3秒放松，然后再开始下一个试验。 </li><li>当所有的试验完成后，读取保存的数据。提取了3秒数据时代是<html>吐温2秒，5秒，其选择的刺激在每次试验的目标后触发。 </li><li>计算使用离线数据分析在MATLAB脚本数据时代的功率谱密度（PSD）。使用下面将详细描述的处理算法: <ol><li>计算多通道脑电数据的典型相关分析（CCA）的系数与参考的数据。 CCA是用于两组数据中找到自己的潜在关联13多变量统计方法。考虑多信道脑电图数据 X和基准数据 Y相同长度的三分段，使用CCA找到的权重向量， 宽 x和 W Y，最大限度的相关系数ρX = X T W＆x 和y = YŤ之间W¯¯ 年。设置参考数据是在一定的刺激频率的周期信号。 /ftp_upload/53558/53558eq1.jpg"/&gt; 其中F 1，F 2，F 3，F 4是四视觉刺激的特征频率。 </li><li>空间地使用所计算的CCA系数宽 x以获得一维滤波的数据x，它具有的基准数据的线性组合的最突出的相关过滤多信道脑电图数据 X。 </li><li>计算使用快速傅立叶变换（FFT）的空间滤波数据 x的PSD。 <img alt="公式（2）" src="/files/ftp_upload/53558/53558eq2.jpg" /> 其中 N FFT为数据 X的样本量和FFT（X，N FFT）返回 x 的 N个FFT -点离散傅里叶变换。 </li><li>规范化PSD相对于3和30赫兹之间的平均值。 <img alt="公式3" src="/files/ftp_upload/53558/53558eq3.jpg" /> 哪里60; <img alt="公式3分母" src="/files/ftp_upload/53558/53558eq3denominator.jpg" />表示3〜30Hz的功率谱的平均值。 </li></ol></li><li>校准分类参数为四个激励频率。首先，观察到对应其目标的刺激的标准化的PSD光谱，和手动配置具有最强的PSD振幅作为特征频率的频率。基于该特征频率，划分归一化的PSD数据分成两组:当相应的刺激被选择作为目标之一获得，并且当非目标刺激被选择的其他被获取。计算每个分别设置的平均值。由两个平均值的中值确定阈值进行分类的目标和非目标集14。 </li></ol> 3.在线处理大脑信号<ol><li>打开在线信号处理器，如图3，该显影bASED对OpenViBE环境和MATLAB脚本，在DPC进行在线处理的大脑信号。 </li><li>双击获取和处理信号盒，标有红色的周期在图3中，打开OpenViBE设置窗口。为主题配置的分类参数:根据离线分析，结果设置采样速率为1千赫兹，设定时间长度进行FFT为3秒，并设定所述特征频率例如 ，4.667，24，15和20赫兹。 </li><li>点击首发按钮运行在线信号处理器，使用以下算法中的三个步骤，其处理所述实时数据。 </li><li>首先，算法获取从通道盎司，O1，O2，和Pz的脑电图系统每0.5s的CPZ脑电图数据，并提取在过去3秒进行在线处理的数据段。 </li><li>第二，该算法处理使用在步骤2.7中描述的算法的3秒的数据段，并计算实时的PSD为classifica化。 </li><li>第三，该算法根据PSD幅度在四个特征频率划分的脑电波模式。当一个特征频率的幅度高于给定阈值，分类的刺激闪烁在相应的频率作为SSVEP目标。 </li></ol> 4.连接仿人机器人<ol><li>按人形机器人的胸部按钮，并等待其启动。 注:此过程大约需要1分钟，当机器人说:"OGNAK GNOUK"，而它的胸口按钮变成白色完毕。 </li><li>建立它的无线保真（WiFi）连接到DPC 15。 </li><li>开放式机器人控制器编程，在DPC的Visual C ++脚本。该控制器接收来自通过虚拟现实外设网络（VRPN）接口的在线信号处理器的分类结果，将其转换为所述机器人的命令，并控制humanoi的对应的行为通过无线连接二维机器人。 </li><li>通过根据其WiFi连接输入机器人的IP地址，定义三套机器人行为它可以是配置机器人控制器容易地切换来执行三个不同的任务，并且设置这些行为的执行参数（例如，步行速度或距离），以适应任务的要求。 注:以上这些行为的详细步骤5.2.1，5.3.1，5.4.1和描述。 </li><li>点击在Visual Studio平台的生成按钮运行机器人控制器。 </li><li>打开对SPC的Choregraphe程序。点击"连接"按钮，弹出一个连接部件。选择要根据其IP地址来控制机器人，并双击图标连接到它。 </li><li>从在Choregraphe菜单栏的视图菜单打开视频显示器，以及监视器移动到用户界面的前以显示从机器人的相机实时视频反馈。 </li></ol> &lt;P类="j​​ove_title"&gt; 5。开展闭环控制实验<ol><li>要求受试者对以评价大脑机器人交互性能执行在不同环境中的三个具体的闭环控制任务。 注:这些任务，这在机器人应用中非常重要，因为他们是在帮助残疾人和老年人在日常生活有帮助的，需要激活不同的机器人行为。 </li><li>进行首次闭环控制任务网真通过大脑信号控制的人形机器人走过的障碍，推动光开关，把灯打开， 如图4。 <ol><li>为了完成第一个任务，控制其通过四个SSVEP刺激编码为机器人的行为:用0.15微米的固定距离步进向前，向左转具有0.3弧度的固定弧度，右转具有0.3弧度的固定弧度，并推动使用其右手的开关。 注意:CONFIGURA化过程在步骤4.4中所述。 </li><li>通知任务目标的主题和四个行为来进行控制。 </li><li>运行机器人控制器按照步骤4.5-4.7，并开始实验。给予受试者的自由感知环境，并根据实时视频反馈的决定，以及由盯着相应的刺激以激活机器人的行为。 注:在没有指令或帮助应给予受试者除非在紧急情况下，例如，机器人将要碰撞的障碍物。 </li></ol></li><li>开展第二次任务为网真控制通过大脑信号的人形机器人朝楼梯走下面的出口标志， 如图5。当面对路人，询问受控制机器人说"对不起"，并等待路人让路。 <ol><li>控制由四SSVEP刺激编码为机器人的行为:向前走continuousl的Y为0.05米/秒的速度，在拐弯0.1弧度/秒的速度连续地离开，在0.1弧度/秒的速度连续地向右转动，并停止所有行走行为。 </li><li>切换机器人控制器的姿势相关的控制功能16。 注:当机器人行走时，该功能采用第四刺激如停止行走行为的命令;当机器人不走，它采用第四刺激来控制机器人说"对不起"。该功能在C ++中的脚本通过探测机器人的行走状态来实现的。 </li><li>通知任务目标的主题和五个行为来进行控制。 </li><li>运行机器人控制器和开始实验，如在步骤5.2.3说明。 </li></ol></li><li>进行第三个任务为网真控制类人机器人行走朝气球目标，把它捡起来，并把它提供给主体的手， 如图6。 <ol><li>控制由四SSVEP刺激作为编码的机器人行为:以0.15米的固定距离挺身而出，左转0.3弧度固定弧度，0.3弧度固定弧度右转，并拿起对象。 </li><li>激活在机器人控制器的姿势有关的控制功能再利用第四刺激作为放下对象时，它已经被拾取的命令。 </li><li>通知任务目标的主题和五个行为来进行控制。 </li><li>运行机器人控制器和开始实验，如在步骤5.2.3说明。 </li></ol></li></ol>

<ol><li>McFarland, D. J., Wolpaw, J. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Brain-Computer+Interface+Operation+of+Robotic+and+Prosthetic+Devices%5BTitle%5D)+AND+%22Computer%22%5BJournal%5D)">Brain-Computer Interface Operation of Robotic and Prosthetic Devices.</a> Computer. 41, 52-56 (2008).</li>
<li>Lebedev, M. A., Nicolelis, M. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Brain-machine+interfaces%3A+Past%2C+present+and+future%5BTitle%5D)+AND+%22Trends+Neruosci%22%5BJournal%5D)">Brain-machine interfaces: Past, present and future.</a> Trends Neruosci. 29 (9), 536-546 (2006).</li>
<li>Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., Vaughan, T. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Brain-computer+interfaces+for+communication+and+control%5BTitle%5D)+AND+%22Clin.+Neurophysiol%22%5BJournal%5D)">Brain-computer interfaces for communication and control.</a> Clin. Neurophysiol. 113, 767-791 (2002).</li>
<li>Bell, C. J., Shenoy, P., Chalodhorn, R., Rao, R. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Control+of+a+humanoid+robot+by+a+noninvasive+brain-computer+interface+in+humans%5BTitle%5D)+AND+%22J.+Neural.+Eng%22%5BJournal%5D)">Control of a humanoid robot by a noninvasive brain-computer interface in humans.</a> J. Neural. Eng. 5, 214-220 (2008).</li>
<li>Li, W., Li, M., Zhao, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Control+of+humanoid+robot+via+motion-onset+visual+evoked+potentials%5BTitle%5D)+AND+%22Front.+Syst.+Neurosci%22%5BJournal%5D)">Control of humanoid robot via motion-onset visual evoked potentials.</a> Front. Syst. Neurosci. 8, 247(2014).</li>
<li>Regan, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Some+characteristics+of+average+steady-state+and+transient+responses+evoked+by+modulated+light%5BTitle%5D)+AND+%22Electroencephalogr.+Clin.+Neurophysiol%22%5BJournal%5D)">Some characteristics of average steady-state and transient responses evoked by modulated light.</a> Electroencephalogr. Clin. Neurophysiol. 20, 238-248 (1966).</li>
<li>Vialatte, F. B., Maurice, M., Dauwels, J., Cichocki, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Steady-state+visually+evoked+potentials%3A+focus+on+essential+paradigms+and+future+perspectives%5BTitle%5D)+AND+%22Prog.+Neurobiol%22%5BJournal%5D)">Steady-state visually evoked potentials: focus on essential paradigms and future perspectives.</a> Prog. Neurobiol. 90, 418-438 (2010).</li>
<li>Bin, G., Gao, X., Wang, Y., Li, Y., Hong, B., Gao, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+high-speed+BCI+based+on+code+modulation+VEP%5BTitle%5D)+AND+%22J.+Neural.+Eng%22%5BJournal%5D)">A high-speed BCI based on code modulation VEP.</a> J. Neural. Eng. 8, 025015(2011).</li>
<li>Sutton, S., Braren, M., Zubin, J., John, E. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Evoked-potential+correlates+of+stimulus+uncertainty%5BTitle%5D)+AND+%22Science%22%5BJournal%5D)">Evoked-potential correlates of stimulus uncertainty.</a> Science. 150, 1187-1188 (1965).</li>
<li>Pfurtscheller, G., Lopes da Silva, H. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Event-related+EEG%2FMEG+synchronization+and+desynchronization%3A+basic+principles%5BTitle%5D)+AND+%22Clin.+Neurophysiol%22%5BJournal%5D)">Event-related EEG/MEG synchronization and desynchronization: basic principles.</a> Clin. Neurophysiol. 110, 1842-1857 (1999).</li>
<li>Zhao, J., Meng, Q., Li, W., Li, M., Sun, F., Chen, G. OpenViBE-based brainwave control system for Cerebot. Proc. IEEE International Conference on Robotics and Biomimetics. , 1169-1174 (2013).</li>
<li>Homan, R. W., Herman, J., Purdy, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Cerebral+location+of+international+10-20+system+electrode+placement%5BTitle%5D)+AND+%22Electroencephalogr.+Clin.+Neurophysiol%22%5BJournal%5D)">Cerebral location of international 10-20 system electrode placement.</a> Electroencephalogr. Clin. Neurophysiol. 66, 376-382 (1987).</li>
<li>Bin, G., Gao, X., Yan, Z., Hong, B., Gao, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((An+online+multi-channel+SSVEP-based+brain-computer+interface+using+a+canonical+correlation+analysis+method%5BTitle%5D)+AND+%22J.+Neural.+Eng%22%5BJournal%5D)">An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method.</a> J. Neural. Eng. 6, 046002(2009).</li>
<li>Wang, Y., Wang, R., Gao, X., Hong, B., Gao, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+practical+VEP-based+brain-computer+interface%5BTitle%5D)+AND+%22IEEE+Trans.+Neural+Syst.+Rehabil.+Eng%22%5BJournal%5D)">A practical VEP-based brain-computer interface.</a> IEEE Trans. Neural Syst. Rehabil. Eng. 14, 234-239 (2006).</li>
<li>Setting NAO's WiFi connection - Aldebaran 2.1.3.3 documentation [Internet]. , Aldebaran Robotics Co. Available from:
 <a target="_blank" href="http://doc.aldebaran.com/2-1/nao/nao-connecting.html">http://doc.aldebaran.com/2-1/nao/nao-connecting.html</a> (2015).</li>
<li>Chae, Y., Jeong, J., Jo, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Toward+brain-actuated+humanoid+robots%3A+asynchronous+direct+control+using+an+EEG-based+BCI%5BTitle%5D)+AND+%22IEEE+T.+Robot%22%5BJournal%5D)">Toward brain-actuated humanoid robots: asynchronous direct control using an EEG-based BCI.</a> IEEE T. Robot. 28, 1131-1144 (2012).</li>
<li>Croft, R. J., Barry, R. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Removal+of+ocular+artifact+from+the+EEG%3A+a+review%5BTitle%5D)+AND+%22Neurophysiol.+Clin%22%5BJournal%5D)">Removal of ocular artifact from the EEG: a review.</a> Neurophysiol. Clin. 30, 5-19 (2000).</li>
<li>Hwang, H. J., Hwan Kim, D., Han, C. H., Im, C. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+new+dual-frequency+stimulation+method+to+increase+the+number+of+visual+stimuli+for+multi-class+SSVEP-based+brain-computer+interface+%28BCI%29%5BTitle%5D)+AND+%22Brain+Res%22%5BJournal%5D)">A new dual-frequency stimulation method to increase the number of visual stimuli for multi-class SSVEP-based brain-computer interface (BCI).</a> Brain Res. 1515, 66-77 (2013).</li>
<li>Wang, M., Daly, I., Allison, B., Jin, J., Zhang, Y., Chen, L., Wang, X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+new+hybrid+BCI+paradigm+based+on+P300+and+SSVEP%5BTitle%5D)+AND+%22J+Neurosci+Methods%22%5BJournal%5D)">A new hybrid BCI paradigm based on P300 and SSVEP.</a> J Neurosci Methods. 244, 16-25 (2015).</li>
<li>Allison, B. Z., Jin, J., Zhang, Y., Wang, X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+four-choice+hybrid+P300+SSVEP+BCI+for+improved+accuracy%5BTitle%5D)+AND+%22Brain-Computer+Interfaces%22%5BJournal%5D)">A four-choice hybrid P300 SSVEP BCI for improved accuracy.</a> Brain-Computer Interfaces. 1, 17-26 (2014).</li>
<li>Pan, J. H., Xie, Q., Herman, Y., Wang, F., Di, H., Laureys, S., Yu, R., Li, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Detecting+awareness+in+patients+with+disorders+of+consciousness+using+a+hybrid+brain-computer+interface%5BTitle%5D)+AND+%22J.+Neural.+Eng%22%5BJournal%5D)">Detecting awareness in patients with disorders of consciousness using a hybrid brain-computer interface.</a> J. Neural. Eng. 11, 056007(2014).</li>
<li>Li, J., Ji, H., Cao, L., Zhang, D., Gu, R., Xia, B., Wu, Q. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Evaluation+and+application+of+a+hybrid+brain+computer+interface+for+real+wheelchair+parallel+control+with+multi-degree+of+freedom%5BTitle%5D)+AND+%22Int+J+Neural+Syst%22%5BJournal%5D)">Evaluation and application of a hybrid brain computer interface for real wheelchair parallel control with multi-degree of freedom.</a> Int J Neural Syst. 24, 1450014-14 (2014).</li>
<li>Zhao, J., Meng, Q., Li, W., Li, M., Chen, G. SSVEP-based hierarchical architecture for control of a humanoid robot with mind. Proc. 11th World Congress on Intelligent Control and Automation. , 2401-2406 (2014).</li>
<li>Zhu, D., Bieger, J., Molina, G. G., Aarts, R. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+survey+of+stimulation+methods+used+in+SSVEP-based+BCIs%5BTitle%5D)+AND+%22Comput+Intell+Neurosci%22%5BJournal%5D)">A survey of stimulation methods used in SSVEP-based BCIs.</a> Comput Intell Neurosci. 2010, 1-12 (2010).</li>
<li>Muller-Putz, G. R., Scherer, R., Brauneis, C., Pfurtscheller, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Steady-state+visual+evoked+potential+%28SSVEP%29-based+communication%3A+impact+of+harmonic+frequency+components%5BTitle%5D)+AND+%22J+Neural+Eng%22%5BJournal%5D)">Steady-state visual evoked potential (SSVEP)-based communication: impact of harmonic frequency components.</a> J Neural Eng. 2, 123-130 (2005).</li>
</ol>

The authors have nothing to disclose.

本文提出了一种SSVEP为基础的实验过程通过集成多个软件程序，建立脑机器人交互系统，人形机器人。因为人的意图是通过解释实时EEG信号感知，关键是在进行实验之前验证电极连接和脑电图信号质量。如果从所有的电极获得的信号是差的品质，就必须首先检查接地电极和参考电极的连接。如果有问题，电极的部分，重新​​注入传导凝胶来调整这些信道的阻抗。 同时获取脑...

脑机器人交互（BRI），它提供了人，并通过大脑信号的磁带设备之间的创新的通信通道，是未来的在帮助残疾人在日常生活中1,2。各种各样的方法都能够获得大脑信号要么侵入或非侵入性，如脑电图（脑电图），脑电图（EEG），功能磁共振成像（fMRI）等用于建立最常用的非侵入性的方法BRI系统是获得从置于头皮上的电极脑电信号。这种方法是便宜的，易于使用，并提供了一个可接受的时间分辨率3。在众多的机器人设备，仿人机器人是先进的，因为他们创造了模仿一些人类每天接受同样的身体和精神的任务。 BRI，带仿人机器人将在协助老弱病残，以及执行不卫生或危险的工作具有重要作用。但控制通过BRI系统仿人机器人是极具挑战性，因为仿人机器人全身运动发展到执行复杂的任务，如个人协助4，5。 稳态视觉诱发电位（SSVEP）是在给定频率6诱发的视觉刺激的调制类型的脑信号的。它包含血窦在闪烁刺激的基波和谐波频率，并显着地出现在整个头皮7枕骨区域的视觉皮层。之所以选择SSVEP信号是，SSVEP为基础的BRI系统产生较高的信息传输速率，并需要更少的培训8。其他类型的脑电波，如事件相关电位（ERP的）9或马达成像（MI）的电位10，也可以嵌入到该实验步骤。 我们与仿人脑机器人互动过程机器人是基于Cerebot -一种思维控制类人机器人平台-包括脑电数据采集系统和人形机器人11。脑电图系统是能够记录，预处理和由各种类型的电极获取的显示生物电势信号。它提供了多个模拟的I / O和数字I / O，并且能够同时记录多达128个信号通道以30千赫具有16位分辨率的采样率的。它在C ++和MATLAB软件开发工具包便于用户设计实验步骤。人形机器人具有25个自由度，并配备有多个传感器，包括2个摄像头，麦克风4，2声纳测距仪，2 IR发射器和接收器，1惯性板，9触觉传感器，和8的压力传感器。它提供了Choregraphe和C ++ SDK用于创建和编辑的动作和互动的机器人行为。 这种方法的总体目标是建立一个SSVEP为基础的实验PROCE杜热通过集成多个软件程序，如OpenViBE，Choregraph，中央软件以及写在C ++和MATLAB用户开发的程序，以实现与人形机器人11脑机器人相互作用的研究。 图1显示了系统的结构。专用刺激呈现计算机（SPC）显示用户界面以提供与视觉刺激，指示和环境反馈的主题。专用数据处理计算机（DPC）运行在数据记录器和离线数据分析器中的离线训练的过程中，并运行在线信号处理器和机器人控制器的人形机器人的在线控制。与其他SSVEP为基础的控制系统相比，我们的系统更可靠，更灵活，尤其是更方便的重复使用和更新，因为它是通过将若干标准化的软件包，诸如OpenViBE，Choregraph，中部软件开发的，并模块用C ++编写和MATLAB。 下面的过程进行了审查并批准了天津医科大学总医院伦理委员会，和所有科目签署知情同意书。

<table><tbody><tr><td>Cerebus EEG Data Acquisition System</td><td>Blackrock Microsystems</td><td>4176-9967</td><td></td></tr><tr><td>NAO humanoid robot</td><td>Aldebaran Robotics</td><td>H25</td><td></td></tr><tr><td>EEG cap</td><td>Neuroscan</td><td>8732</td><td></td></tr><tr><td>Ten20 Conductive gel</td><td>Weaver and company</td><td>10-20-8</td><td></td></tr></tbody></table>

ssvep-based experimental procedure for brain-robot interaction with humanoid robots

Brain-Robot Interaction (BRI), which provides an innovative communication pathway between human and a robotic device via brain signals, is prospective in helping the disabled in their daily lives. The overall goal of our method is to establish an SSVEP-based experimental procedure by integrating multiple software programs, such as OpenViBE, Choregraph, and Central software as well as user developed programs written in C++ and MATLAB, to enable the study of brain-robot interaction with humanoid robots.
This is achieved by first placing EEG electrodes on a human subject to measure the brain responses through an EEG data acquisition system. A user interface is used to elicit SSVEP responses and to display video feedback in the closed-loop control experiments. The second step is to record the EEG signals of first-time subjects, to analyze their SSVEP features offline, and to train the classifier for each subject. Next, the Online Signal Processor and the Robot Controller are configured for the online control of a humanoid robot. As the final step, the subject completes three specific closed-loop control experiments within different environments to evaluate the brain-robot interaction performance.
The advantage of this approach is its reliability and flexibility because it is developed by integrating multiple software programs. The results show that using this approach, the subject is capable of interacting with the humanoid robot via brain signals. This allows the mind-controlled humanoid robot to perform typical tasks that are popular in robotic research and are helpful in assisting the disabled.

此处呈现的结果是从男性受试者具有矫正到正常获得的版本。 图7示出的处理的EEG数据的方法，包括提取一个多信道数据纪元（图7A），空间滤波使用CCA系数数据（图7B） ，并计算归一化的PSD（图7C）。 图8示出了使用单一的试验中，受试者盯着闪烁在不同频率靶?...

Watch this Scientific Journal Video about SSVEP-based Experimental Procedure for Brain-Robot Interaction with Humanoid Robots at JoVE.com

SSVEP-based Experimental Procedure for Brain-Robot Interaction with Humanoid Robots

The overall goal of this method is to establish an SSVEP-based experimental procedure by integrating multiple software programs to enable the study of brain-robot interaction with humanoid robots, which is prospective in assisting the sick and elderly as well as performing unsanitary or dangerous jobs.

SSVEP基础实验步骤与人形机器人脑机器人互动

Brain-Robot Interaction (BRI), which provides an innovative communication pathway between human and a robotic device via brain signals, is prospective in ...

ssvep-based-experimental-procedure-for-brain-robot-interaction-with

Research

Education

JoVE Core

JoVE Journal

Electrical Engineering

Engineering

Unit 1

Introduction to Control Systems

SCIENTISTS IN ACTION

13.1K 次观看.  Tianjin University. The overall goal of this method is to establish an SSVEP-based experimental procedure by integrating multiple software programs to enable the study of brain-robot interaction with humanoid robots, which is prospective in assisting the sick and elderly as well as performing unsanitary or dangerous jobs.

视频: SSVEP-based Experimental Procedure for Brain-Robot Interaction with Humanoid Robots

An Experimental Platform to Study the Closed-loop Performance of Brain-machine Interfaces

The non-stationary nature and variability of neuronal signals is a fundamental problem in brain-machine interfacing. We developed a brain-machine interface to assess the robustness of different control-laws applied to a closed-loop image stabilization task. Taking advantage of the well-characterized fly visuomotor pathway we record the electrical activity from an identified, motion-sensitive neuron, H1, to control the yaw rotation of a two-wheeled robot. The robot is equipped with 2 high-speed video cameras providing visual motion input to a fly placed in front of 2 CRT computer monitors. The activity of the H1 neuron indicates the direction and relative speed of the robot's rotation. The neural activity is filtered and fed back into the steering system of the robot by means of proportional and proportional/adaptive control. Our goal is to test and optimize the performance of various control laws under closed-loop conditions for a broader application also in other brain machine interfaces.

We use a closed-loop fly-machine interface to investigate general principles in neuronal control.

研究脑机接口的闭环性能的实验平台

The non-stationary nature and variability of neuronal signals is a fundamental problem in brain-machine interfacing. We developed a brain-machine ...

科研

神经科学

Haptic/Graphic Rehabilitation: Integrating a Robot into a Virtual Environment Library and Applying it to Stroke Therapy

Recent research that tests interactive devices for prolonged therapy practice has revealed new prospects for robotics combined with graphical and other forms of biofeedback. Previous human-robot interactive systems have required different software commands to be implemented for each robot leading to unnecessary developmental overhead time each time a new system becomes available. For example, when a haptic/graphic virtual reality environment has been coded for one specific robot to provide haptic feedback, that specific robot would not be able to be traded for another robot without recoding the program. However, recent efforts in the open source community have proposed a wrapper class approach that can elicit nearly identical responses regardless of the robot used. The result can lead researchers across the globe to perform similar experiments using shared code. Therefore modular "switching out"of one robot for another would not affect development time. In this paper, we outline the successful creation and implementation of a wrapper class for one robot into the open-source H3DAPI, which integrates the software commands most commonly used by all robots.

Recently, a vast amount of prospects have come available for human-robot interactive systems. In this paper we outline the integration of a new robotic device with open source software that can rapidly make possible a library of interactive functionality. We then outline a clinical application for a neurorehabilitation application.

触觉/图形康复：整合成一个虚拟的环境下图书馆的机器人，它应用于中风治疗

Recent research that tests interactive devices for prolonged therapy practice has revealed new prospects for robotics combined with graphical and other ...

生物工程

Adaptation of a Haptic Robot in a 3T fMRI

Functional magnetic resonance imaging (fMRI) provides excellent functional brain imaging via the BOLD signal 1 with advantages including non-ionizing radiation, millimeter spatial accuracy of anatomical and functional data 2, and nearly real-time analyses 3. Haptic robots provide precise measurement and control of position and force of a cursor in a reasonably confined space. Here we combine these two technologies to allow precision experiments involving motor control with haptic/tactile environment interaction such as reaching or grasping. The basic idea is to attach an 8 foot end effecter supported in the center to the robot 4 allowing the subject to use the robot, but shielding it and keeping it out of the most extreme part of the magnetic field from the fMRI machine (Figure 1).
The Phantom Premium 3.0, 6DoF, high-force robot (SensAble Technologies, Inc.) is an excellent choice for providing force-feedback in virtual reality experiments 5, 6, but it is inherently non-MR safe, introduces significant noise to the sensitive fMRI equipment, and its electric motors may be affected by the fMRI's strongly varying magnetic field. We have constructed a table and shielding system that allows the robot to be safely introduced into the fMRI environment and limits both the degradation of the fMRI signal by the electrically noisy motors and the degradation of the electric motor performance by the strongly varying magnetic field of the fMRI. With the shield, the signal to noise ratio (SNR: mean signal/noise standard deviation) of the fMRI goes from a baseline of &tilde;380 to &tilde;330, and &tilde;250 without the shielding. The remaining noise appears to be uncorrelated and does not add artifacts to the fMRI of a test sphere (Figure 2). The long, stiff handle allows placement of the robot out of range of the most strongly varying parts of the magnetic field so there is no significant effect of the fMRI on the robot. The effect of the handle on the robot's kinematics is minimal since it is lightweight (&tilde;2.6 lbs) but extremely stiff 3/4" graphite and well balanced on the 3DoF joint in the middle. The end result is an fMRI compatible, haptic system with about 1 cubic foot of working space, and, when combined with virtual reality, it allows for a new set of experiments to be performed in the fMRI environment including naturalistic reaching, passive displacement of the limb and haptic perception, adaptation learning in varying force fields, or texture identification 5, 6.

The adaptation and use of a haptic robot in a 3T fMRI is described.

触觉机器人适应在3T功能磁共振成像

Functional magnetic resonance imaging (fMRI) provides excellent functional brain imaging via the BOLD signal 1 with advantages including non-ionizing ...

Functional Near Infrared Spectroscopy of the Sensory and Motor Brain Regions with Simultaneous Kinematic and EMG Monitoring During Motor Tasks

There are several advantages that functional near-infrared spectroscopy (fNIRS) presents in the study of the neural control of human movement. It is relatively flexible with respect to participant positioning and allows for some head movements during tasks. Additionally, it is inexpensive, light weight, and portable, with very few contraindications to its use. This presents a unique opportunity to study functional brain activity during motor tasks in individuals who are typically developing, as well as those with movement disorders, such as cerebral palsy. An additional consideration when studying movement disorders, however, is the quality of actual movements performed and the potential for additional, unintended movements. Therefore, concurrent monitoring of both blood flow changes in the brain and actual movements of the body during testing is required for appropriate interpretation of fNIRS results. Here, we show a protocol for the combination of fNIRS with muscle and kinematic monitoring during motor tasks. We explore gait, a unilateral multi-joint movement (cycling), and two unilateral single-joint movements (isolated ankle dorsiflexion, and isolated hand squeezing). The techniques presented can be useful in studying both typical and atypical motor control, and can be modified to investigate a broad range of tasks and scientific questions.

Monitoring brain activity during upright motor tasks is of great value when investigating the neural source of movement disorders. Here, we demonstrate a protocol that combines functional near infrared spectroscopy with continuous monitoring of muscle and kinematic activity during 4 types of motor tasks.

感觉和运动脑区与运动同步和肌电图监测功能近红外光谱在运动任务

There are several advantages that functional near-infrared spectroscopy (fNIRS) presents in the study of the neural control of human movement. It is ...

行为学

Multifunctional Setup for Studying Human Motor Control Using Transcranial Magnetic Stimulation, Electromyography, Motion Capture, and Virtual Reality

The study of neuromuscular control of movement in humans is accomplished with numerous technologies. Non-invasive methods for investigating neuromuscular function include transcranial magnetic stimulation, electromyography, and three-dimensional motion capture. The advent of readily available and cost-effective virtual reality solutions has expanded the capabilities of researchers in recreating &#8220;real-world&#8221; environments and movements in a laboratory setting. Naturalistic movement analysis will not only garner a greater understanding of motor control in healthy individuals, but also permit the design of experiments and rehabilitation strategies that target specific motor impairments (e.g. stroke). The combined use of these tools will lead to increasingly deeper understanding of neural mechanisms of motor control. A key requirement when combining these data acquisition systems is fine temporal correspondence between the various data streams. This protocol describes a multifunctional system&#8217;s overall connectivity, intersystem signaling, and the temporal synchronization of recorded data. Synchronization of the component systems is primarily accomplished through the use of a customizable circuit, readily made with off the shelf components and minimal electronics assembly skills.

Transcranial magnetic stimulation, electromyography, and 3D motion capture are commonly used non-invasive techniques for investigating neuromuscular function in humans. In this paper, we describe a protocol that synchronously samples data generated by all three of these tools along with the unique addition of virtual reality stimulus presentation and feedback.

多功能设置研究人类电机控制用经颅磁刺激，肌电图，运动捕捉和虚拟现实

The study of neuromuscular control of movement in humans is accomplished with numerous technologies. Non-invasive methods for investigating neuromuscular ...

Designing and Implementing Nervous System Simulations on LEGO Robots

We present a method to use the commercially available LEGO Mindstorms NXT robotics platform to test systems level neuroscience hypotheses. The first step of the method is to develop a nervous system simulation of specific reflexive behaviors of an appropriate model organism; here we use the American Lobster. Exteroceptive reflexes mediated by decussating (crossing) neural connections can explain an animal's taxis towards or away from a stimulus as described by Braitenberg and are particularly well suited for investigation using the NXT platform.1 The nervous system simulation is programmed using LabVIEW software on the LEGO Mindstorms platform. Once the nervous system is tuned properly, behavioral experiments are run on the robot and on the animal under identical environmental conditions. By controlling the sensory milieu experienced by the specimens, differences in behavioral outputs can be observed. These differences may point to specific deficiencies in the nervous system model and serve to inform the iteration of the model for the particular behavior under study. This method allows for the experimental manipulation of electronic nervous systems and serves as a way to explore neuroscience hypotheses specifically regarding the neurophysiological basis of simple innate reflexive behaviors. The LEGO Mindstorms NXT kit provides an affordable and efficient platform on which to test preliminary biomimetic robot control schemes. The approach is also well suited for the high school classroom to serve as the foundation for a hands-on inquiry-based biorobotics curriculum.

An approach to neural network modeling on the LEGO Mindstorms robotics platform is presented. The method provides a simulation tool for invertebrate neuroscience research in both the research lab and the classroom. This technique enables the investigation of biomimetic robot control principles.

LEGO机器人神经系统仿真设计和实现

We present a method to use the commercially available LEGO Mindstorms NXT robotics platform to test systems level neuroscience hypotheses. The first step ...

Operative Technique and Nuances for the Stereoelectroencephalographic (SEEG) Methodology Utilizing a Robotic Stereotactic Guidance System

The SEEG methodology has gained favor in North America over the last decade as a means of localizing the epileptogenic zone (EZ) prior to epilepsy surgery. Recently, the application of a robotic stereotactic guidance system for implantation of SEEG electrodes has become more popular in many epilepsy centers. The technique for the use of the robot requires extreme precision in the pre-surgical planning phase and then the technique is streamlined during the operative portion of the methodology, as the robot and surgeon work in concert to implant the electrodes. Herein is detailed precise operative methodology of using the robot to guide implantation of SEEG electrodes. A major limitation of the procedure, namely its heavy reliance on the ability to register the patient to a preoperative volumetric magnetic resonance image (MRI), is also discussed. Overall, this procedure has been shown to have a low morbidity rate and an extremely low mortality rate. The use of a robotic stereotactic guidance system for the implantation of SEEG electrodes is an efficient, fast, safe, and accurate alternative to conventional manual implantation strategies.

Operative Technique and Nuances for the Stereoelectroencephalographic SEEG Methodology Utilizing a Robotic Stereotactic Guidance System

The SEEG methodology is simplified and made faster with a stereotactic robot. Careful attention must be paid to the registration of the preoperative volumetric MRI to the patient prior to use of the robot in the OR. The robot streamlines the procedure, leading to decreased operative times and accurate implantations.

利用机器人立体定向引导系统的立体脑电图（SEEG）方法的操作技术和细微差别

The SEEG methodology has gained favor in North America over the last decade as a means of localizing the epileptogenic zone (EZ) prior to epilepsy ...

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks

Information coding in the Central Nervous System (CNS) remains unexplored. There is mounting evidence that, even at a very low level, the representation of a given stimulus might be dependent on context and history. If this is actually the case, bi-directional interactions between the brain (or if need be a reduced model of it) and sensory-motor system can shed a light on how encoding and decoding of information is performed. Here an experimental system is introduced and described in which the activity of a neuronal element (i.e., a network of neurons extracted from embryonic mammalian hippocampi) is given context and used to control the movement of an artificial agent, while environmental information is fed back to the culture as a sequence of electrical stimuli. This architecture allows a quick selection of diverse encoding, decoding, and learning algorithms to test different hypotheses on the computational properties of neuronal networks.

In this paper, an experimental framework to perform closed-loop experiments is presented, in which information processing (i.e., coding and decoding) and learning of neuronal assemblies are studied during the continuous interaction with a robotic body.

闭环神经机器人实验来检验神经网络的计算性能

Information coding in the Central Nervous System (CNS) remains unexplored. There is mounting evidence that, even at a very low level, the representation ...

在脑机接口系统中使用 VR 数字化身的运动图像

Motor Imagery Performance Through Embodied Digital Twins in a Virtual Reality-Enabled Brain-Computer Interface Environment

This study introduces an innovative framework for neurological rehabilitation by integrating brain-computer interfaces (BCI) and virtual reality (VR) technologies with the customization of three-dimensional (3D) avatars. Traditional approaches to rehabilitation often fail to fully engage patients, primarily due to their inability to provide a deeply immersive and interactive experience. This research endeavors to fill this gap by utilizing motor imagery (MI) techniques, where participants visualize physical movements without actual execution. This method capitalizes on the brain's neural mechanisms, activating areas involved in movement execution when imagining movements, thereby facilitating the recovery process. The integration of VR's immersive capabilities with the precision of electroencephalography (EEG) to capture and interpret brain activity associated with imagined movements forms the core of this system. Digital Twins in the form of personalized 3D avatars are employed to significantly enhance the sense of immersion within the virtual environment. This heightened sense of embodiment is crucial for effective rehabilitation, aiming to bolster the connection between the patient and their virtual counterpart. By doing so, the system not only aims to improve motor imagery performance but also seeks to provide a more engaging and efficacious rehabilitation experience. Through the real-time application of BCI, the system allows for the direct translation of imagined movements into virtual actions performed by the 3D avatar, offering immediate feedback to the user. This feedback loop is essential for reinforcing the neural pathways involved in motor control and recovery. The ultimate goal of the developed system is to significantly enhance the effectiveness of motor imagery exercises by making them more interactive and responsive to the user's cognitive processes, thereby paving a new path in the field of neurological rehabilitation.

作者聚焦：通过 EEG、运动图像和虚拟现实增强神经康复

Motor imagery in a virtual reality environment has wide applications in brain-computer interface systems. This manuscript outlines the use of personalized digital avatars that resemble the participants performing movements imagined by the participant in a virtual reality environment to enhance immersion and a sense of body ownership.

在支持虚拟现实的脑机接口环境中通过具身数字孪生实现的运动图像性能

This study introduces an innovative framework for neurological rehabilitation by integrating brain-computer interfaces (BCI) and virtual reality (VR) ...

SSVEP基础实验步骤与人形机器人脑机器人互动

摘要

探索更多视频

研究脑机接口的闭环性能的实验平台

触觉/图形康复：整合成一个虚拟的环境下图书馆的机器人，它应用于中风治疗

触觉机器人适应在3T功能磁共振成像

感觉和运动脑区与运动同步和肌电图监测功能近红外光谱在运动任务

多功能设置研究人类电机控制用经颅磁刺激，肌电图，运动捕捉和虚拟现实

LEGO机器人神经系统仿真设计和实现

利用机器人立体定向引导系统的立体脑电图（SEEG）方法的操作技术和细微差别

闭环神经机器人实验来检验神经网络的计算性能

在支持虚拟现实的脑机接口环境中通过具身数字孪生实现的运动图像性能

在支持虚拟现实的脑机接口环境中通过具身数字孪生实现的运动图像性能