The authors would like to express their gratitude to Mr. Hong Hu for his assistance in performing the experiments reported in this paper.&#160;&#160;This work was supported in part by the National Natural Science Foundation of China (No. 61473207).

1. Acquiring EEG Signals
<ol>
	<li>Explain the experimental procedure to the subject and obtain written informed consent to participate in experiments.</li>
	<li>Measure the circumference of the subject&#39;s head using a tape measure and select the EEG cap size that is close to the measurement. The electrodes arrangement is based on the &#34;International 10-20 System&#34; 12.</li>
	<li>Measure the distance between the nasion and inion. Use a skin marker pencil to mark 10% of the distance as a reference for aligning the cap, and mark the midpoint of the distance as the vertex on the subject&#39;s scalp.</li>
	<li>Place the EEG cap on the subject by aligning the 10% mark with the midpoint of the FP1 and FP2 electrodes. Position electrode Cz of the cap on the vertex.</li>
	<li>Make sure that the Fz, Cz, Pz and Oz are on the midline of the head, and that each pair of electrodes located on either side of the midline are in a horizontal line.</li>
	<li>Smear the REF1 and REF2 electrodes with conductive gel. Place the reference electrodes on the left and right mastoids using medical tape. Tighten the chinstrap.</li>
	<li>Place a blunt-tipped syringe in the electrode holders and inject conductive gel into each electrode in the following order: first, the &#34;ground&#34; electrode on the forehead and second, the five electrodes used in the experiment, O2, O1, Oz, Pz, and Cz.</li>
	<li>Seat the subject in a comfortable chair 60 cm in front of a stimulus monitor. Instruct the subject to keep his /her eyes in the same horizontal level with the center of the monitor.</li>
	<li>Connect the electrode wires to the EEG data acquisition system. Configure the sampling rate to 1 kHz.</li>
	<li>Examine the EEG signal quality on the dedicated DPC. If there is a problem with a particular electrode, re-inject gel to adjust the impedance of the channel.</li>
	<li>Use the dedicated SPC to flicker four robot images at frequencies of 4.615, 12, 15 and 20 Hz as visual stimuli on the User Interface as shown in Figure 2.</li>
</ol>
2. Offline Analyzing SSVEP Features
<ol>
	<li>Conduct 32 trials of offline training experiments for each subject and record their brain signals acquired from the EEG system throughout this process. 
	NOTE: this process is only conducted by first-time subjects of the experiment to establish their SSVEP feature vectors and to train the classifier.</li>
	<li>When a trial starts, select one stimulus as the target randomly and display a yellow arrow above it.</li>
	<li>1 sec later, flicker the four visual stimuli at different frequencies on the User Interface for 5 sec.</li>
	<li>Request the subject to focus on the selected stimulus target while keeping his/her body movement to a minimum.</li>
	<li>After each trial, give the subject 3 sec to relax, and then start the next trial.</li>
	<li>When all trials are finished, read the saved data. Extract a 3-sec data epoch between 2 sec and 5 sec after the trigger which selects a stimulus as the target in each trial.</li>
	<li>Calculate the power spectrum density (PSD) of the data epochs using the Offline Data Analyzer in MATLAB scripts. Use the processing algorithm described in details below:
	<ol>
		<li>Calculate the canonical correlation analysis (CCA) coefficients of the multichannel EEG data with a reference data. CCA is a multivariable statistical method used for two sets of data to find their underlying correlation 13. Considering the 3s segment of multichannel EEG data X and reference data Y of the same length, use CCA to find the weight vectors, Wx and Wy, to maximize the correlation coefficient&#160;&#961; between x = XTWx and y = YTWy. Set the reference data to be the periodic signals at certain stimulus frequencies. 
		<img alt="Equation 1" src="/files/ftp_upload/53558/53558eq1.jpg" /> 
		where f1, f2, f3, f4 are the feature frequencies of the four visual stimuli.</li>
		<li>Spatially filter the multichannel EEG data X using the calculated CCA coefficients Wx to obtain an one-dimension filtered data x, which has the most prominent correlation with the linear combination of the reference data.</li>
		<li>Calculate the PSD of the spatially filtered data x using Fast Fourier Transform (FFT). 
		<img alt="Equation 2" src="/files/ftp_upload/53558/53558eq2.jpg" /> 
		where NFFT is the sample size of data x, and FFT(x, NFFT) returns the NFFT-point discrete Fourier transform of x.</li>
		<li>Normalize the PSD with respect to its mean value between 3 and 30 Hz. 
		<img alt="Equation 3" src="/files/ftp_upload/53558/53558eq3.jpg" /> 
		where&#160;<img alt="Equation 3 Denominator" src="/files/ftp_upload/53558/53558eq3denominator.jpg" /> denotes the average of the power spectrum between 3 and 30Hz.</li>
	</ol>
	</li>
	<li>Calibrate the classification parameters for the four stimulus frequencies. First, observe the normalized PSD spectrum that corresponds to its stimulus target, and manually configure the frequency with the strongest PSD amplitude as the feature frequency. Based on the feature frequency, divide the normalized PSD data into two sets: one is acquired when the corresponding stimulus is selected as the target, and the other is acquired when the non-target stimuli are selected. Calculate the mean value of each set respectively. Determine the threshold by the median of the two mean values to classify the target and non-target sets 14.</li>
</ol>
3. Online Processing Brain Signals
<ol>
	<li>Open the Online Signal Processor as shown in Figure 3, which is developed based on the OpenViBE environment and the MATLAB scripts, on the DPC for online processing brain signals.</li>
	<li>Double click the Acquire and Process Signal box, which is marked with a red cycle in Figure 3, to open the OpenViBE setting window. Configure the classification parameters for the subject: set the sampling rate as 1 kHz, set the time length for FFT as 3 sec, and set the feature frequencies according to the results of offline analysis, e.g., 4.667, 24, 15 and 20 Hz.</li>
	<li>Click the Starts button to run the Online Signal Processor, which processes the real-time data using the following algorithm in three steps.</li>
	<li>First, the algorithm acquires EEG data from channels Oz, O1, O2, Pz and CPz of the EEG system every 0.5s, and extracts the data segment of the last 3s for online processing.</li>
	<li>Second, the algorithm processes the 3-sec data segment using the algorithms described in step 2.7, and calculates the real-time PSD for classification.</li>
	<li>Third, the algorithm classifies the brainwave patterns according to the PSD amplitudes at the four feature frequencies. When the amplitude of one feature frequency is above a given threshold, classify the stimulus flickering at the corresponding frequency as the SSVEP target.</li>
</ol>
4. Connecting the Humanoid Robot
<ol>
	<li>Press the Chest button of the humanoid robot and wait for it to start up. 
	NOTE: this process takes approximately 1 min and is completed when the robot says &#34;OGNAK GNOUK&#34; while its Chest button becomes white.</li>
	<li>Establish its Wireless Fidelity (WiFi) connection to the DPC 15.</li>
	<li>Open the Robot Controller programmed in Visual C++ scripts on the DPC. The controller receives the classification results from the Online Signal Processor via Virtual-Reality Peripheral Network (VRPN) interface, translates them into the robot commands, and controls the corresponding behaviors of the humanoid robot via wireless connection.</li>
	<li>Configure the Robot Controller by inputting the IP address of the robot according to its WiFi connection, defining three sets of robot behaviors which can be easily switched to perform the three different tasks, and setting the execution parameters of these behaviors (e.g., walking speed or distance) to fit the task requirement. 
	NOTE: more details of these behaviors are described in steps 5.2.1, 5.3.1, and 5.4.1.</li>
	<li>Click the Build button of the Visual Studio platform to run the Robot Controller.</li>
	<li>Open the Choregraphe program on the SPC. Click &#34;Connect to&#34; button to pop up a Connection widget. Select the robot to be controlled according to its IP address, and double click its icon to connect to it.</li>
	<li>Open the video monitor from the view menu in the Choregraphe menu bar, and move the monitor to the front of the User Interface to display live video feedback from the robot&#39;s camera.</li>
</ol>
5. Conducting Closed-loop Control Experiments
<ol>
	<li>Request the subject to perform three specific closed-loop control tasks within different environments in order to evaluate the brain-robot interaction performance. 
	NOTE: these tasks, which are important in robotic applications because they are helpful in assisting the disabled and elderly in their daily lives, need to activate different robot behaviors.</li>
	<li>Conduct the first closed-loop control task to telepresence control the humanoid robot via brain signals to walk through obstacles and push a light switch to turn on the light, as shown in Figure 4.
	<ol>
		<li>To accomplish the first task, control the robot behaviors which are encoded by the four SSVEP stimuli as: stepping forward with a fixed distance of 0.15 m, turning left with a fixed radian of 0.3 rad, turning right with a fixed radian of 0.3 rad, and pushing the switch using its right hand. 
		NOTE: the configuration process is described in step 4.4.</li>
		<li>Inform the subject of the task objective and the four behaviors to be controlled.</li>
		<li>Run the Robot Controller as in steps 4.5-4.7, and start the experiment. Give the subject the freedom to perceive the environment and make decisions based on live video feedback, and to activate the robot behaviors by staring at the corresponding stimuli. 
		NOTE: no instruction or help should be given to the subject unless in emergency situations, e.g., the robot is about to collide with an obstacle.</li>
	</ol>
	</li>
	<li>Conduct the second task to telepresence control the humanoid robot via brain signals to walk toward the staircase following the exit sign, as shown in Figure 5. When confronting passersby, ask the subject to control the robot to say &#34;excuse me&#34; and wait for the passersby to give way.
	<ol>
		<li>Control the robot behaviors encoded by the four SSVEP stimuli as: walking forward continuously at a speed of 0.05 m/sec, turning left continuously at a speed of 0.1 rad/sec, turning right continuously at a speed of 0.1 rad/sec, and stopping all the walking behaviors.</li>
		<li>Switch the Robot Controller to the posture-dependent control function 16. 
		NOTE: when the robot is walking, this function uses the fourth stimulus as the command of stopping the walking behaviors; when the robot is not walking, it uses the fourth stimulus to control the robot to say &#34;excuse me&#34;. The function is implemented in C++ scripts by detecting the robot&#39;s walking status.</li>
		<li>Inform the subject of the task objective and the five behaviors to be controlled.</li>
		<li>Run the Robot Controller and start the experiment, as described in step 5.2.3.</li>
	</ol>
	</li>
	<li>Conduct the third task to telepresence control the humanoid robot to walk toward a balloon target, to pick it up, and to deliver it to the subject&#39;s hand, as shown in Figure 6.
	<ol>
		<li>Control the robot behaviors encoded by the four SSVEP stimuli as: stepping forward with a fixed distance of 0.15 m, turning left with a fixed radian of 0.3 rad, turning right with a fixed radian of 0.3 rad, and picking up objects.</li>
		<li>Activate the posture-dependent control function in the Robot Controller to reuse the fourth stimulus as the command of putting down the object when it has already been picked up.</li>
		<li>Inform the subject of the task objective and the five behaviors to be controlled.</li>
		<li>Run the Robot Controller and start the experiment, as described in step 5.2.3.</li>
	</ol>
	</li>
</ol>

<ol>
	<li>McFarland, D. J., Wolpaw, J. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Brain-Computer+Interface+Operation+of+Robotic+and+Prosthetic+Devices.">Brain-Computer Interface Operation of Robotic and Prosthetic Devices.</a> Computer. 41, 52-56 (2008).</li><li>Lebedev, M. A., Nicolelis, M. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Brain-machine+interfaces:+Past,+present+and+future.">Brain-machine interfaces: Past, present and future.</a> Trends Neruosci. 29 (9), 536-546 (2006).</li><li>Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., Vaughan, T. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Brain-computer+interfaces+for+communication+and+control.">Brain-computer interfaces for communication and control.</a> Clin. Neurophysiol. 113, 767-791 (2002).</li><li>Bell, C. J., Shenoy, P., Chalodhorn, R., Rao, R. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Control+of+a+humanoid+robot+by+a+noninvasive+brain-computer+interface+in+humans.">Control of a humanoid robot by a noninvasive brain-computer interface in humans.</a> J. Neural. Eng. 5, 214-220 (2008).</li><li>Li, W., Li, M., Zhao, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Control+of+humanoid+robot+via+motion-onset+visual+evoked+potentials.">Control of humanoid robot via motion-onset visual evoked potentials.</a> Front. Syst. Neurosci. 8, 247 (2014).</li><li>Regan, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Some+characteristics+of+average+steady-state+and+transient+responses+evoked+by+modulated+light.">Some characteristics of average steady-state and transient responses evoked by modulated light.</a> Electroencephalogr. Clin. Neurophysiol. 20, 238-248 (1966).</li><li>Vialatte, F. B., Maurice, M., Dauwels, J., Cichocki, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Steady-state+visually+evoked+potentials:+focus+on+essential+paradigms+and+future+perspectives.">Steady-state visually evoked potentials: focus on essential paradigms and future perspectives.</a> Prog. Neurobiol. 90, 418-438 (2010).</li><li>Bin, G., Gao, X., Wang, Y., Li, Y., Hong, B., Gao, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+high-speed+BCI+based+on+code+modulation+VEP.">A high-speed BCI based on code modulation VEP.</a> J. Neural. Eng. 8, 025015 (2011).</li><li>Sutton, S., Braren, M., Zubin, J., John, E. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Evoked-potential+correlates+of+stimulus+uncertainty.">Evoked-potential correlates of stimulus uncertainty.</a> Science. 150, 1187-1188 (1965).</li><li>Pfurtscheller, G., Lopes da Silva, H. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Event-related+EEG/MEG+synchronization+and+desynchronization:+basic+principles.">Event-related EEG/MEG synchronization and desynchronization: basic principles.</a> Clin. Neurophysiol. 110, 1842-1857 (1999).</li><li>Zhao, J., Meng, Q., Li, W., Li, M., Sun, F., Chen, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=OpenViBE-based+brainwave+control+system+for+Cerebot.">OpenViBE-based brainwave control system for Cerebot.</a> Proc. IEEE International Conference on Robotics and Biomimetics. , 1169-1174 (2013).</li><li>Homan, R. W., Herman, J., Purdy, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cerebral+location+of+international+10-20+system+electrode+placement.">Cerebral location of international 10-20 system electrode placement.</a> Electroencephalogr. Clin. Neurophysiol. 66, 376-382 (1987).</li><li>Bin, G., Gao, X., Yan, Z., Hong, B., Gao, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+online+multi-channel+SSVEP-based+brain-computer+interface+using+a+canonical+correlation+analysis+method.">An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method.</a> J. Neural. Eng. 6, 046002 (2009).</li><li>Wang, Y., Wang, R., Gao, X., Hong, B., Gao, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+practical+VEP-based+brain-computer+interface.">A practical VEP-based brain-computer interface.</a> IEEE Trans. Neural Syst. Rehabil. Eng. 14, 234-239 (2006).</li><li>Chae, Y., Jeong, J., Jo, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Toward+brain-actuated+humanoid+robots:+asynchronous+direct+control+using+an+EEG-based+BCI.">Toward brain-actuated humanoid robots: asynchronous direct control using an EEG-based BCI.</a> IEEE T. Robot. 28, 1131-1144 (2012).</li><li>Croft, R. J., Barry, R. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Removal+of+ocular+artifact+from+the+EEG:+a+review.">Removal of ocular artifact from the EEG: a review.</a> Neurophysiol. Clin. 30, 5-19 (2000).</li><li>Hwang, H. J., Hwan Kim, D., Han, C. H., Im, C. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+new+dual-frequency+stimulation+method+to+increase+the+number+of+visual+stimuli+for+multi-class+SSVEP-based+brain-computer+interface+(BCI).">A new dual-frequency stimulation method to increase the number of visual stimuli for multi-class SSVEP-based brain-computer interface (BCI).</a> Brain Res. 1515, 66-77 (2013).</li><li>Wang, M., Daly, I., Allison, B., Jin, J., Zhang, Y., Chen, L., Wang, X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+new+hybrid+BCI+paradigm+based+on+P300+and+SSVEP.">A new hybrid BCI paradigm based on P300 and SSVEP.</a> J Neurosci Methods. 244, 16-25 (2015).</li><li>Allison, B. Z., Jin, J., Zhang, Y., Wang, X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+four-choice+hybrid+P300+SSVEP+BCI+for+improved+accuracy.">A four-choice hybrid P300 SSVEP BCI for improved accuracy.</a> Brain-Computer Interfaces. 1, 17-26 (2014).</li><li>Pan, J. H., Xie, Q., Herman, Y., Wang, F., Di, H., Laureys, S., Yu, R., Li, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Detecting+awareness+in+patients+with+disorders+of+consciousness+using+a+hybrid+brain-computer+interface.">Detecting awareness in patients with disorders of consciousness using a hybrid brain-computer interface.</a> J. Neural. Eng. 11, 056007 (2014).</li><li>Li, J., Ji, H., Cao, L., Zhang, D., Gu, R., Xia, B., Wu, Q. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Evaluation+and+application+of+a+hybrid+brain+computer+interface+for+real+wheelchair+parallel+control+with+multi-degree+of+freedom.">Evaluation and application of a hybrid brain computer interface for real wheelchair parallel control with multi-degree of freedom.</a> Int J Neural Syst. 24, 1450014-14 (2014).</li><li>Zhao, J., Meng, Q., Li, W., Li, M., Chen, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=SSVEP-based+hierarchical+architecture+for+control+of+a+humanoid+robot+with+mind.">SSVEP-based hierarchical architecture for control of a humanoid robot with mind.</a> Proc. 11th World Congress on Intelligent Control and Automation. , 2401-2406 (2014).</li><li>Zhu, D., Bieger, J., Molina, G. G., Aarts, R. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+survey+of+stimulation+methods+used+in+SSVEP-based+BCIs.">A survey of stimulation methods used in SSVEP-based BCIs.</a> Comput Intell Neurosci. 2010, 1-12 (2010).</li><li>Muller-Putz, G. R., Scherer, R., Brauneis, C., Pfurtscheller, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Steady-state+visual+evoked+potential+(SSVEP)-based+communication:+impact+of+harmonic+frequency+components.">Steady-state visual evoked potential (SSVEP)-based communication: impact of harmonic frequency components.</a> J Neural Eng. 2, 123-130 (2005).</li></ol>

The authors have nothing to disclose.

This paper presents an SSVEP-based experimental procedure to establish the brain-robot interaction system with humanoid robots by integrating multiple software programs. Because human intent is perceived by interpreting real-time EEG signals, it is critical to verify the electrode connections and EEG signal qualities before conducting the experiment. If signals acquired from all the electrodes are of poor qualities, it is necessary to check the connection of the ground and reference electrodes first. If there are problem...

Brain-Robot Interaction (BRI), which provides an innovative communication pathway between human and a robotic device via brain signals, is prospective in helping the disabled in their daily lives 1,2. A variety of methods are able to acquire brain signals either invasively or non-invasively, such as electrocorticography (ECoG), electroencephalograph (EEG), functional magnetic resonance imaging (fMRI), etc. The most commonly used non-invasive method for building the BRI system is to acquire EEG signals from electrodes placed on the scalp. This method is inexpensive, easy to use, and provides an acceptable temporal resolution 3. Among a variety of robotic devices, humanoid robots are advanced as they are created to imitate some of the same physical and mental tasks that humans undergo daily. BRI with a humanoid robot will play an important role in assisting the sick and elderly, as well as performing unsanitary or dangerous jobs. But control of a humanoid robot through BRI system is highly challenging, as the humanoid robot with full body movement is developed to perform complex tasks such as personal assistance 4, 5.
Steady-State Visual Evoked Potential (SSVEP) is a type of brain signal evoked by the modulation of visual stimulus at a given frequency 6. It contains sinusoids at the fundamental and harmonic frequencies of the flickering stimulus, and prominently appears throughout the visual cortex in the occipital region of the scalp 7. The reason for choosing the SSVEP signals is that the SSVEP-based BRI system yields relatively high information transfer rate and requires less training 8. Other types of brainwaves, such as event-related potentials (ERPs) 9 or motor-imagery (MI) potentials 10, can also be embedded into this experimental procedure.
Our procedure for brain-robot interaction with humanoid robots is based on Cerebot - a mind-controlled humanoid robot platform - consisting of an EEG data acquisition system and a humanoid robot 11. The EEG system is able to record, pre-process and display bio-potential signals acquired by various types of electrodes. It provides multiple analog I/Os and digital I/Os and is capable of recording up to 128 signal channels simultaneously at a sampling rate of 30 kHz with 16-bits resolution. Its software development kits in C++ and MATLAB are easy for users to design the experimental procedures. The humanoid robot has 25 degrees of freedom and is equipped with multiple sensors, including 2 cameras, 4 microphones, 2 sonar rangefinders, 2 IR emitters and receivers, 1 inertial board, 9 tactile sensors, and 8 pressure sensors. It provides Choregraphe and C++ SDK for creating and editing movements and interactive robot behaviors.
The overall goal of this method is to establish an SSVEP-based experimental procedure by integrating multiple software programs, such as OpenViBE, Choregraph, Central software as well as user developed programs written in C++ and MATLAB, to enable the study of brain-robot interaction with humanoid robots 11. Figure 1 shows the system structure. The dedicated stimulus presentation computer (SPC) displays the User Interface to provide the subject with visual stimuli, instructions and environmental feedbacks. The dedicated data processing computer (DPC) runs the Data Recorder and the Offline Data Analyzer in the offline training process, and runs the Online Signal Processor and the Robot Controller for the online control of the humanoid robot. Compared with other SSVEP-based control systems, our system is more reliable, more flexible, and especially more convenient to be reused and upgraded as it is developed by integrating a number of standardized software packages, such as OpenViBE, Choregraph, Central software, and modules written in C++ and MATLAB.
The following procedure was reviewed and approved by Tianjin medical university general hospital ethics committee, and all subjects gave written consent.

<table border="0" cellpadding="0" cellspacing="0" width="748">
	<tbody>
		<tr height="21">
			<td height="21" style="height:21px;width:284px;">Cerebus EEG Data Acquisition System</td>
			<td style="width:179px;">Blackrock Microsystems</td>
			<td style="width:119px;">4176-9967</td>
			<td style="width:167px;"></td>
		</tr>
		<tr height="21">
			<td height="21" style="height:21px;width:284px;">NAO humanoid robot</td>
			<td style="width:179px;">Aldebaran Robotics</td>
			<td style="width:119px;">H25</td>
			<td></td>
		</tr>
		<tr height="21">
			<td height="21" style="height:21px;width:284px;">EEG cap</td>
			<td style="width:179px;">Neuroscan</td>
			<td style="width:119px;">8732</td>
			<td></td>
		</tr>
		<tr height="21">
			<td height="21" style="height:21px;width:284px;">Ten20 Conductive gel</td>
			<td style="width:179px;">Weaver and company</td>
			<td style="width:119px;">10-20-8</td>
			<td></td>
		</tr>
	</tbody>
</table>

ssvep-based experimental procedure for brain-robot interaction with humanoid robots

Brain-Robot Interaction (BRI), which provides an innovative communication pathway between human and a robotic device via brain signals, is prospective in helping the disabled in their daily lives. The overall goal of our method is to establish an SSVEP-based experimental procedure by integrating multiple software programs, such as OpenViBE, Choregraph, and Central software as well as user developed programs written in C++ and MATLAB, to enable the study of brain-robot interaction with humanoid robots.
This is achieved by first placing EEG electrodes on a human subject to measure the brain responses through an EEG data acquisition system. A user interface is used to elicit SSVEP responses and to display video feedback in the closed-loop control experiments. The second step is to record the EEG signals of first-time subjects, to analyze their SSVEP features offline, and to train the classifier for each subject. Next, the Online Signal Processor and the Robot Controller are configured for the online control of a humanoid robot. As the final step, the subject completes three specific closed-loop control experiments within different environments to evaluate the brain-robot interaction performance.
The advantage of this approach is its reliability and flexibility because it is developed by integrating multiple software programs. The results show that using this approach, the subject is capable of interacting with the humanoid robot via brain signals. This allows the mind-controlled humanoid robot to perform typical tasks that are popular in robotic research and are helpful in assisting the disabled.

The results presented here were obtained from a male subject having corrected-to-normal version. Figure 7 shows the procedure of processing EEG data, including extracting a multichannel data epoch (Figure 7A), spatially filtering the data using CCA coefficients (Figure 7B), and calculating the normalized PSD (Figure 7C).
Figure 8 shows the norma...

Watch this Scientific Journal Video about SSVEP-based Experimental Procedure for Brain-Robot Interaction with Humanoid Robots at JoVE.com

SSVEP-based Experimental Procedure for Brain-Robot Interaction with Humanoid Robots

The overall goal of this method is to establish an SSVEP-based experimental procedure by integrating multiple software programs to enable the study of brain-robot interaction with humanoid robots, which is prospective in assisting the sick and elderly as well as performing unsanitary or dangerous jobs.

Brain-Robot Interaction (BRI), which provides an innovative communication pathway between human and a robotic device via brain signals, is prospective in ...

ssvep-based-experimental-procedure-for-brain-robot-interaction-with

Research

Education

JoVE Core

JoVE Journal

Electrical Engineering

Engineering

Unit 1

Introduction to Control Systems

SCIENTISTS IN ACTION

13.0K Views.  Tianjin University. The overall goal of this method is to establish an SSVEP-based experimental procedure by integrating multiple software programs to enable the study of brain-robot interaction with humanoid robots, which is prospective in assisting the sick and elderly as well as performing unsanitary or dangerous jobs.

Video: SSVEP-based Experimental Procedure for Brain-Robot Interaction with Humanoid Robots

Advanced Experimental Methods for Low-temperature Magnetotransport Measurement of Novel Materials

Novel electronic materials are often produced for the first time by synthesis processes that yield bulk crystals (in contrast to single crystal thin film synthesis) for the purpose of exploratory materials research. Certain materials pose a challenge wherein the traditional bulk Hall bar device fabrication method is insufficient to produce a measureable device for sample transport measurement, principally because the single crystal size is too small to attach wire leads to the sample in a Hall bar configuration. This can be, for example, because the first batch of a new material synthesized yields very small single crystals or because flakes of samples of one to very few monolayers are desired. In order to enable rapid characterization of materials that may be carried out in parallel with improvements to their growth methodology, a method of device fabrication for very small samples has been devised to permit the characterization of novel materials as soon as a preliminary batch has been produced. A slight variation of this methodology is applicable to producing devices using exfoliated samples of two-dimensional materials such as graphene, hexagonal boron nitride (hBN), and transition metal dichalcogenides (TMDs), as well as multilayer heterostructures of such materials. Here we present detailed protocols for the experimental device fabrication of fragments and flakes of novel materials with micron-sized dimensions onto substrate and subsequent measurement in a commercial superconducting magnet, dry helium close-cycle cryostat magnetotransport system at temperatures down to 0.300 K and magnetic fields up to 12 T.

We describe the methodology of mechanical exfoliation and deposition of flakes of novel materials with micron-sized dimensions onto substrate, fabrication of experimental device structures for transport experimentation, and the magnetotransport measurement in a dry helium close-cycle cryostat at temperatures down to 0.300 K and magnetic fields up to 12 T.

Novel electronic materials are often produced for the first time by synthesis processes that yield bulk crystals (in contrast to single crystal thin film ...

Experimental Methods for Investigation of Shape Memory Based Elastocaloric Cooling Processes and Model Validation

Shape Memory Alloys (SMA) using elastocaloric cooling processes have the potential to be an environmentally friendly alternative to the conventional vapor compression based cooling process. Nickel-Titanium (Ni-Ti) based alloy systems, especially, show large elastocaloric effects. Furthermore, exhibit large latent heats which is a necessary material property for the development of an efficient solid-state based cooling process. A scientific test rig has been designed to investigate these processes and the elastocaloric effects in SMAs. The realized test rig enables independent control of an SMA&#39;s mechanical loading and unloading cycles, as well as conductive heat transfer between SMA cooling elements and a heat source/sink. The test rig is equipped with a comprehensive monitoring system capable of synchronized measurements of mechanical and thermal parameters. In addition to determining the process-dependent mechanical work, the system also enables measurement of thermal caloric aspects of the elastocaloric cooling effect through use of a high-performance infrared camera. This combination is of particular interest, because it allows illustrations of localization and rate effects &#8212; both important for efficient heat transfer from the medium to be cooled.
The work presented describes an experimental method to identify elastocaloric material properties in different materials and sample geometries. Furthermore, the test rig is used to investigate different cooling process variations. The introduced analysis methods enable a differentiated consideration of material, process and related boundary condition influences on the process efficiency. The comparison of the experimental data with the simulation results (of a thermomechanically coupled finite element model) allows for better understanding of the underlying physics of the elastocaloric effect. In addition, the experimental results, as well as the findings based on the simulation results, are used to improve the material properties.

Experimental methods for investigation of solid state cooling processes and characterization of elastocaloric material properties of Shape Memory Alloys (SMA) are presented. A custom-built test rig has been designed for controlling and comprehensive monitoring of elastocaloric cooling processes. Furthermore, it provides a validation platform for thermomechanically coupled modeling approaches.

Shape Memory Alloys (SMA) using elastocaloric cooling processes have the potential to be an environmentally friendly alternative to the conventional vapor ...

Experimental Procedure for Warm Spinning of Cast Aluminum Components

High performance, cast aluminum automotive wheels are increasingly being incrementally formed via flow forming/metal spinning at elevated temperatures to improve material properties. With a wide array of processing parameters which can affect both the shape attained and resulting material properties, this type of processing is notoriously difficult to commission. A simplified, light-duty version of the process has been designed and implemented for full-size automotive wheels. The apparatus is intended to assist in understanding the deformation mechanisms and the material response to this type of processing. An experimental protocol has been developed to prepare for, and subsequently perform forming trials and is described for as-cast A356 wheel blanks. The thermal profile attained, along with instrumentation details are provided. Similitude with full-scale forming operations which impart significantly more deformation at faster rates is discussed.

An experimental protocol for instrumented warm rotary forming of cast aluminum alloys employing a bespoke industrially scaled apparatus is presented. Experimental considerations including thermal and mechanical effects are discussed, as well as similitude with full-scale processing of automotive wheels.

High performance, cast aluminum automotive wheels are increasingly being incrementally formed via flow forming/metal spinning at elevated temperatures to ...

Experimental Methods for Trapping Ions Using Microfabricated Surface Ion Traps

Ions trapped in a quadrupole Paul trap have been considered one of the strong physical candidates to implement quantum information processing. This is due to their long coherence time and their capability to manipulate and detect individual quantum bits (qubits). In more recent years, microfabricated surface ion traps have received more attention for large-scale integrated qubit platforms. This paper presents a microfabrication methodology for ion traps using micro-electro-mechanical system (MEMS) technology, including the fabrication method for a 14 &#181;m-thick dielectric layer and metal overhang structures atop the dielectric layer. In addition, an experimental procedure for trapping ytterbium (Yb) ions of isotope 174 (174Yb+) using 369.5 nm, 399 nm, and 935 nm diode lasers is described. These methodologies and procedures involve many scientific and engineering disciplines, and this paper first presents the detailed experimental procedures. The methods discussed in this paper can easily be extended to the trapping of Yb ions of isotope 171 (171Yb+) and to the manipulation of qubits.

This paper presents a microfabrication methodology for surface ion traps, as well as a detailed experimental procedure for trapping ytterbium ions in a room-temperature environment.

Ions trapped in a quadrupole Paul trap have been considered one of the strong physical candidates to implement quantum information processing. This is due ...

Experimental Methods for Spin- and Angle-Resolved Photoemission Spectroscopy Combined with Polarization-Variable Laser

The goal of this protocol is to present how to perform spin- and angle-resolved photoemission spectroscopy combined with polarization-variable 7-eV laser (laser-SARPES), and demonstrate a power of this technique for studying solid state physics. Laser-SARPES achieves two great capabilities. Firstly, by examining orbital selection rule of linearly polarized lasers, orbital selective excitation can be carried out in SAPRES experiment. Secondly, the technique can show full information of a variation of the spin quantum axis as a function of the light polarization. To demonstrate the power of the collaboration of these capabilities in laser-SARPES, we apply this technique for the investigations of spin-orbit coupled surface states of Bi2Se3. This technique affords to decompose spin and orbital components from the spin-orbit coupled wavefunctions. Moreover, as a representative advantage of using the direct spin detection collaborated with the polarization-variable laser, the technique unambiguously visualizes the light polarization dependence of the spin quantum axis in three-dimension. Laser-SARPES dramatically increases a capability of photoemission technique.

Here, we combine polarization-variable 7-eV laser with spin- and angle-resolved photoemission technique to visualize the spin-orbital coupling effect in solid states.

The goal of this protocol is to present how to perform spin- and angle-resolved photoemission spectroscopy combined with polarization-variable 7-eV laser ...

Enhanced Electron Injection and Exciton Confinement for Pure Blue Quantum-Dot Light-Emitting Diodes by Introducing Partially Oxidized Aluminum Cathode

Stable and efficient red (R), green (G), and blue (B) light sources based on solution-processed quantum dots (QDs) play important roles in next-generation displays and solid-state lighting technologies. The brightness and efficiency of blue QDs-based light-emitting diodes (LEDs) remain inferior to their red and green counterparts, due to the inherently unfavorable energy levels of different colors of light. To solve these problems, a device structure should be designed to balance the injection holes and electrons into the emissive QD layer. Herein, through a simple autoxidation strategy, pure blue QD-LEDs which are highly bright and efficient are demonstrated, with a structure of ITO/PEDOT:PSS/Poly-TPD/QDs/Al:Al2O3. The autoxidized Al:Al2O3 cathode can effectively balance the injected charges and enhance radiative recombination without introducing an additional electron transport layer (ETL). As a result, high color-saturated blue QD-LEDs are achieved with a maximum luminance over 13,000 cd m-2, and a maximum current efficiency of 1.15 cd A-1. The easily controlled autoxidation procedure paves the way for achieving high-performance blue QD-LEDs.

A protocol is presented for fabricating high-performance, pure blue ZnCdS/ZnS-based quantum dots light-emitting diodes by employing an autoxidized aluminum cathode.

Stable and efficient red (R), green (G), and blue (B) light sources based on solution-processed quantum dots (QDs) play important roles in next-generation ...

Experimental Procedure for Laboratory Studies of In Situ Burning : Flammability and Burning Efficiency of Crude Oil

A new method for the simultaneous study of the flammability and burning efficiency of fresh and weathered crude oil through two experimental laboratory setups is presented. The experiments are easily repeatable compared to operational scale experiments (pool diameter &#8805;2 m), while still featuring quite realistic in situ burning conditions of crude oil on water. Experimental conditions include a flowing water sub-layer that cools the oil slick and an external heat flux (up to 50 kW/m2) that simulates the higher heat feedback to the fuel surface in operational scale crude oil pool fires. These conditions enable a controlled laboratory study of the burning efficiency of crude oil pool fires that are equivalent to operational scale experiments. The method also provides quantitative data on the requirements for igniting crude oils in terms of the critical heat flux, ignition delay time as a function of the incident heat flux, the surface temperature upon ignition, and the thermal inertia. This type of data can be used to determine the required strength and duration of an ignition source to ignite a certain type of fresh or weathered crude oil. The main limitation of the method is that the cooling effect of the flowing water sub-layer on the burning crude oil as a function of the external heat flux has not been fully quantified. Experimental results clearly showed that the flowing water sub-layer does improve how representative this setup is of in situ burning conditions, but to what extent this representation is accurate is currently uncertain. The method nevertheless features the most realistic in situ burning laboratory conditions currently available for simultaneously studying the flammability and burning efficiency of crude oil on water.

Experimental Procedure for Laboratory Studies of In Situ Burning : Flammability and Burning Efficiency of Crude Oil

Here, we present a protocol to simultaneously study the flammability and burning efficiency of fresh and weathered crude oil under conditions that simulate in situ burning operations on the sea.

A new method for the simultaneous study of the flammability and burning efficiency of fresh and weathered crude oil through two experimental laboratory ...

Theoretical Calculation and Experimental Verification for Dislocation Reduction in Germanium Epitaxial Layers with Semicylindrical Voids on Silicon

Reduction of threading dislocation density (TDD) in epitaxial germanium (Ge) on silicon (Si) has been one of the most important challenges for the realization of monolithically integrated photonics circuits. The present paper describes methods of theoretical calculation and experimental verification of a novel model for the reduction of TDD. The method of theoretical calculation describes the bending of threading dislocations (TDs) based on the interaction of TDs and non-planar growth surfaces of selective epitaxial growth (SEG) in terms of dislocation image force. The calculation reveals that the presence of voids on SiO2 masks help to reduce TDD. Experimental verification is described by germanium (Ge) SEG, using an ultra-high vacuum chemical vapor deposition method and TD observations of the grown Ge via etching and cross-sectional transmission electron microscope (TEM). It is strongly suggested that the TDD reduction would be due to the presence of semicylindrical voids over the SiO2 SEG masks and growth temperature. For experimental verification, epitaxial Ge layers with semicylindrical voids are formed as the result of SEG of Ge layers and their coalescence. The experimentally obtained TDDs reproduce the calculated TDDs based on the theoretical model. Cross-sectional TEM observations reveal that both the termination and generation of TDs occur at semicylindrical voids. Plan-view TEM observations reveal a unique behavior of TDs in Ge with semicylindrical voids (i.e., TDs are bent to be parallel to the SEG masks and the Si substrate).

Theoretical calculation and experimental verification are proposed for a reduction of threading dislocation (TD) density in germanium epitaxial layers with semicylindrical voids on silicon. Calculations based on the interaction of TDs and surface via image force, TD measurements, and transmission electron microscope observations of TDs are presented.

Reduction of threading dislocation density (TDD) in epitaxial germanium (Ge) on silicon (Si) has been one of the most important challenges for the ...

Procedure for the Transfer of Polymer Films Onto Porous Substrates with Minimized Defects

The fabrication of devices containing thin film composite membranes necessitates the transfer of these films onto the surfaces of arbitrary support substrates. Accomplishing this transfer in a highly controlled, mechanized, and reproducible manner can eliminate the creation of macroscale defect structures (e.g., tears, cracks, and wrinkles) within the thin film that compromise device performance and the usable area per sample. Here, we describe a general protocol for the highly controlled and mechanized transfer of a polymeric thin film onto an arbitrary porous support substrate for eventual use as a water filtration membrane device. Specifically, we fabricate a block copolymer (BCP) thin film on top of a sacrificial, water-soluble poly(acrylic acid) (PAA) layer and silicon wafer substrate. We then utilize a custom-designed, 3D-printed transfer tool and drain chamber system to deposit, lift-off, and transfer the BCP thin film onto the center of a porous anodized aluminum oxide (AAO) support disc. The transferred BCP thin film is shown to be consistently placed onto the center of the support surface due to the guidance of the meniscus formed between the water and the 3D-printed plastic drain chamber. We also compare our mechanized transfer-processed thin films to those that have been transferred by hand with the use of tweezers. Optical inspection and image analysis of the transferred thin films from the mechanized process confirm that little-to-no macroscale inhomogeneities or plastic deformations are produced, as compared to the multitude of tears and wrinkles produced from manual transfer by hand. Our results suggest that the proposed strategy for thin film transfer can reduce defects when compared to other methods across many systems and applications.

We present a procedure for highly controlled and wrinkle-free transfer of block copolymer thin films onto porous support substrates using a 3D-printed drain chamber. The drain chamber design is of general relevance to all procedures involving transfer of macromolecular films onto porous substrates, which is normally done by hand in an irreproducible fashion.

The fabrication of devices containing thin film composite membranes necessitates the transfer of these films onto the surfaces of arbitrary support ...

Rapid Manufacturing of Thin Soft Pneumatic Actuators and Robots

This protocol describes a method for rapid manufacturing of soft pneumatic actuators and robots with an ultrathin form factor using a heat press and a laser cutter machine. The method starts with the lamination of thermoplastic polyurethane (TPU) sheets using a heat press for 10 min at the temperature of ~93 &#176;C. Next, the parameters of the laser cutter machine are optimized to produce a rectangular balloon with maximum burst pressure. Using the optimized parameters, the soft actuators are laser cut/welded three times sequentially. Next, a dispensing needle is attached to the actuator, allowing it to be inflated. The effect of geometrical parameters on the deflection of the actuator are studied systematically by varying the channel width and length. Finally, the performance of the actuator is characterized using an optical camera and a fluid dispenser. Conventional fabrication methods of soft pneumatic actuators based on silicone molding are time consuming (several hours). They also result in strong but bulky actuators, which limits the actuator&#8217;s applications. Moreover, microfabrication of thin pneumatic actuators is both time-consuming and expensive. The proposed manufacturing method in the current work resolves these issues by introducing a fast, simple, and cost-effective fabrication method of ultrathin pneumatic actuators.

This protocol describes a method for rapid manufacturing of soft pneumatic actuators and robots with a thin form factor. The fabrication method starts with lamination of thermoplastic polyurethane (TPU) sheets followed by laser cutting/welding of a two-dimensional pattern to form actuators and robots.

This protocol describes a method for rapid manufacturing of soft pneumatic actuators and robots with an ultrathin form factor using a heat press and a ...