The authors would like to thank all the participants for their time and involvement.

The current study aims to investigate the feasibility of controlling a 3D avatar in real-time within a VR environment using MI signals recorded via EEG. The study focuses on enhancing immersion and the sense of body ownership by personalizing the avatar to resemble the subject closely. The protocol received approval from the Vellore Institute of Technology Review Board. Participants provided written informed consent after reviewing the study&#39;s purpose, procedures, and potential risks.
1. Experimental setup
NOTE: Make sure that the system incorporates all the components as depicted in the diagram of the experimental setup in Figure 1 (see Table of Materials for the equipment used).
<ol>
	<li>3D avatar development
	<ol>
		<li>Modelling the avatar
		<ol>
			<li>The day before data collection, collect multiple facial photographs from various angles and precise body measurements from each participant.</li>
			<li>Click on Modeling Software to open it. Immediately after opening, find the slider for Gender. Adjust this slider to match the gender of the model aiming to create.</li>
			<li>Navigate to the Modelling Tab at the top of the screen and click on the tab to access the body customization options.</li>
			<li>Use the sliders under various sections, like Torso, Arms, Legs, etc., to model the body. Focus on the following basic measurements: Height, Chest/Bust, Waist, Hips, Leg Length, and Arm Length.</li>
			<li>Click on the Pose/Animate tab and select the default skeleton for basic animations. Go to the Files menu at the top, select Export, and then choose the. mhx2 format for compatibility with the animation software. Ensure to select the option to Export with Rig to include the skeleton. Choose a destination folder, name the file, and click Export.</li>
			<li>Open the animation software. Go to File &#62; Import and select .mhx2 or the format to expect. Navigate to the saved file, select it, and import it into the software.</li>
			<li>Go to the Edit menu, select Preferences &#62; Add-ons, and ensure the appropriate plugin to build faces is enabled.</li>
			<li>In the 3D viewport, switch to the layout preset provided by the plugin or go to the plugin panel, usually located on the tool shelf on the left side.</li>
			<li>Click on Create a New Head in the plugin panel to start the head model. Use the Add photo button to import photos of the participant. Use front and side profiles for accurate modeling.</li>
			<li>Follow the prompts to align points on the photos with the corresponding points on the 3D model. The plugin will then adjust the head model to match the participant&#39;s features. Once satisfied with the likeness, finalize the head model.</li>
			<li>Manually position the head model to align with the neck of the body model. Adjust the scale and rotation of the head for a seamless fit.</li>
			<li>Use the Snap tool (Shift+Tab) to precisely align the vertices of the neck on the head with those on the body.</li>
			<li>Once aligned, join the head and body by selecting both meshes, pressing Ctrl+J to join them into a single object.</li>
			<li>Import or model a pair of bongos and position them in front of the model at an appropriate height.</li>
		</ol>
		</li>
		<li>Animating the avatar
		<ol>
			<li>Switch to Pose Mode for the rigged model. At frame 1, select all bones and insert a keyframe (use the I key) for LocRotScale to record their initial positions.</li>
			<li>Move the timeline forward to frame 30 to place the left hand to hit the bongo.</li>
			<li>Move and rotate the armature of the left hand to simulate striking the bongo. Insert a keyframe for these bones. Repeat this process to return the hand to its starting position at frame 60, inserting another keyframe to complete the action.</li>
			<li>Move the timeline to frame 90, where the right hand starts its action. Similar to the left hand, adjust the right hand&#39;s position and rotation to simulate striking the other bongo and insert a keyframe.</li>
			<li>Return the hand to its starting position and insert a keyframe to end the motion at frame 150.</li>
			<li>Scrub through the timeline to review the animation. Adjust as needed for smoother motion or better timing between the bongo hits. Save the file.</li>
		</ol>
		</li>
	</ol>
	</li>
	<li>Equipment setup
	<ol>
		<li>Assemble the 16-channel EEG data acquisition system by attaching the Daisy module with 8 EEG channels on top of the board with 8 EEG channels.</li>
		<li>Connect the reference electrode via a Y-Splitter cable to the bottom reference pin on the Daisy Board and the bottom reference pin of the board in the bottom, which are both labeled as SRB.</li>
		<li>Connect the ground electrode to the BIAS pin on the bottom board.</li>
		<li>Connect the 16 EEG electrodes to the bottom board pins labeled N1P-N8P and the daisy bottom pins labeled N1P-N8P.</li>
		<li>Insert the electrodes on the gel-free cap at the labeled locations adhering to the international 10-20 system for electrode placement with electrodes labeled as FP1, FP2, C3, C4, CZ, P3, P4, PZ, O1, O2, F7, F8, F3, F4, T3, and T4.</li>
		<li>Soak 18 sponges provided for the EEG electrodes in a saline solution with 5 g of sodium chloride mixed in 200 mL of tap water for 15 min.</li>
		<li>Insert the soaked sponges on the underside of each electrode to establish contact between the scalp and the electrode.</li>
		<li>Make the participants sit comfortably in a quiet room. Place the gel-free EEG cap on the participant&#39;s scalp, making sure the cap is aligned correctly to fit over the participant&#39;s ears.</li>
		<li>Connect the USB dongle to the laptop. Open the EEG GUI, click on the EEG system, under the Data Source option select Serial (from Dongle), 16 channels, and AUTO-CONNECT.</li>
		<li>Inside the Data Acquisition screen, select the signal widget to check the signal quality of the connected electrodes by verifying an optimal impedance level of &#60;10 k&#937; at each electrode site26.</li>
		<li>If the impedance is higher than 10 k&#937; then add a few drops of saline solution to the sponge under the electrode. After the impedance check, close the GUI.</li>
		<li>Open the Acquisition Server software, select the appropriate EEG board under Driver and click Connect &#62; Play to establish a connection with the EEG system.</li>
		<li>Prepare the VR headset by sanitizing it with wipes and placing it on the participant&#39;s head over the EEG cap to facilitate an immersive interaction while capturing EEG data.</li>
	</ol>
	</li>
	<li>Game setup 
	&#8203;NOTE: The following instructions outline the setup of two game engine scenarios using Open Sound Control (OSC): one for motor imagery training (feedforward) and another for testing motor imagery (feedback). The feedforward scenario trains users in motor imagery through observed animations triggered by OSC messages. The feedback scenario tests motor imagery efficacy by animating user-imagined movements based on OSC inputs.
	<ol>
		<li>Open the game engine software and select Motor Imagery Training Project. Enable VR Support: Go to Edit &#62; Project Settings &#62; Player &#62; XR Settings, check Virtual Reality Supported, and ensure the VR headset is listed under virtual reality SDKs.</li>
		<li>Delete the default camera and drag the VR camera into the scene from the VR integration package.</li>
		<li>Place the imported animation file in the scene. Adjust for scale and orientation as needed. Ensure the OSCListener GameObject with pre-written scripts is set to trigger model animations for left- and right-hand movements based on OSC messages, simulating the bongo-hitting action for motor imagery training.</li>
		<li>Open File &#62; Build Settings in the game engine software. Select PC, Mac &#38; Linux Standalone, target Windows, then click Build and Run.</li>
		<li>For the motor imagery testing project perform similar steps as the motor imagery training project. Use the OSCListener GameObject configured with scripts designed to receive OSC signals indicative of the participant&#39;s imagined hand movements, triggering the corresponding animations for the testing project.</li>
	</ol>
	</li>
</ol>
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/66859/66859fig01.jpg" /> 
Figure 1: VR-BCI setup. The entire VR-BCI setup shows the participant wearing the VR headset and EEG cap. The participants viewed the personalized 3D avatar in the virtual environment and controlled its action using brain signals transmitted to the computer wirelessly. <a href="https://www.jove.com/files/ftp_upload/66859/66859fig01large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
2. Experimental design
<ol>
	<li>Signal verification stage
	<ol>
		<li>Open the software tool to design and run motor imagery scenarios, go to File and load the six Motor-Imagery-BCI scenarios labeled Signal Verification, Acquisition, CSP Training, Classifier Training, Testing, and Confusion Matrix.</li>
		<li>Navigate to the signal verification scenario. Apply a band-pass filter between 1 to 40 Hz with a filter order of 4 to the raw signals using designer boxes for optimized signal processing.</li>
	</ol>
	</li>
	<li>Training stage
	<ol>
		<li>Guide and instruct participants to undergo motor imagery tasks, imagining hand movements in response to visual cues.</li>
		<li>Open the file for motor imagery training and display the prepared 3D avatar standing over a set of bongos through the VR headset.</li>
		<li>Navigate to the Acquisition scenario and double click the Graz Motor Imagery Stimulator to configure the box.</li>
		<li>Configure 50, 5 s trials (cue-1.25 s and MI-3.75 s) for both left- and right-hand movements, incorporating a 20 s baseline period followed by intervals of 10 s rest after every 10 trials to avoid mental fatigue.</li>
		<li>Configure the left- and right-hand trials to be randomized and have a cue before the trial indicating the hand to be imagined.</li>
		<li>Connect an OSC box with the IP address and port to transmit the cue for the hand to be imagined to the motor imagery training game engine program.</li>
		<li>Direct participants to imagine executing the movement of their hand along with the 3D avatar following the same pace as the avatar as it hits the bongo with the corresponding hand, following a text cue displaying which hand is to be imagined.</li>
	</ol>
	</li>
	<li>CSP and LDA training
	<ol>
		<li>Following the acquisition, run the CSP Training scenario to analyze the EEG data from the acquisition stage and compute Common Spatial Patterns (CSP), creating filters to distinguish between left and right-hand imagery.</li>
		<li>After the CSP Training, navigate to the classifier training scenario and run it to utilize Linear Discriminant Analysis (LDA) using the CSP filters for efficient task classification, preparing the system for real-time avatar control.</li>
	</ol>
	</li>
	<li>Testing stage
	<ol>
		<li>Navigate to the testing scenario for participants to control their 3D avatars in real-time using brain-computer interface (BCI) technology.</li>
		<li>Load in the appropriate boxes the classifiers trained during the previous scenario on EEG data captured while participants imagined hand movements to interpret these imagined actions in real-time.</li>
		<li>Ensure the EEG system and VR setup are operational and correctly configured as per the training stage settings.</li>
		<li>Brief participants on the testing procedure, emphasizing the need to clearly imagine hand movements (bongo hitting using left or right hand) as prompted by text cues.</li>
		<li>Similar to the training stage, conduct 20 trials for each participant, divided equally between imagining movements of the left and right hand and randomized.</li>
		<li>Connect and configure an OSC box to transmit the cue information to be displayed as text indicating which hand to be imaged in the game engine program.</li>
		<li>Connect to another OSC box to transmit the predicted value for the left-and right-hand movements for the game engine program to play the corresponding animation based on the hand imagined by the participant.</li>
		<li>Run the testing scenario. Run the Motor Imagery Testing game engine program.</li>
	</ol>
	</li>
</ol>
3. Data collection and analysis
<ol>
	<li>Continuously record EEG data and classifier outputs during the acquisition and testing stages of the experiment, with data sampled at 125 Hz.</li>
	<li>Navigate to the Confusion Matrix scenario and load the acquired EEG file in the box labeled Generic stream reader for each participant and for both the acquisition and training stages.</li>
	<li>Run the scenario to obtain the Confusion matrix to evaluate how accurately the BCI system interprets motor imagery signals.</li>
	<li>Gather feedback from participants regarding their experience with the avatar&#39;s ease of use, control capabilities, immersion level, and comfort while wearing the EEG cap and VR headset.</li>
</ol>

Motor Imagery Using VR Digital Avatars in a Brain-Computer Interface System

<ol>
	<li>Andrade, J., Cec&#237;lio, J., Sim&#245;es, M., Sales, F., Castelo-Branco, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Separability+of+motor+imagery+of+the+self+from+interpretation+of+motor+intentions+of+others+at+the+single+trial+level:+An+eeg+study.">Separability of motor imagery of the self from interpretation of motor intentions of others at the single trial level: An eeg study.</a> J. NeuroEng. Rehabil. 14 (1), 1-13 (2017).</li><li>Lorey, B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Neural+simulation+of+actions:+Effector-versus+action-specific+motor+maps+within+the+human+premotor+and+posterior+parietal+area.">Neural simulation of actions: Effector-versus action-specific motor maps within the human premotor and posterior parietal area.</a> Hum. Brain Mapp. 35 (4), 1212-1225 (2014).</li><li>Ehrsson, H. H., Geyer, S., Naito, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Imagery+of+voluntary+movement+of+fingers,+toes,+and+tongue+activates+corresponding+body-part-specific+motor+representations.">Imagery of voluntary movement of fingers, toes, and tongue activates corresponding body-part-specific motor representations.</a> J Neurophysiol. 90 (5), 3304-3316 (2003).</li><li>Sauvage, C., Jissendi, P., Seignan, S., Manto, M., Habas, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Brain+areas+involved+in+the+control+of+speed+during+a+motor+sequence+of+the+foot:+Real+movement+versus+mental+imagery.">Brain areas involved in the control of speed during a motor sequence of the foot: Real movement versus mental imagery.</a> J Neuroradiol. 40 (4), 267-280 (2013).</li><li>Pfurtscheller, G., Neuper, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Motor+imagery+activates+primary+sensorimotor+area+in+humans.">Motor imagery activates primary sensorimotor area in humans.</a> Neurosci Lett. 239 (2-3), 65-68 (1997).</li><li>Jeon, Y., Nam, C. S., Kim, Y. J., Whang, M. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Event-related+(de)synchronization+(erd/ers)+during+motor+imagery+tasks:+Implications+for+brain-computer+interfaces.">Event-related (de)synchronization (erd/ers) during motor imagery tasks: Implications for brain-computer interfaces.</a> Int. J. Ind. Ergon. 41 (5), 428-436 (2011).</li><li>Mcfarland, D. J., Miner, L. A., Vaughan, T. M., Wolpaw, J. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mu+and+beta+rhythm+topographies+during+motor+imagery+and+actual+movements.">Mu and beta rhythm topographies during motor imagery and actual movements.</a> Brain Topogr. 12 (3), 177-186 (2000).</li><li>Di Pellegrino, G., Fadiga, L., Fogassi, L., Gallese, V., Rizzolatti, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Understanding+motor+events:+A+neurophysiological+study.">Understanding motor events: A neurophysiological study.</a> Exp Brain Res. 91 (1), 176-180 (1992).</li><li>Rizzolatti, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+mirror+neuron+system+and+its+function+in+humans.">The mirror neuron system and its function in humans.</a> Anat Embryol (Berl). 210 (5-6), 419-421 (2005).</li><li>Jackson, P. L., Lafleur, M. F., Malouin, F., Richards, C. L., Doyon, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Functional+cerebral+reorganization+following+motor+sequence+learning+through+mental+practice+with+motor+imagery.">Functional cerebral reorganization following motor sequence learning through mental practice with motor imagery.</a> Neuroimage. 20 (2), 1171-1180 (2003).</li><li>Cramer, S. C., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Harnessing+neuroplasticity+for+clinical+applications.">Harnessing neuroplasticity for clinical applications.</a> Brain. 134, 1591-1609 (2011).</li><li>Nojima, I., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Human+motor+plasticity+induced+by+mirror+visual+feedback).">Human motor plasticity induced by mirror visual feedback).</a> J Neurosci. 32 (4), 1293-1300 (2012).</li><li>Slater, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Implicit learning through embodiment in immersive virtual reality. , (2017).</li><li>Tham, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Understanding+virtual+reality:+Presence,+embodiment,+and+professional+practice.">Understanding virtual reality: Presence, embodiment, and professional practice.</a> IEEE Trans. Prof. Commun. 61 (2), 178-195 (2018).</li><li>Choi, J. W., Kim, B. H., Huh, S., Jo, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Observing+actions+through+immersive+virtual+reality+enhances+motor+imagery+training.">Observing actions through immersive virtual reality enhances motor imagery training.</a> IEEE Trans. Neural Syst. Rehabil. Eng. 28 (7), 1614-1622 (2020).</li><li>Lakshminarayanan, K., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+effect+of+combining+action+observation+in+virtual+reality+with+kinesthetic+motor+imagery+on+cortical+activity.">The effect of combining action observation in virtual reality with kinesthetic motor imagery on cortical activity.</a> Front Neurosci. 17, 1201865 (2023).</li><li>Juliano, J. M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Embodiment+is+related+to+better+performance+on+a+brain-computer+interface+in+immersive+virtual+reality:+A+pilot+study.">Embodiment is related to better performance on a brain-computer interface in immersive virtual reality: A pilot study.</a> Sensors. 20 (4), 1204 (2020).</li><li>Lakshminarayanan, K., Shah, R., Yao, Y., Madathil, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+effects+of+subthreshold+vibratory+noise+on+cortical+activity+during+motor+imagery.">The effects of subthreshold vibratory noise on cortical activity during motor imagery.</a> Motor Control. 27 (3), 559-572 (2023).</li><li>Cole, S. W., Yoo, D. J., Knutson, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Interactivity+and+reward-related+neural+activation+during+a+serious+videogame.">Interactivity and reward-related neural activation during a serious videogame.</a> PLoS One. 7 (3), e33909 (2012).</li><li>Cameirao, M. S., Badia, S. B., Duarte, E., Frisoli, A., Verschure, P. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+combined+impact+of+virtual+reality+neurorehabilitation+and+its+interfaces+on+upper+extremity+functional+recovery+in+patients+with+chronic+stroke.">The combined impact of virtual reality neurorehabilitation and its interfaces on upper extremity functional recovery in patients with chronic stroke.</a> Stroke. 43 (10), 2720-2728 (2012).</li><li>Turolla, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Virtual+reality+for+the+rehabilitation+of+the+upper+limb+motor+function+after+stroke:+A+prospective+controlled+trial.">Virtual reality for the rehabilitation of the upper limb motor function after stroke: A prospective controlled trial.</a> J Neuroeng Rehabil. 10, 85 (2013).</li><li>Isaacson, B. M., Swanson, T. M., Pasquina, P. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+use+of+a+computer-assisted+rehabilitation+environment+(caren)+for+enhancing+wounded+warrior+rehabilitation+regimens.">The use of a computer-assisted rehabilitation environment (caren) for enhancing wounded warrior rehabilitation regimens.</a> J Spinal Cord Med. 36 (4), 296-299 (2013).</li><li>Alchalabi, B., Faubert, J., Labbe, D. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=EEG+can+be+used+to+measure+embodiment+when+controlling+a+walking+self-avatar.">EEG can be used to measure embodiment when controlling a walking self-avatar.</a> IEEE Conf. on Virtual Reality and 3D User Interfaces (VR). , 776-783 (2019).</li><li>Luu, T. P., Nakagome, S., He, Y., Contreras-Vidal, J. L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Real-time+eeg-based+brain-computer+interface+to+a+virtual+avatar+enhances+cortical+involvement+in+human+treadmill+walking.">Real-time eeg-based brain-computer interface to a virtual avatar enhances cortical involvement in human treadmill walking.</a> Sci Rep. 7 (1), 8895 (2017).</li><li>Longo, B. B., Benevides, A. B., Castillo, J., Bastos-Filho, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Using+brain-computer+interface+to+control+an+avatar+in+a+virtual+reality+environment.">Using brain-computer interface to control an avatar in a virtual reality environment.</a> 5th ISSNIP-IEEE Biosignals and Biorobotics Conf. (BRC). , 1-4 (2014).</li><li>Hinrichs, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Comparison+between+a+wireless+dry+electrode+eeg+system+with+a+conventional+wired+wet+electrode+eeg+system+for+clinical+applications.">Comparison between a wireless dry electrode eeg system with a conventional wired wet electrode eeg system for clinical applications.</a> Scientific reports. 10 (1), 5218 (2020).</li><li>&#352;kola, F., Tinkov&#225;, S., Liarokapis, F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Progressive+training+for+motor+imagery+brain-computer+interfaces+using+gamification+and+virtual+reality+embodiment.">Progressive training for motor imagery brain-computer interfaces using gamification and virtual reality embodiment.</a> Front Human Neurosci. 13, 329 (2019).</li></ol>

The authors do not have any conflict of interest to disclose.

The application of MI in conjunction with VR technology offers a promising avenue for rehabilitation by leveraging the brain&#39;s natural mechanisms for motor planning and execution. MI&#39;s ability to induce event-related desynchronization in specific brain frequency bands, mirroring the neural activity of physical movement2,3,4, provides a robust framework for engaging and strengthening the neural networks involved in motor ...

Rehabilitation paradigms for patients with neurological impairments are undergoing a transformative shift with the integration of advanced technologies such as brain-computer interfaces (BCI) and immersive virtual reality (VR), offering a more nuanced and effective method for fostering recovery. Motor imagery (MI), the technique at the heart of BCI-based rehabilitation, involves the mental rehearsal of physical movements without actual motor execution1. MI exploits a neural mechanism where imagining a movement triggers a pattern of brain activity that closely mirrors that of performing the physical action itself2,3,4. Specifically, engaging in MI leads to a phenomenon known as event-related desynchronization (ERD) in the alpha (8-13 Hz) and beta (13-25 Hz) frequency bands of the brain&#39;s electrical activity5,6,7. ERD is indicative of a suppression of the baseline brain rhythms, a pattern also observed during actual movement, thereby providing a neural substrate for the use of MI within BCI-assisted rehabilitation frameworks7. Such a similarity in cortical activation between MI and physical movement suggests that MI can effectively stimulate the neural networks involved in motor control, making it a valuable tool for patients with motor deficits8. Furthermore, the practice of MI has been extended beyond mere mental rehearsal to include action observation strategies9. Observing the movement of task-related body parts or actions in others can activate the mirror neuron network (MNN), a group of neurons that respond both to action observation and execution9. Activation of the MNN through observation has been demonstrated to induce cortical plasticity, as evidenced by various neuroimaging modalities, including functional MRI10, positron emission tomography11, and transcranial magnetic stimulation12. The evidence supports the notion that MI training, enhanced by action observation, can lead to significant neural adaptation and recovery in affected individuals.
Virtual reality technology has revolutionized the realm of MI-based rehabilitation by offering an immersive environment that enhances the sense of body ownership and blurs the distinctions between the real and virtual worlds13,14,15. The immersive quality of VR makes it an effective tool for action observation and motor imagery practice, as it allows participants to perceive the virtual environment as real15. Research has shown that VR devices have a more pronounced effect on MI training compared to traditional 2D monitor displays15,16. Such findings are evidenced by enhanced neural activity, such as increased ERD amplitude ratios in the sensorimotor cortex, highlighting the benefits of higher immersion levels in stimulating brain activity during visually guided MI exercises16. The system aids in improving MI performance for tasks involving arm or limb movements by providing direct feedback, thereby enhancing the rehabilitation process16,17. The synergy between MI and VR emphasizes integrating sensory, perceptual, cognitive, and motor activities18,19. The combination has been particularly beneficial for stroke survivors20,21 and war veterans22, as studies have shown that integrating VR into MI-based rehabilitation protocols can significantly reduce rehabilitation time and improve recovery outcomes. The unique feature of VR in rehabilitation lies in its ability to create a sense of presence within a specifically designed virtual environment, enhancing the rehabilitation experience that is further augmented by the inclusion of virtual avatars representing the user&#39;s body, which has been increasingly utilized in motor rehabilitation studies23. These avatars offer a realistic three-dimensional representation of limb movements, aiding in MI and significantly impacting motor cortex activation. By allowing participants to visualize their virtual selves performing specific tasks, VR not only enriches the MI experience but also fosters a more rapid and effective neural reorganization and recovery process24. The implementation of virtual avatars and simulated environments in MI training emphasizes the natural and integrated use of virtual bodies within immersive virtual worlds.
Despite the remarkable advantages of BCI-based control of 3D avatars in MI for rehabilitation, a significant limitation remains in the predominant use of offline methodologies. Currently, most BCI applications involve capturing pre-recorded electroencephalography (EEG) data that is subsequently utilized to manipulate an avatar24,25. Even in scenarios where real-time avatar control is achieved, these avatars are often generic and do not resemble the participants they represent23. This generic approach misses a critical opportunity to deepen the immersion and sense of body ownership, which is crucial for effective rehabilitation24. The creation of a 3D avatar that mirrors the exact likeness of the subject could significantly enhance the immersive experience of the experience16. By visualizing themselves in the virtual world, participants could foster a stronger connection between their imagined and actual movements, potentially leading to more pronounced ERD patterns and, thus, more effective neural adaptation and recovery16. By advancing towards real-time control of personalized 3D avatars, the field of BCI and VR can significantly improve rehabilitation paradigms, offering a more nuanced, engaging, and efficacious method for patient recovery.
The current manuscript presents the creation, design, and technological aspects of both hardware and software of the VR-based real-time BCI control of 3D avatars, highlighting its innovative results that support its integration into motor rehabilitation settings. The proposed system will utilize electroencephalography (EEG) to capture motor imagery signals generated by the subject, which will then be used to control the movements and actions of the avatar in real time. The current approach will combine the advanced capabilities of VR technology with the precision of EEG in recognizing and interpreting brain activity related to imagined movements, aiming to create a more engaging and effective interface for users to interact with digital environments through the power of their thoughts.

<table><tbody><tr><td>Alienware Laptop</td><td>Dell</td><td></td><td>High-end gaming laptop with GTX1070 Graphics Card</td></tr><tr><td>Oculus Rift-S VR headset</td><td>Meta</td><td></td><td>VR headset</td></tr><tr><td>OpenBCI Cyton Daisy</td><td>OpenBCI</td><td></td><td>EEG system</td></tr><tr><td>OpenBCI Gel-free cap</td><td>OpenBCI</td><td></td><td>Gel-free cap for placing the EEG electrodes over the participant&#039;s scalp</td></tr></tbody></table>

motor imagery performance through embodied digital twins in a virtual reality-enabled brain-computer interface environment

This study introduces an innovative framework for neurological rehabilitation by integrating brain-computer interfaces (BCI) and virtual reality (VR) technologies with the customization of three-dimensional (3D) avatars. Traditional approaches to rehabilitation often fail to fully engage patients, primarily due to their inability to provide a deeply immersive and interactive experience. This research endeavors to fill this gap by utilizing motor imagery (MI) techniques, where participants visualize physical movements without actual execution. This method capitalizes on the brain's neural mechanisms, activating areas involved in movement execution when imagining movements, thereby facilitating the recovery process. The integration of VR's immersive capabilities with the precision of electroencephalography (EEG) to capture and interpret brain activity associated with imagined movements forms the core of this system. Digital Twins in the form of personalized 3D avatars are employed to significantly enhance the sense of immersion within the virtual environment. This heightened sense of embodiment is crucial for effective rehabilitation, aiming to bolster the connection between the patient and their virtual counterpart. By doing so, the system not only aims to improve motor imagery performance but also seeks to provide a more engaging and efficacious rehabilitation experience. Through the real-time application of BCI, the system allows for the direct translation of imagined movements into virtual actions performed by the 3D avatar, offering immediate feedback to the user. This feedback loop is essential for reinforcing the neural pathways involved in motor control and recovery. The ultimate goal of the developed system is to significantly enhance the effectiveness of motor imagery exercises by making them more interactive and responsive to the user's cognitive processes, thereby paving a new path in the field of neurological rehabilitation.

The results shown are from 5 individuals who followed the protocol described above. A total of 5 healthy adults (3 females) with ages ranging from 21 to 38 years participated in the study.
The individual classification performance for each participant under both motor imagery training and testing conditions is shown in Figure 2. An average confusion matrix for all subjects was calculated to evaluate the classifier&#39;s accuracy in distinguishing between left and ...

Read this Scientific Article Protocol about Motor Imagery Performance through Embodied Digital Twins in a Virtual Reality-Enabled Brain-Computer Interface Environment at JoVE.com

Author Spotlight: Enhancing Neurorehabilitation Through EEG, Motor Imagery, and Virtual Reality

Motor imagery in a virtual reality environment has wide applications in brain-computer interface systems. This manuscript outlines the use of personalized digital avatars that resemble the participants performing movements imagined by the participant in a virtual reality environment to enhance immersion and a sense of body ownership.

Motor Imagery Performance Through Embodied Digital Twins in a Virtual Reality-Enabled Brain-Computer Interface Environment

This study introduces an innovative framework for neurological rehabilitation by integrating brain-computer interfaces (BCI) and virtual reality (VR) ...

motor-imagery-performance-through-embodied-digital-twins-virtual

Research

JoVE Journal

Bioengineering

879 Views.  Vellore Institute of Technology. Mijn onderzoek in neurorevalidatie richt zich op het verbeteren van de functie van de bovenste ledematen voor mensen met neurologische verwondingen of aandoeningen met behulp van een combinatie van EEG, motorische beelden en virtual reality-technologieën. Het primaire doel is om te begrijpen hoe deze technologieën kunnen worden geïntegreerd om de effectiviteit van revalidatie van motorische vaardigheden te verbeteren. Virtual reality-omgevingen zijn geavan...

Video: Auteur in de schijnwerpers: neurorevalidatie verbeteren door middel van EEG, motorische beelden en virtual reality

Haptic/Graphic Rehabilitation: Integrating a Robot into a Virtual Environment Library and Applying it to Stroke Therapy

Recent research that tests interactive devices for prolonged therapy practice has revealed new prospects for robotics combined with graphical and other forms of biofeedback. Previous human-robot interactive systems have required different software commands to be implemented for each robot leading to unnecessary developmental overhead time each time a new system becomes available. For example, when a haptic/graphic virtual reality environment has been coded for one specific robot to provide haptic feedback, that specific robot would not be able to be traded for another robot without recoding the program. However, recent efforts in the open source community have proposed a wrapper class approach that can elicit nearly identical responses regardless of the robot used. The result can lead researchers across the globe to perform similar experiments using shared code. Therefore modular "switching out"of one robot for another would not affect development time. In this paper, we outline the successful creation and implementation of a wrapper class for one robot into the open-source H3DAPI, which integrates the software commands most commonly used by all robots.

Recently, a vast amount of prospects have come available for human-robot interactive systems. In this paper we outline the integration of a new robotic device with open source software that can rapidly make possible a library of interactive functionality. We then outline a clinical application for a neurorehabilitation application.

Recent research that tests interactive devices for prolonged therapy practice has revealed new prospects for robotics combined with graphical and other ...

Creating Two-Dimensional Patterned Substrates for Protein and Cell Confinement

Microcontact printing provides a rapid, highly reproducible method for the creation of well-defined patterned substrates.1 While microcontact printing can be employed to directly print a large number of molecules, including proteins,2 DNA,3 and silanes,4 the formation of self-assembled monolayers (SAMs) from long chain alkane thiols on gold provides a simple way to confine proteins and cells to specific patterns containing adhesive and resistant regions. This confinement can be used to control cell morphology and is useful for examining a variety of questions in protein and cell biology. Here, we describe a general method for the creation of well-defined protein patterns for cellular studies.5 This process involves three steps: the production of a patterned master using photolithography, the creation of a PDMS stamp, and microcontact printing of a gold-coated substrate. Once patterned, these cell culture substrates are capable of confining proteins and/or cells (primary cells or cell lines) to the pattern.
The use of self-assembled monolayer chemistry allows for precise control over the patterned protein/cell adhesive regions and non-adhesive regions; this cannot be achieved using direct protein stamping. Hexadecanethiol, the long chain alkane thiol used in the microcontact printing step, produces a hydrophobic surface that readily adsorbs protein from solution. The glycol-terminated thiol, used for backfilling the non-printed regions of the substrate, creates a monolayer that is resistant to protein adsorption and therefore cell growth.6 These thiol monomers produce highly structured monolayers that precisely define regions of the substrate that can support protein adsorption and cell growth. As a result, these substrates are useful for a wide variety of applications from the study of intercellular behavior7 to the creation of microelectronics.8
While other types of monolayer chemistry have been used for cell culture studies, including work from our group using trichlorosilanes to create patterns directly on glass substrates,9 patterned monolayers formed from alkane thiols on gold are straight-forward to prepare. Moreover, the monomers used for monolayer preparation are commercially available, stable, and do not require storage or handling under inert atmosphere. Patterned substrates prepared from alkane thiols can also be recycled and reused several times, maintaining cell confinement.10

Self-assembled monolayers (SAMs) formed from long chain alkane thiols on gold provide well-defined substrates for the formation of protein patterns and cell confinement. Microcontact printing of hexadecanethiol using a polydimethylsiloxane (PDMS) stamp followed by backfilling with a glycol-terminated alkane thiol monomer produces a pattern where protein and cells adsorb only to the stamped hexadecanethiol region.

Microcontact printing provides a rapid, highly reproducible method for the creation of well-defined patterned substrates.1  While microcontact printing ...

Brain Imaging Investigation of the Neural Correlates of Observing Virtual Social Interactions

The ability to gauge social interactions is crucial in the assessment of others&#x2019; intentions. Factors such as facial expressions and body language affect our decisions in personal and professional life alike 1. These "friend or foe" judgements are often based on first impressions, which in turn may affect our decisions to "approach or avoid". Previous studies investigating the neural correlates of social cognition tended to use static facial stimuli 2. Here, we illustrate an experimental design in which whole-body animated characters were used in conjunction with functional magnetic resonance imaging (fMRI) recordings. Fifteen participants were presented with short movie-clips of guest-host interactions in a business setting, while fMRI data were recorded; at the end of each movie, participants also provided ratings of the host behaviour. This design mimics more closely real-life situations, and hence may contribute to better understanding of the neural mechanisms of social interactions in healthy behaviour, and to gaining insight into possible causes of deficits in social behaviour in such clinical conditions as social anxiety and autism 3.

This article demonstrates an experimental design in which whole-body animated characters are used in conjunction with functional magnetic resonance imaging (fMRI) to investigate the neural correlates of observing virtual social interactions.

The ability to gauge social interactions is crucial in the assessment of others&#x2019; intentions. Factors such as facial expressions and body language ...

Neuroscience

Multifunctional Setup for Studying Human Motor Control Using Transcranial Magnetic Stimulation, Electromyography, Motion Capture, and Virtual Reality

The study of neuromuscular control of movement in humans is accomplished with numerous technologies. Non-invasive methods for investigating neuromuscular function include transcranial magnetic stimulation, electromyography, and three-dimensional motion capture. The advent of readily available and cost-effective virtual reality solutions has expanded the capabilities of researchers in recreating &#8220;real-world&#8221; environments and movements in a laboratory setting. Naturalistic movement analysis will not only garner a greater understanding of motor control in healthy individuals, but also permit the design of experiments and rehabilitation strategies that target specific motor impairments (e.g. stroke). The combined use of these tools will lead to increasingly deeper understanding of neural mechanisms of motor control. A key requirement when combining these data acquisition systems is fine temporal correspondence between the various data streams. This protocol describes a multifunctional system&#8217;s overall connectivity, intersystem signaling, and the temporal synchronization of recorded data. Synchronization of the component systems is primarily accomplished through the use of a customizable circuit, readily made with off the shelf components and minimal electronics assembly skills.

Transcranial magnetic stimulation, electromyography, and 3D motion capture are commonly used non-invasive techniques for investigating neuromuscular function in humans. In this paper, we describe a protocol that synchronously samples data generated by all three of these tools along with the unique addition of virtual reality stimulus presentation and feedback.

The study of neuromuscular control of movement in humans is accomplished with numerous technologies. Non-invasive methods for investigating neuromuscular ...

Behavior

Creating Virtual-hand and Virtual-face Illusions to Investigate Self-representation

Studies investigating how people represent themselves and their own body often use variants of &#34;ownership illusions&#34;, such as the traditional rubber-hand illusion or the more recently discovered enfacement illusion. However, these examples require rather artificial experimental setups, in which the artificial effector needs to be stroked in synchrony with the participants&#39; real hand or face&#8212;a situation in which participants have no control over the stroking or the movements of their real or artificial effector. Here, we describe a technique to establish ownership illusions in a setup that is more realistic, more intuitive, and of presumably higher ecological validity. It allows creating the virtual-hand illusion by having participants control the movements of a virtual hand presented on a screen or in virtual space in front of them. If the virtual hand moves in synchrony with the participants&#39; own real hand, they tend to perceive the virtual hand as part of their own body. The technique also creates the virtual-face illusion by having participants control the movements of a virtual face in front of them, again with the effect that they tend to perceive the face as their own if it moves in synchrony with their real face. Studying the circumstances that illusions of this sort can be created, increased, or reduced provides important information about how people create and maintain representations of themselves.

Here, we describe virtual-hand and virtual-face illusion paradigms that can be used to study body-related self-perception/-representation. They have already been used in various studies to demonstrate that, under specific conditions, a virtual hand or face can be incorporated into one&#39;s body representation, suggesting that body representations are rather flexible.

Studies investigating how people represent themselves and their own body often use variants of &#34;ownership illusions&#34;, such as the traditional ...

Using Virtual Reality to Transfer Motor Skill Knowledge from One Hand to Another

As far as acquiring motor skills is concerned, training by voluntary physical movement is superior to all other forms of training (e.g. training by observation or passive movement of trainee&#39;s hands by a robotic device). This obviously presents a major challenge in the rehabilitation of a paretic limb since voluntary control of physical movement is limited. Here, we describe a novel training scheme we have developed that has the potential to circumvent this major challenge. We exploited the voluntary control of one hand and provided real-time movement-based manipulated sensory feedback as if the other hand is moving. Visual manipulation through virtual reality (VR) was combined with a device that yokes left-hand fingers to passively follow right-hand voluntary finger movements. In healthy subjects, we demonstrate enhanced within-session performance gains of a limb in the absence of voluntary physical training. Results in healthy subjects suggest that training with the unique VR setup might also be beneficial for patients with upper limb hemiparesis by exploiting the voluntary control of their healthy hand to improve rehabilitation of their affected hand.

We describe a novel virtual-reality based setup which exploits voluntary control of one hand to improve motor-skill performance in the other (non-trained) hand. This is achieved by providing real-time movement-based sensory feedback as if the non-trained hand is moving. This new approach may be used to enhance rehabilitation of patients with unilateral hemiparesis.

As far as acquiring motor skills is concerned, training by voluntary physical movement is superior to all other forms of training (e.g. training by ...

Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality

The inability to complete instrumental activities of daily living (IADL) is a precursor to various neuropsychological diseases. Questionnaire-based assessments of IADL are easy to use but prone to subjective bias. Here, we describe a novel virtual reality (VR) test to assess two complex IADL tasks: handling financial transactions and using public transportation. While a participant performs the tasks in a VR setting, a motion capture system traces the position and orientation of the dominant hand and head in a three-dimensional Cartesian coordinate system. Kinematic raw data are collected and converted into &#39;kinematic performance measures,&#39; i.e., motion trajectory, moving distance, and time to completion. Motion trajectory is the path of a particular body part (e.g., dominant hand or head) in space. Moving distance refers to the total distance of the trajectory, and time to completion is how long it took to complete an IADL task. These kinematic measures could discriminate patients with cognitive impairment from healthy controls. The development of this kinematic measuring protocol allows detection of early IADL-related cognitive impairments.

We designed a virtual reality test to assess instrumental activities of daily living (IADL) with a motion capture system. We propose a detailed kinematic analysis to interpret the participant&#39;s various movements, including trajectory, moving distance, and time to completion to evaluate IADL capabilities.

The inability to complete instrumental activities of daily living (IADL) is a precursor to various neuropsychological diseases. Questionnaire-based ...

Real-time Video Projection in an MRI for Characterization of Neural Correlates Associated with Mirror Therapy for Phantom Limb Pain

Mirror therapy (MT) has been proposed as an effective rehabilitative strategy to alleviate pain symptoms in amputees with phantom limb pain (PLP). However, establishing the neural correlates associated with MT therapy have been challenging given that it is difficult to administer the therapy effectively within a magnetic resonance imaging (MRI) scanner environment. To characterize the functional organization of cortical regions associated with this rehabilitative strategy, we have developed a combined behavioral and functional neuroimaging protocol that can be applied in participants with a leg amputation. This novel approach allows participants to undergo MT within the MRI scanner environment by viewing real-time video images captured by a camera. The images are viewed by the participant through a system of mirrors and a monitor that the participant views while lying on the scanner bed. In this manner, functional changes in cortical areas of interest (e.g., sensorimotor cortex) can be characterized in response to the direct application of MT.

We present a novel combined behavioral and neuroimaging protocol employing real-time video projection for the purpose of characterizing the neural correlates associated with mirror therapy within the magnetic resonance imaging scanner environment in leg amputee subjects with phantom limb pain.

Mirror therapy (MT) has been proposed as an effective rehabilitative strategy to alleviate pain symptoms in amputees with phantom limb pain (PLP). ...

Restoring Post-Stroke Motor Function by Motor Imagery Brain-Computer Interface

Motor Imagery Brain-Computer Interface in Rehabilitation of Upper Limb Motor Dysfunction After Stroke

The rehabilitation effect of patients with moderate or severe upper limb motor dysfunction after stroke is poor, which has been the focus of research owing to the difficulties encountered. Brain-computer interface (BCI) represents a hot frontier technology in brain neuroscience research. It refers to the direct conversion of the sensory perception, imagery, cognition, and thinking of users or subjects into actions, without reliance on peripheral nerves or muscles, to establish direct communication and control channels between the brain and external devices. Motor imagery brain-computer interface (MI-BCI) is the most common clinical application of rehabilitation as a non-invasive means of rehabilitation. Previous clinical studies have confirmed that MI-BCI positively improves motor dysfunction in patients after stroke. However, there is a lack of clinical operation demonstration. To that end, this study describes in detail the treatment of MI-BCI for patients with moderate and severe upper limb dysfunction after stroke and shows the intervention effect of MI-BCI through clinical function evaluation and brain function evaluation results, thereby providing ideas and references for clinical rehabilitation application and mechanism research.

Author Spotlight: Using Motor Imagery Brain-Computer Interface to Improve Motor and Cognitive Function in Stroke Patients

The purpose of this study is to provide an important reference for the standard clinical operation of motor imagery brain-computer interface (MI-BCI) for upper limb motor dysfunction after stroke.

The rehabilitation effect of patients with moderate or severe upper limb motor dysfunction after stroke is poor, which has been the focus of research ...

Enhancing Stroke Rehabilitation with Virtual Reality-Based Occupational Training

Cognitive Function and Upper Limb Rehabilitation Training Post-Stroke Using a Digital Occupational Training System

Stroke rehabilitation often requires frequent and intensive therapy to improve functional recovery. Virtual reality (VR) technology has shown the potential to meet these demands by providing engaging and motivating therapy options. The digital occupational training system is a VR application that utilizes cutting-edge technologies, including multi-touch screens, virtual reality, and human-computer interaction, to offer diverse training techniques for advanced cognitive capacity and hand-eye coordination abilities. The objective of this study was to assess the effectiveness of this program in enhancing cognitive function and upper extremity rehabilitation in stroke patients. The training and assessment consist of five cognitive modules covering perception, attention, memory, logical reasoning, and calculation, along with hand-eye coordination training. This research indicates that after eight weeks of training, the digital occupational training system can significantly improve cognitive function, daily living skills, attention, and self-care abilities in stroke patients. This software can be employed as a time-saving and clinically effective rehabilitation aid to complement traditional one-on-one occupational therapy sessions. In summary, the digital occupational training system shows promise and offers potential financial benefits as a tool to support the functional recovery of stroke patients.

Author Spotlight: Rehabilitation of Stroke Patients With a Digital Occupational Training System

The current protocol outlines how the VR-based digital occupational training system enhances the rehabilitation of patients with cognitive impairment and upper limb dysfunction following a stroke.

Motor Imagery Performance Through Embodied Digital Twins in a Virtual Reality-Enabled Brain-Computer Interface Environment

Samenvatting

Meer video's verkennen