Name: measuring the kinematics of daily living movements with motion capture systems in virtual reality
Uploaded: 2018-04-05T14:00:00
Description: Watch this Scientific Journal Video about Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality at JoVE.com

K.S. and A.L. contribute equally. This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT &#38; Future Planning (NRF-2016R1D1A1B03931389).

All experimental procedures described here were approved by the Institutional Review Board of Hanyang University, according to the Declaration of Helsinki (HYI-15-029-2). 6 healthy controls (4 males and 2 females) and 6 MCI patients (3 males and 3 females) were recruited from a tertiary medical center, Hanyang University Hospital.
1. Recruit Participants
<ol>
	<li>Recruit MCI patients (i.e., impaired IADL capabilities) and healthy controls (i.e., normal IADL capabilities) a...

<ol>
	<li>Reppermund, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Impairment+in+instrumental+activities+of+daily+living+with+high+cognitive+demand+is+an+early+marker+of+mild+cognitive+impairment:+the+Sydney+Memory+and+Ageing+Study.">Impairment in instrumental activities of daily living with high cognitive demand is an early marker of mild cognitive impairment: the Sydney Memory and Ageing Study.</a> Psychol. Med. 43 (11), 2437-2445 (2013).</li><li>Graf, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+Lawton+instrumental+activities+of+daily+living+scale.">The Lawton instrumental activities of daily living scale.</a> Am. J. Nurs. 108 (4), 52-62 (2008).</li><li>Gold, D. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+examination+of+instrumental+activities+of+daily+living+assessment+in+older+adults+and+mild+cognitive+impairment.">An examination of instrumental activities of daily living assessment in older adults and mild cognitive impairment.</a> J. Clin. Exp. Neuropsychol. 34 (1), 11-34 (2012).</li><li>Jekel, K., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mild+cognitive+impairment+and+deficits+in+instrumental+activities+of+daily+living:+a+systematic+review.">Mild cognitive impairment and deficits in instrumental activities of daily living: a systematic review.</a> Alzheimers. Res. Ther. 7 (1), 17 (2015).</li><li>Suchy, Y., Kraybill, M. L., Franchow, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Instrumental+activities+of+daily+living+among+community-dwelling+older+adults:+discrepancies+between+self-report+and+performance+are+mediated+by+cognitive+reserve.">Instrumental activities of daily living among community-dwelling older adults: discrepancies between self-report and performance are mediated by cognitive reserve.</a> J. Clin. Exp. Neuropsychol. 33 (1), 92-100 (2011).</li><li>Desai, A. K., Grossberg, G. T., Sheth, D. N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Activities+of+Daily+Living+in+patients+with+Dementia.">Activities of Daily Living in patients with Dementia.</a> CNS drugs. 18 (13), 853-875 (2004).</li><li>Ma, M., Jain, L. C., Anderson, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Virtual, augmented reality and serious games for healthcare 1. 68, (2014).</li><li>Allain, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Detecting+everyday+action+deficits+in+Alzheimer's+disease+using+a+nonimmersive+virtual+reality+kitchen.">Detecting everyday action deficits in Alzheimer's disease using a nonimmersive virtual reality kitchen.</a> J. Int. Neuropsychol. Soc. 20 (5), 468-477 (2014).</li><li>Klinger, E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=AGATHE:+A+tool+for+personalized+rehabilitation+of+cognitive+functions+based+on+simulated+activities+of+daily+living.">AGATHE: A tool for personalized rehabilitation of cognitive functions based on simulated activities of daily living.</a> IRBM. 34 (2), 113-118 (2013).</li><li>White, D., Burdick, K., Fulk, G., Searleman, J., Carroll, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+virtual+reality+application+for+stroke+patient+rehabilitation.">A virtual reality application for stroke patient rehabilitation.</a> ICMA. 2, 1081-1086 (2005).</li><li>Dimbwadyo-Terrer, I., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Activities+of+daily+living+assessment+in+spinal+cord+injury+using+the+virtual+reality+system+Toyra:+functional+and+kinematic+correlations.">Activities of daily living assessment in spinal cord injury using the virtual reality system Toyra: functional and kinematic correlations.</a> Virtual Real. 20 (1), 17-26 (2016).</li><li>Preische, O., Heymann, P., Elbing, U., Laske, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Diagnostic+value+of+a+tablet-based+drawing+task+for+discrimination+of+patients+in+the+early+course+of+Alzheimer's+disease+from+healthy+individuals.">Diagnostic value of a tablet-based drawing task for discrimination of patients in the early course of Alzheimer's disease from healthy individuals.</a> J. Alzheimers. Dis. 55 (4), 1463-1469 (2017).</li><li>Seo, K., Kim, J. K., Oh, D. H., Ryu, H., Choi, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Virtual+daily+living+test+to+screen+for+mild+cognitive+impairment+using+kinematic+movement+analysis.">Virtual daily living test to screen for mild cognitive impairment using kinematic movement analysis.</a> PLOS ONE. 12 (7), e0181883 (2017).</li><li>Albert, M. S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+diagnosis+of+mild+cognitive+impairment+due+to+Alzheimer's+disease:+Recommendations+from+the+National+Institute+on+Aging-Alzheimer's+Association+workgroups+on+diagnostic+guidelines+for+Alzheimer's+disease.">The diagnosis of mild cognitive impairment due to Alzheimer's disease: Recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease.</a> Alzheimers. Dement. 7 (3), 270-279 (2011).</li><li>. Motive Quick Start Guide Available from: <a target="_blank" href="https://optitrack.com/public/documents/motive-quick-start-guide-v1.10.0.pdf">https://optitrack.com/public/documents/motive-quick-start-guide-v1.10.0.pdf</a> (2017)</li><li>Kennedy, R. S., Lane, N. E., Berbaum, K. S., Lilienthal, M. G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Simulator+sickness+questionnaire:+An+enhanced+method+for+quantifying+simulator+sickness.">Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness.</a> Int. J. Aviat. Psychol. 3 (3), 203-220 (1993).</li><li>Singh, N. B., Baumann, C. R., Taylor, W. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Can+Gait+Signatures+Provide+Quantitative+Measures+for+Aiding+Clinical+Decision-Making?+A+Systematic+Meta-Analysis+of+Gait+Variability+Behavior+in+Patients+with+Parkinson's+Disease.">Can Gait Signatures Provide Quantitative Measures for Aiding Clinical Decision-Making? A Systematic Meta-Analysis of Gait Variability Behavior in Patients with Parkinson's Disease.</a> Front. Hum. Neurosci. 10, 319 (2016).</li><li>Hernandez, F., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Six+degree-of-freedom+measurements+of+human+mild+traumatic+brain+injury.">Six degree-of-freedom measurements of human mild traumatic brain injury.</a> Ann. Biomed. Eng. 43 (8), 1918-1934 (2015).</li><li>Gamito, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cognitive+training+on+stroke+patients+via+virtual+reality-based+serious+games.">Cognitive training on stroke patients via virtual reality-based serious games.</a> Disabil. Rehabil. 39 (4), 385-388 (2017).</li></ol>

We detailed a kinematic measuring protocol of daily living movements with motion capture systems in an immersive VR environment. First, the experimental setting guided to how to set up, prepare, and familiarize participants with the immersive VR environment. Second, we developed two standardized IADL tasks in VR. Third, Step 3 and Step 5 in the Protocol section are the most critical steps to minimize VR sickness. When setting up the motion capture systems in the virtual environment (Step 3), it is important to mount the ...

Instrumental activities of daily living (IADL), such as handling financial transactions, using public transportation, and cooking, are medical markers since they require multiple neuropsychological functions1. Impaired IADL capabilities are thus considered precursors to neurological diseases, such as mild cognitive impairment (MCI) and dementia2. Gold&#39;s comprehensive review of IADL tasks3 indicated that more cognitively demanding tasks, such as managing finances and using public transportation, were the earliest predictor of MCI and dementia.
To date, the most commonly used assessments of IADL are self-reported questionnaires, informant-based questionnaires, and performance-based assessments4. Questionnaire-based assessments of IADL are cost-effective and easy to use, but are prone to subjective bias. For instance, when self-reporting, patients tend to over- or under-estimate their IADL capabilities5. Similarly, informants misjudge IADL capabilities due to the observer&#39;s misperceptions or knowledge gaps4. Thus, performance-based assessments that ask patients to carry out specific IADL tasks have been preferred, although many of the tasks are inappropriate for a general clinical setting6.
Recently, virtual reality (VR) studies have shown that this technology could have significant applications in medicine and healthcare, which includes everything from training to rehabilitation to medical assessment7. All participants can be tested under the same VR conditions, which mimic the real world. For instance, Allain et al.8 developed a virtual coffee-making task and showed that patients with cognitive impairment performed the task poorly. Klinger et al.9 developed another VR environment for mailing and shopping tasks and found a meaningful relationship between task completion time in VR and neuropsychological test results. Previous VR studies of IADL assessment have mostly focused on simple performance measures such as reaction time or accuracy when using conventional input devices such as a mouse and keyboard8,9. More detailed performance data about IADL is thus needed to efficiently screen for patients with MCI4.
Kinematic analysis of real-time motion capture data is a powerful approach to quantitatively document detailed performance data associated with IADL tasks. For example, White et al.10 developed a virtual kitchen that captures the participant&#39;s joint angle data during daily living tasks and used captured data to quantitatively assess the effectiveness of physical therapy. Dimbwadyo-Terrer et al.11 developed an immersive VR environment to assess upper limb performance when conducting basic daily living tasks and showed that kinematic data recorded in a VR environment highly correlated with functional scales of the upper limb. These kinematic analyses with motion capture systems could provide further opportunity to quickly assess a patient&#39;s cognitive impairment12. Inclusion of the detailed kinematic data in screening for patients with MCI significantly improved the classification of patients compared to healthy controls13.
Here, we describe a protocol to assess the kinematics of daily living movements with motion capture systems in an immersive VR environment. The protocol comprised two complex IADL tasks: &#34;Task 1: Withdraw money&#34; (handling financial transactions) and &#34;Task 2: Take a bus&#34; (using public transportation). While the tasks were performed, a motion capture system traced the position and orientation of the dominant hand and head. After completing Task 1, dominant hand trajectory, moving distance, and time to completion were collected. In Task 2, head trajectory, moving distance, and time to completion were collected. The Representative Results section in this article details the preliminary test of patients with MCI (i.e., IADL capabilities are impaired) compared to healthy controls (i.e., IADL capabilities are intact).

<table border="0" cellpadding="0" cellspacing="0" width="961">
	<tbody>
		<tr height="357">
			<td height="357" style="height:357px;width:216px;">Computer</td>
			<td style="width:161px;">N/A</td>
			<td style="width:127px;">N/A</td>
			<td style="width:457px;">Computer requirements:&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; - Single socket H3 (LGA 1150) supports 
			- Intel&#174; Xeon&#174; E3-1200 v3, 4th gen. Core i7/i5/i3 processors 
			- Intel&#174; C226 Express PCH 
			- Up to 32GB DDR3 ECC/non-ECC 1600MHz UDIMM in 4 sockets 
			- Dual Gigabit Ethernet LAN ports 
			- 8x SATA3 (6Gbps) 
			- 2x PCI-E 3.0 x16, 3x PCI-E 2.0 x1, and 2x PCI 5V 32-bit slots 
			- 6x USB 3.0 (2 rear + 4 via headers) 
			- 10x USB 2.0 (4 rear + 6 via headers) 
			- HD Audio 7.1 channel connector by Realtek ALC1150 
			- 1x DOM power connector and 1x SPDIF Out Header 
			- 800W High Efficiency Power Supply 
			- Intel Xeon E3-1230v3 
			- DDR3 PC12800 8GB ECC 
			- WD 1TB BLUE WD 10EZEX&#160; 3.5&#34; 
			- NVIDIA QUADRO K5000 &#38; SYNC</td>
		</tr>
		<tr height="84">
			<td height="84" style="height:84px;width:216px;">Stereoscopic 3D Projector</td>
			<td style="width:161px;">Barco</td>
			<td style="width:127px;">F35 AS3D WUXGA&#160;</td>
			<td style="width:457px;">Resolution:&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; - WQXGA (2,560 x 1,600)&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; - Panorama (2,560 x 1,080)&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; - WUXGA (1,920 x 1,200), 1080p (1,920 x 1,080)&#160;</td>
		</tr>
		<tr height="21">
			<td height="21" style="height:21px;width:216px;">Stereoscopic Glasses</td>
			<td style="width:161px;">Volfoni&#160;</td>
			<td style="width:127px;">Edge 1.2</td>
			<td>For further information, visit http://volfoni.com/en/edge-1-2/</td>
		</tr>
		<tr height="42">
			<td height="42" style="height:42px;width:216px;">Motion Capture Systems</td>
			<td style="width:161px;">NaturalPoint OptiTrack</td>
			<td style="width:127px;">17W</td>
			<td style="width:457px;">For further information, visit http://optitrack.com/products/prime-17w/</td>
		</tr>
		<tr height="42">
			<td height="42" style="height:42px;width:216px;">OptiTrack (Motion capture software)</td>
			<td style="width:161px;">NaturalPoint OptiTrack</td>
			<td style="width:127px;">Motive 2.0</td>
			<td style="width:457px;">For further information, visit https://optitrack.com/downloads/motive.html</td>
		</tr>
		<tr height="42">
			<td height="42" style="height:42px;width:216px;">MiddleVR (Middleware software)</td>
			<td style="width:161px;">MiddleVR</td>
			<td style="width:127px;">MiddleVR For Unity</td>
			<td style="width:457px;">For further information, visit http://www.middlevr.com/middlevr-for-unity/</td>
		</tr>
		<tr height="42">
			<td height="42" style="height:42px;width:216px;">VRDaemon (Middleware software)</td>
			<td style="width:161px;">MiddleVR</td>
			<td style="width:127px;">MiddleVR For Unity</td>
			<td style="width:457px;">For further information, visit http://www.middlevr.com/middlevr-for-unity/</td>
		</tr>
		<tr height="21">
			<td height="21" style="height:21px;width:216px;">Unity3D (Game engine)</td>
			<td style="width:161px;">Unity Technologies</td>
			<td style="width:127px;">Personal</td>
			<td>For further information, visit https://unity3d.com/unity</td>
		</tr>
	</tbody>
</table>

measuring the kinematics of daily living movements with motion capture systems in virtual reality

The inability to complete instrumental activities of daily living (IADL) is a precursor to various neuropsychological diseases. Questionnaire-based assessments of IADL are easy to use but prone to subjective bias. Here, we describe a novel virtual reality (VR) test to assess two complex IADL tasks: handling financial transactions and using public transportation. While a participant performs the tasks in a VR setting, a motion capture system traces the position and orientation of the dominant hand and head in a three-dimensional Cartesian coordinate system. Kinematic raw data are collected and converted into &#39;kinematic performance measures,&#39; i.e., motion trajectory, moving distance, and time to completion. Motion trajectory is the path of a particular body part (e.g., dominant hand or head) in space. Moving distance refers to the total distance of the trajectory, and time to completion is how long it took to complete an IADL task. These kinematic measures could discriminate patients with cognitive impairment from healthy controls. The development of this kinematic measuring protocol allows detection of early IADL-related cognitive impairments.

CSV files from &#34;Task 1: Withdraw money&#34; were analyzed using the statistical software R to calculate the dominant hand trajectory, moving distance, and time to completion. The trajectory of the dominant hand movement is visualized (Figure 6). The moving distance of the dominant hand is calculated by summing the total distances between sequential hand positions while performing Task 1. The distance between positions is the Euclidian distance. Time to co...

Watch this Scientific Journal Video about Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality at JoVE.com

Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality

We designed a virtual reality test to assess instrumental activities of daily living (IADL) with a motion capture system. We propose a detailed kinematic analysis to interpret the participant&#39;s various movements, including trajectory, moving distance, and time to completion to evaluate IADL capabilities.

The inability to complete instrumental activities of daily living (IADL) is a precursor to various neuropsychological diseases. Questionnaire-based ...

The overall goal of this experimental procedure is to assess kinematic performance measures of daily living movements with motion capture systems in an immersive virtual environment. This method can help answer key questions in the medical field, such as how to diagnose dementia and mild cognitive impairment early through the ability to complete daily tasks. The main advantage of this technique is that it can quantitatively document detailed kinematic data, such as trajectory, moving distance, and timed completion while performing daily tasks in real time.The implication of this technique extends towards early diagnosis of dementia in EMSCI because the ability to complete daily tasks such as using public transportation are closely intertwined with neuropsychological functions. We first had the idea for this method when we realized the potential of virtual reality as an assessment environment, where the participants can be tested under the same conditions which mimic the real world experiences. Generally, individuals new to this method will struggle, because conducting experiments in virtual reality and analyzing kinematic data are hard to perform.Begin by setting up the hardware, including four computers, four stereoscopic three-dimensional projectors, and eight motion tracking cameras to track the position and orientation of the dominant hand and head during the two instrumental activities of daily living, IADL tasks. Ensure that all computers are equipped with the required software. For middleVR, check the website to obtain the latest versions of the libraries for the input devices, stereoscopy, clustering and interactions.Next, create a Windows 10 HomeGroup to connect the four computers to a home network. On the primary computer, create a folder and share it with other HomeGroup computers. Next, mount eight motion tracking cameras, in a virtual environment to fully cover the capture volume.Fix cameras securely so that they remain stationary during capture. Ensure objects in a virtual environment will be visible by at least two cameras at all times. Install motion capture software on the primary computer using the installation manual.Connect the primary computer with the motion capture systems with category six Ethernet cables. Next, prepare the virtual environment by moving all reflective objects. Ensure that all computers, stereoscopic 3D projectors, and motion capture systems are turned on.Once four computers are running, launch the VR software by double clicking on VRDaemon.exe. Finally, on the primary computer, initiate the motion capture software. Click the button near the top menu labeled open existing project, load the camera calibration file.Begin by providing the participant with stereoscopic glasses, weighing around 50 grams, and ensure that they are placed comfortably over the eyes and ears. Next, attach reflective markers, weighing less than one gram, to the participant's dominant hand and head. Be careful to attach reflective markers tightly.Inform the participant that they can freely move or rotate in the virtual environment using head movement, and can click virtual objects with the dominant hand. Let them know that a virtual hand will appear in the virtual environment to mimic the position of their index finger. Next, ask the participant to freely move in the virtual environment for five minutes to familiarize themselves with the VR environment.Then, ask them to click virtual buttons for five minutes in order to become familiar with how to interact with virtual objects with the dominant hand. Lastly, check whether the participant is immune to VR sickness with a simulator sickness questionnaire. First, explain to the participant the details of the task, and provide the eight action steps to complete the task in the virtual environment.Inform them the steps are one, insert the card into the ATM two, select the withdraw menu, three, select the amount to withdraw, four, select the bill type, five, enter the PIN or personal identification number, six, select the receipt option, seven, remove the card, and eight, take the money from the ATM. On the primary computer, initiate the middleware software. On the simulations tab, select a simulation file for task one, and a configuration file.Then, press the run button. Observe the participant completing task one. Once the first task is finished, check the saved cinematic data and csv files for further analysis from a shared folder.Provide the participant with a five minute break. Then start the second task, take a bus, by explaining the details. On the primary computer, initiate the middleware software.On the simulations tab, select a simulation file for task two and a configuration file. Press the run button to start task two. Instruct the participant to wait in the bus stop.Click the space bar key on the keyboard to make buses arrive at the bus stop. Finally, once the task is finished, check the saved kinematic data and csv files for further analysis from a shared folder. Help the participant remove the stereoscopic glasses and detach the reflective markers from the dominant hand and head.Here, the trajectory of the hand is visualized from task one, withdraw money, in healthy controls, and patients with mild cognitive impairment. Further, the trajectory of the head movement is visualized from task two, take a bus, in healthy controls and patients with mild cognitive impairment. Once mastered, this technique can be done in ten minutes, if it is performed properly, while attempting this procedure, it's important to remember to fix motion tracking camera securely to fully cover the capture volume, and to familiarize the participant sufficiently with the virtual environment to prevent VR sickness.Following this procedure, other non-invasive neuroimaging methods like electroencephalogram and functional near-infrared spectroscopy can be performed in order to answer additional questions like the relationship between kinematic performance measures and its underlying neurophysiological manifestation. After its development, this technique paved the way for researchers in the field of medicine to explore the kinematics of daily living movements in patients with neuropsychological disease. After watching this video, you should have a good understanding of how to set up a virtual environment, familiarize the participant, conduct tasks in VR, and measure the kinematics of daily tasks with motion capture system in the virtual environment.Don't forget that performing tasks in virtual reality can cause VR sickness and precautions such as sufficient familiarization with the virtual environment, and check whether the participant is immune to VR sickness should always be taken while performing this procedure.

measuring-kinematics-daily-living-movements-with-motion-capture

Research

JoVE Journal

Behavior

Two-photon Calcium Imaging in Mice Navigating a Virtual Reality Environment

In recent years, two-photon imaging has become an invaluable tool in neuroscience, as it allows for chronic measurement of the activity of genetically identified cells during behavior1-6. Here we describe methods to perform two-photon imaging in mouse cortex while the animal navigates a virtual reality environment. We focus on the aspects of the experimental procedures that are key to imaging in a behaving animal in a brightly lit virtual environment. The key problems that arise in this experimental setup that we here address are: minimizing brain motion related artifacts, minimizing light leak from the virtual reality projection system, and minimizing laser induced tissue damage. We also provide sample software to control the virtual reality environment and to do pupil tracking. With these procedures and resources it should be possible to convert a conventional two-photon microscope for use in behaving mice.

Here we describe the experimental procedures involved in two-photon imaging of mouse cortex during behavior in a virtual reality environment.

In recent years, two-photon imaging has become an invaluable tool in neuroscience, as it allows for chronic measurement of the activity of genetically ...

Development of a Virtual Reality Assessment of Everyday Living Skills

Cognitive impairments affect the majority of patients with schizophrenia and these impairments predict poor long term psychosocial outcomes.&#160; Treatment studies aimed at cognitive impairment in patients with schizophrenia not only require demonstration of improvements on cognitive tests, but also evidence that any cognitive changes lead to clinically meaningful improvements.&#160; Measures of &#8220;functional capacity&#8221; index the extent to which individuals have the potential to perform skills required for real world functioning. &#160;Current data do not support the recommendation of any single instrument for measurement of functional capacity.&#160; The Virtual Reality Functional Capacity Assessment Tool (VRFCAT) is a novel, interactive gaming based measure of functional capacity that uses a realistic simulated environment to recreate routine activities of daily living. Studies are currently underway to evaluate and establish the VRFCAT&#8217;s sensitivity, reliability, validity, and practicality.&#160;This new measure of functional capacity is practical, relevant, easy to use, and has several features that improve validity and sensitivity of measurement of function in clinical trials of patients with CNS disorders.

A challenge for proving treatment efficacy for cognitive impairments in schizophrenia is finding the optimizing measurement of skills related to everyday functioning.&#160; The Virtual Reality Functional Capacity Assessment Tool (VRFCAT) is an interactive gaming based computerized measure aimed at skills associated with everyday functioning, including baseline impairments and treatment related changes.

Cognitive impairments affect the majority of patients with schizophrenia and these impairments predict poor long term psychosocial outcomes.&#160; ...

Multifunctional Setup for Studying Human Motor Control Using Transcranial Magnetic Stimulation, Electromyography, Motion Capture, and Virtual Reality

The study of neuromuscular control of movement in humans is accomplished with numerous technologies. Non-invasive methods for investigating neuromuscular function include transcranial magnetic stimulation, electromyography, and three-dimensional motion capture. The advent of readily available and cost-effective virtual reality solutions has expanded the capabilities of researchers in recreating &#8220;real-world&#8221; environments and movements in a laboratory setting. Naturalistic movement analysis will not only garner a greater understanding of motor control in healthy individuals, but also permit the design of experiments and rehabilitation strategies that target specific motor impairments (e.g. stroke). The combined use of these tools will lead to increasingly deeper understanding of neural mechanisms of motor control. A key requirement when combining these data acquisition systems is fine temporal correspondence between the various data streams. This protocol describes a multifunctional system&#8217;s overall connectivity, intersystem signaling, and the temporal synchronization of recorded data. Synchronization of the component systems is primarily accomplished through the use of a customizable circuit, readily made with off the shelf components and minimal electronics assembly skills.

Transcranial magnetic stimulation, electromyography, and 3D motion capture are commonly used non-invasive techniques for investigating neuromuscular function in humans. In this paper, we describe a protocol that synchronously samples data generated by all three of these tools along with the unique addition of virtual reality stimulus presentation and feedback.

The study of neuromuscular control of movement in humans is accomplished with numerous technologies. Non-invasive methods for investigating neuromuscular ...

Evaluation of a Smartphone-based Human Activity Recognition System in a Daily Living Environment

An evaluation method that includes continuous activities in a daily-living environment was developed for Wearable Mobility Monitoring Systems (WMMS) that attempt to recognize user activities. Participants performed a pre-determined set of daily living actions within a continuous test circuit that included mobility activities (walking, standing, sitting, lying, ascending/descending stairs), daily living tasks (combing hair, brushing teeth, preparing food, eating, washing dishes), and subtle environment changes (opening doors, using an elevator, walking on inclines, traversing staircase landings, walking outdoors). 
To evaluate WMMS performance on this circuit, fifteen able-bodied participants completed the tasks while wearing a smartphone at their right front pelvis. The WMMS application used smartphone accelerometer and gyroscope signals to classify activity states. A gold standard comparison data set was created by video-recording each trial and manually logging activity onset times. Gold standard and WMMS data were analyzed offline. Three classification sets were calculated for each circuit: (i) mobility or immobility, ii) sit, stand, lie, or walking, and (iii) sit, stand, lie, walking, climbing stairs, or small standing movement. Sensitivities, specificities, and F-Scores for activity categorization and changes-of-state were calculated.
The mobile versus immobile classification set had a sensitivity of 86.30% &#177; 7.2% and specificity of 98.96% &#177; 0.6%, while the second prediction set had a sensitivity of 88.35% &#177; 7.80% and specificity of 98.51% &#177; 0.62%. For the third classification set, sensitivity was 84.92% &#177; 6.38% and specificity was 98.17 &#177; 0.62. F1 scores for the first, second and third classification sets were 86.17 &#177; 6.3, 80.19 &#177; 6.36, and 78.42 &#177; 5.96, respectively. This demonstrates that WMMS performance depends on the evaluation protocol in addition to the algorithms. The demonstrated protocol can be used and tailored for evaluating human activity recognition systems in rehabilitation medicine where mobility monitoring may be beneficial in clinical decision-making.

A standardized evaluation method was developed for Wearable Mobility Monitoring Systems (WMMS) that includes continuous activities in a realistic daily living environment. Testing with a series of daily living activities can decrease activity recognition sensitivity; therefore, realistic testing circuits are encouraged for valid evaluation of WMMS performance.

An evaluation method that includes continuous activities in a daily-living environment was developed for Wearable Mobility Monitoring Systems (WMMS) that ...

Substantiating Appropriate Motion Capture Techniques for the Assessment of Nordic Walking Gait and Posture in Older Adults

Nordic walking (NW) has become a safe and simple form of exercise in recent years, and in studying this gait pattern, various data collection techniques have been employed, each with positives and negatives. The aim was to determine the effect of NW on older adult gait and posture and to determine optimal use of different data collection systems in both short and long duration analysis. Gait and posture during NW and normal walking were assessed in 17 healthy older adults (age: 69 &#177; 7.3). Participants performed two trials of 6 Minute Walk Tests (6MWT) (1 with poles (WP) and 1 without poles (NP)) and 6 trials of a 5m walk (3 WP and 3 NP). Motion was recorded using two systems, a 6-sensor accelerometry system and an 8-camera 3-dimensional motion capture system, in order to quantify spatial-temporal, kinematic, and kinetic parameters.
With both systems, participants demonstrated increased stride length and double support and decreased gait speed and cadence WP compared to NP (p &#60;0.05). Also, with motion capture, larger single support time was found WP (p &#60;0.05). With 3-D capture, smaller hip power generation and moments of force were found at heel contact and pre-swing as well as smaller knee power absorption at heel contact, pre-swing, and terminal swing WP compared to NP, when assessed over one cycle (p &#60;0.05). Also, WP yielded smaller moments of force at heel contact and terminal swing along with larger moments at mid-stance of a gait cycle (p &#60;0.05). No changes were found for posture.
NW seems appropriate for promoting a normal gait pattern in older adults. Three-dimensional motion capture should primarily be used during short duration gait analysis (i.e. single gait cycle), while accelerometry systems should be primarily employed in instances requiring longer duration analysis such as during the 6MWT.

The aim was to substantiate optimal use of data collection techniques for Nordic walking gait and posture analysis. Three-dimensional motion capture should be used during short duration analysis (i.e. single gait cycle), while accelerometry should be employed for longer duration analysis (i.e. repeated cycles) like a 6 Minute Walk Test.

Nordic walking (NW) has become a safe and simple form of exercise in recent years, and in studying this gait pattern, various data collection techniques ...

Using Virtual Reality to Transfer Motor Skill Knowledge from One Hand to Another

As far as acquiring motor skills is concerned, training by voluntary physical movement is superior to all other forms of training (e.g. training by observation or passive movement of trainee&#39;s hands by a robotic device). This obviously presents a major challenge in the rehabilitation of a paretic limb since voluntary control of physical movement is limited. Here, we describe a novel training scheme we have developed that has the potential to circumvent this major challenge. We exploited the voluntary control of one hand and provided real-time movement-based manipulated sensory feedback as if the other hand is moving. Visual manipulation through virtual reality (VR) was combined with a device that yokes left-hand fingers to passively follow right-hand voluntary finger movements. In healthy subjects, we demonstrate enhanced within-session performance gains of a limb in the absence of voluntary physical training. Results in healthy subjects suggest that training with the unique VR setup might also be beneficial for patients with upper limb hemiparesis by exploiting the voluntary control of their healthy hand to improve rehabilitation of their affected hand.

We describe a novel virtual-reality based setup which exploits voluntary control of one hand to improve motor-skill performance in the other (non-trained) hand. This is achieved by providing real-time movement-based sensory feedback as if the non-trained hand is moving. This new approach may be used to enhance rehabilitation of patients with unilateral hemiparesis.

As far as acquiring motor skills is concerned, training by voluntary physical movement is superior to all other forms of training (e.g. training by ...

Mobile Game-based Virtual Reality Program for Upper Extremity Stroke Rehabilitation

Stroke rehabilitation requires repetitive, intensive, goal-oriented therapy. Virtual reality (VR) has the potential to satisfy these requirements. Game-based therapy can promote patients&#39; engagement in rehabilitation therapy as a more interesting and a motivating tool. Mobile devices such as smartphones and tablet PCs can provide personalized home-based therapy with interactive communication between patients and clinicians. In this study, a mobile VR upper extremity rehabilitation program using game applications was developed. The findings from the study show that the mobile game-based VR program effectively promotes upper extremity recovery in patients with stroke. In addition, patients completed two weeks of treatment using the program without adverse effects and were generally satisfied with the program. This mobile game-based VR upper extremity rehabilitation program can substitute for some parts of the conventional therapy that are delivered one-on-one by an occupational therapist. This time-efficient, easy to implement, and clinically effective program would be a good candidate tool for tele-rehabilitation for upper extremity recovery in patients with stroke. Patients and therapists can collaborate remotely through these e-health rehabilitation programs while reducing economic and social costs.

Here, we present a protocol to develop and apply a mobile game-based virtual reality program for the recovery of upper limb dysfunction in patients with stroke. The present study shows that the mobile program is feasible and effectively promotes upper limb recovery in stroke patients.

Stroke rehabilitation requires repetitive, intensive, goal-oriented therapy. Virtual reality (VR) has the potential to satisfy these requirements. ...

A Networked Desktop Virtual Reality Setup for Decision Science and Navigation Experiments with Multiple Participants

Investigating the interactions among multiple participants is a challenge for researchers from various disciplines, including the decision sciences and spatial cognition. With a local area network and dedicated software platform, experimenters can efficiently monitor the behavior of the participants that are simultaneously immersed in a desktop virtual environment and digitalize the collected data. These capabilities allow for experimental designs in spatial cognition and navigation research that would be difficult (if not impossible) to conduct in the real world. Possible experimental variations include stress during an evacuation, cooperative and competitive search tasks, and other contextual factors that may influence emergent crowd behavior. However, such a laboratory requires maintenance and strict protocols for data collection in a controlled setting. While the external validity of laboratory studies with human participants is sometimes questioned, a number of recent papers suggest that the correspondence between real and virtual environments may be sufficient for studying social behavior in terms of trajectories, hesitations, and spatial decisions. In this article, we describe a method for conducting experiments on decision-making and navigation with up to 36 participants in a networked desktop virtual reality setup (i.e., the Decision Science Laboratory or DeSciL). This experiment protocol can be adapted and applied by other researchers in order to set up a networked desktop virtual reality laboratory.

This paper describes a method for conducting multi-user experiments on decision-making and navigation using a networked computer laboratory.

Investigating the interactions among multiple participants is a challenge for researchers from various disciplines, including the decision sciences and ...

Virtual Reality Experiments with Physiological Measures

Virtual reality (VR) experiments are increasingly employed because of their internal and external validity compared to real-world observation and laboratory experiments, respectively. VR is especially useful for geographic visualizations and investigations of spatial behavior. In spatial behavior research, VR provides a platform for studying the relationship between navigation and physiological measures (e.g., skin conductance, heart rate, blood pressure). Specifically, physiological measures allow researchers to address novel questions and constrain previous theories of spatial abilities, strategies, and performance. For example, individual differences in navigation performance may be explained by the extent to which changes in arousal mediate the effects of task difficulty. However, the complexities in the design and implementation of VR experiments can distract experimenters from their primary research goals and introduce irregularities in data collection and analysis. To address these challenges, the Experiments in Virtual Environments (EVE) framework includes standardized modules such as participant training with the control interface, data collection using questionnaires, the synchronization of physiological measurements, and data storage. EVE also provides the necessary infrastructure for data management, visualization, and evaluation. The present paper describes a protocol that employs the EVE framework to conduct navigation experiments in VR with physiological sensors. The protocol lists the steps necessary for recruiting participants, attaching the physiological sensors, administering the experiment using EVE, and assessing the collected data with EVE evaluation tools. Overall, this protocol will facilitate future research by streamlining the design and implementation of VR experiments with physiological sensors.

Virtual reality (VR) experiments can be difficult to implement and require meticulous planning. This protocol describes a method for the design and implementation of VR experiments that collect physiological data from human participants. The Experiments in Virtual Environments (EVE) framework is employed to accelerate this process.

Virtual reality (VR) experiments are increasingly employed because of their internal and external validity compared to real-world observation and ...

Online Explorative Study on the Learning Uses of Virtual Reality Among Early Adopters

Virtual reality (VR) has shown great educational potential because it makes it possible to simulate any desired situation or event, thus playing an important role in addressing current educational challenges. Despite the unlimited learning possibilities that VR may offer, unless users are willing to apply virtual devices to education, the investment of time, money, and effort will be fruitless. It is therefore crucial to assess the educational interest of the first generation of VR users and to identify their current needs. To this end, in this study we designed an online questionnaire and applied it through the SaaS (Software as a service) of a private server. The sample consisted of 117 early VR adopters recruited via a main portal of communication and information technologies in Spain. In order to engage participants, we posted a thread in the main forum, which is dedicated to the advances and potential uses of VR. Once the responses were gathered, we analyzed the relationship between 12 variables (mean contrasts with Snedecor&#39;s F, and contingency analysis with chi-square and Sommer&#39;s d). The results showed that the current profile of a VR user is a male over 35 years old, with university studies, and who has purchased his viewer recently (&#60;1 year). As for the learning and teaching applications that these users were interested in, only 13.7% of the participants in this study use VR for educational purposes, although 28.2% were interested, indicating that perhaps the lack of applications or learning experiences may be hampering the use of VR within education. Almost half of the early adopters surveyed would like to learn using VR technology and are somehow optimistic about the relationship between VR and education, particularly those who are younger.

This article describes the profile of Spanish early adopters of virtual reality and their interests and preferences regarding learning and educational applications for this technology. To this aim, we designed an online questionnaire and interviewed 117 users of the main virtual reality forum on the Internet.

Virtual reality (VR) has shown great educational potential because it makes it possible to simulate any desired situation or event, thus playing an ...