Name: measuring the kinematics of daily living movements with motion capture systems in virtual reality
Uploaded: 2018-04-05T14:00:00
Description: Watch this Scientific Journal Video about Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality at JoVE.com

K.S. and A.L. contribute equally. This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT &#38; Future Planning (NRF-2016R1D1A1B03931389).

All experimental procedures described here were approved by the Institutional Review Board of Hanyang University, according to the Declaration of Helsinki (HYI-15-029-2). 6 healthy controls (4 males and 2 females) and 6 MCI patients (3 males and 3 females) were recruited from a tertiary medical center, Hanyang University Hospital.
1. Recruit Participants
<ol>
	<li>Recruit MCI patients (i.e., impaired IADL capabilities) and healthy controls (i.e., normal IADL capabilities) a...

<ol>
	<li>Reppermund, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Impairment+in+instrumental+activities+of+daily+living+with+high+cognitive+demand+is+an+early+marker+of+mild+cognitive+impairment:+the+Sydney+Memory+and+Ageing+Study.">Impairment in instrumental activities of daily living with high cognitive demand is an early marker of mild cognitive impairment: the Sydney Memory and Ageing Study.</a> Psychol. Med. 43 (11), 2437-2445 (2013).</li><li>Graf, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+Lawton+instrumental+activities+of+daily+living+scale.">The Lawton instrumental activities of daily living scale.</a> Am. J. Nurs. 108 (4), 52-62 (2008).</li><li>Gold, D. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+examination+of+instrumental+activities+of+daily+living+assessment+in+older+adults+and+mild+cognitive+impairment.">An examination of instrumental activities of daily living assessment in older adults and mild cognitive impairment.</a> J. Clin. Exp. Neuropsychol. 34 (1), 11-34 (2012).</li><li>Jekel, K., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mild+cognitive+impairment+and+deficits+in+instrumental+activities+of+daily+living:+a+systematic+review.">Mild cognitive impairment and deficits in instrumental activities of daily living: a systematic review.</a> Alzheimers. Res. Ther. 7 (1), 17 (2015).</li><li>Suchy, Y., Kraybill, M. L., Franchow, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Instrumental+activities+of+daily+living+among+community-dwelling+older+adults:+discrepancies+between+self-report+and+performance+are+mediated+by+cognitive+reserve.">Instrumental activities of daily living among community-dwelling older adults: discrepancies between self-report and performance are mediated by cognitive reserve.</a> J. Clin. Exp. Neuropsychol. 33 (1), 92-100 (2011).</li><li>Desai, A. K., Grossberg, G. T., Sheth, D. N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Activities+of+Daily+Living+in+patients+with+Dementia.">Activities of Daily Living in patients with Dementia.</a> CNS drugs. 18 (13), 853-875 (2004).</li><li>Ma, M., Jain, L. C., Anderson, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Virtual, augmented reality and serious games for healthcare 1. 68, (2014).</li><li>Allain, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Detecting+everyday+action+deficits+in+Alzheimer's+disease+using+a+nonimmersive+virtual+reality+kitchen.">Detecting everyday action deficits in Alzheimer's disease using a nonimmersive virtual reality kitchen.</a> J. Int. Neuropsychol. Soc. 20 (5), 468-477 (2014).</li><li>Klinger, E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=AGATHE:+A+tool+for+personalized+rehabilitation+of+cognitive+functions+based+on+simulated+activities+of+daily+living.">AGATHE: A tool for personalized rehabilitation of cognitive functions based on simulated activities of daily living.</a> IRBM. 34 (2), 113-118 (2013).</li><li>White, D., Burdick, K., Fulk, G., Searleman, J., Carroll, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+virtual+reality+application+for+stroke+patient+rehabilitation.">A virtual reality application for stroke patient rehabilitation.</a> ICMA. 2, 1081-1086 (2005).</li><li>Dimbwadyo-Terrer, I., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Activities+of+daily+living+assessment+in+spinal+cord+injury+using+the+virtual+reality+system+Toyra:+functional+and+kinematic+correlations.">Activities of daily living assessment in spinal cord injury using the virtual reality system Toyra: functional and kinematic correlations.</a> Virtual Real. 20 (1), 17-26 (2016).</li><li>Preische, O., Heymann, P., Elbing, U., Laske, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Diagnostic+value+of+a+tablet-based+drawing+task+for+discrimination+of+patients+in+the+early+course+of+Alzheimer's+disease+from+healthy+individuals.">Diagnostic value of a tablet-based drawing task for discrimination of patients in the early course of Alzheimer's disease from healthy individuals.</a> J. Alzheimers. Dis. 55 (4), 1463-1469 (2017).</li><li>Seo, K., Kim, J. K., Oh, D. H., Ryu, H., Choi, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Virtual+daily+living+test+to+screen+for+mild+cognitive+impairment+using+kinematic+movement+analysis.">Virtual daily living test to screen for mild cognitive impairment using kinematic movement analysis.</a> PLOS ONE. 12 (7), e0181883 (2017).</li><li>Albert, M. S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+diagnosis+of+mild+cognitive+impairment+due+to+Alzheimer's+disease:+Recommendations+from+the+National+Institute+on+Aging-Alzheimer's+Association+workgroups+on+diagnostic+guidelines+for+Alzheimer's+disease.">The diagnosis of mild cognitive impairment due to Alzheimer's disease: Recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease.</a> Alzheimers. Dement. 7 (3), 270-279 (2011).</li><li>. Motive Quick Start Guide Available from: <a target="_blank" href="https://optitrack.com/public/documents/motive-quick-start-guide-v1.10.0.pdf">https://optitrack.com/public/documents/motive-quick-start-guide-v1.10.0.pdf</a> (2017)</li><li>Kennedy, R. S., Lane, N. E., Berbaum, K. S., Lilienthal, M. G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Simulator+sickness+questionnaire:+An+enhanced+method+for+quantifying+simulator+sickness.">Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness.</a> Int. J. Aviat. Psychol. 3 (3), 203-220 (1993).</li><li>Singh, N. B., Baumann, C. R., Taylor, W. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Can+Gait+Signatures+Provide+Quantitative+Measures+for+Aiding+Clinical+Decision-Making?+A+Systematic+Meta-Analysis+of+Gait+Variability+Behavior+in+Patients+with+Parkinson's+Disease.">Can Gait Signatures Provide Quantitative Measures for Aiding Clinical Decision-Making? A Systematic Meta-Analysis of Gait Variability Behavior in Patients with Parkinson's Disease.</a> Front. Hum. Neurosci. 10, 319 (2016).</li><li>Hernandez, F., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Six+degree-of-freedom+measurements+of+human+mild+traumatic+brain+injury.">Six degree-of-freedom measurements of human mild traumatic brain injury.</a> Ann. Biomed. Eng. 43 (8), 1918-1934 (2015).</li><li>Gamito, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cognitive+training+on+stroke+patients+via+virtual+reality-based+serious+games.">Cognitive training on stroke patients via virtual reality-based serious games.</a> Disabil. Rehabil. 39 (4), 385-388 (2017).</li></ol>

We detailed a kinematic measuring protocol of daily living movements with motion capture systems in an immersive VR environment. First, the experimental setting guided to how to set up, prepare, and familiarize participants with the immersive VR environment. Second, we developed two standardized IADL tasks in VR. Third, Step 3 and Step 5 in the Protocol section are the most critical steps to minimize VR sickness. When setting up the motion capture systems in the virtual environment (Step 3), it is important to mount the ...

Instrumental activities of daily living (IADL), such as handling financial transactions, using public transportation, and cooking, are medical markers since they require multiple neuropsychological functions1. Impaired IADL capabilities are thus considered precursors to neurological diseases, such as mild cognitive impairment (MCI) and dementia2. Gold&#39;s comprehensive review of IADL tasks3 indicated that more cognitively demanding tasks, such as managing finances and using public transportation, were the earliest predictor of MCI and dementia.
To date, the most commonly used assessments of IADL are self-reported questionnaires, informant-based questionnaires, and performance-based assessments4. Questionnaire-based assessments of IADL are cost-effective and easy to use, but are prone to subjective bias. For instance, when self-reporting, patients tend to over- or under-estimate their IADL capabilities5. Similarly, informants misjudge IADL capabilities due to the observer&#39;s misperceptions or knowledge gaps4. Thus, performance-based assessments that ask patients to carry out specific IADL tasks have been preferred, although many of the tasks are inappropriate for a general clinical setting6.
Recently, virtual reality (VR) studies have shown that this technology could have significant applications in medicine and healthcare, which includes everything from training to rehabilitation to medical assessment7. All participants can be tested under the same VR conditions, which mimic the real world. For instance, Allain et al.8 developed a virtual coffee-making task and showed that patients with cognitive impairment performed the task poorly. Klinger et al.9 developed another VR environment for mailing and shopping tasks and found a meaningful relationship between task completion time in VR and neuropsychological test results. Previous VR studies of IADL assessment have mostly focused on simple performance measures such as reaction time or accuracy when using conventional input devices such as a mouse and keyboard8,9. More detailed performance data about IADL is thus needed to efficiently screen for patients with MCI4.
Kinematic analysis of real-time motion capture data is a powerful approach to quantitatively document detailed performance data associated with IADL tasks. For example, White et al.10 developed a virtual kitchen that captures the participant&#39;s joint angle data during daily living tasks and used captured data to quantitatively assess the effectiveness of physical therapy. Dimbwadyo-Terrer et al.11 developed an immersive VR environment to assess upper limb performance when conducting basic daily living tasks and showed that kinematic data recorded in a VR environment highly correlated with functional scales of the upper limb. These kinematic analyses with motion capture systems could provide further opportunity to quickly assess a patient&#39;s cognitive impairment12. Inclusion of the detailed kinematic data in screening for patients with MCI significantly improved the classification of patients compared to healthy controls13.
Here, we describe a protocol to assess the kinematics of daily living movements with motion capture systems in an immersive VR environment. The protocol comprised two complex IADL tasks: &#34;Task 1: Withdraw money&#34; (handling financial transactions) and &#34;Task 2: Take a bus&#34; (using public transportation). While the tasks were performed, a motion capture system traced the position and orientation of the dominant hand and head. After completing Task 1, dominant hand trajectory, moving distance, and time to completion were collected. In Task 2, head trajectory, moving distance, and time to completion were collected. The Representative Results section in this article details the preliminary test of patients with MCI (i.e., IADL capabilities are impaired) compared to healthy controls (i.e., IADL capabilities are intact).

<table border="0" cellpadding="0" cellspacing="0" width="961">
	<tbody>
		<tr height="357">
			<td height="357" style="height:357px;width:216px;">Computer</td>
			<td style="width:161px;">N/A</td>
			<td style="width:127px;">N/A</td>
			<td style="width:457px;">Computer requirements:&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; - Single socket H3 (LGA 1150) supports 
			- Intel&#174; Xeon&#174; E3-1200 v3, 4th gen. Core i7/i5/i3 processors 
			- Intel&#174; C226 Express PCH 
			- Up to 32GB DDR3 ECC/non-ECC 1600MHz UDIMM in 4 sockets 
			- Dual Gigabit Ethernet LAN ports 
			- 8x SATA3 (6Gbps) 
			- 2x PCI-E 3.0 x16, 3x PCI-E 2.0 x1, and 2x PCI 5V 32-bit slots 
			- 6x USB 3.0 (2 rear + 4 via headers) 
			- 10x USB 2.0 (4 rear + 6 via headers) 
			- HD Audio 7.1 channel connector by Realtek ALC1150 
			- 1x DOM power connector and 1x SPDIF Out Header 
			- 800W High Efficiency Power Supply 
			- Intel Xeon E3-1230v3 
			- DDR3 PC12800 8GB ECC 
			- WD 1TB BLUE WD 10EZEX&#160; 3.5&#34; 
			- NVIDIA QUADRO K5000 &#38; SYNC</td>
		</tr>
		<tr height="84">
			<td height="84" style="height:84px;width:216px;">Stereoscopic 3D Projector</td>
			<td style="width:161px;">Barco</td>
			<td style="width:127px;">F35 AS3D WUXGA&#160;</td>
			<td style="width:457px;">Resolution:&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; - WQXGA (2,560 x 1,600)&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; - Panorama (2,560 x 1,080)&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; - WUXGA (1,920 x 1,200), 1080p (1,920 x 1,080)&#160;</td>
		</tr>
		<tr height="21">
			<td height="21" style="height:21px;width:216px;">Stereoscopic Glasses</td>
			<td style="width:161px;">Volfoni&#160;</td>
			<td style="width:127px;">Edge 1.2</td>
			<td>For further information, visit http://volfoni.com/en/edge-1-2/</td>
		</tr>
		<tr height="42">
			<td height="42" style="height:42px;width:216px;">Motion Capture Systems</td>
			<td style="width:161px;">NaturalPoint OptiTrack</td>
			<td style="width:127px;">17W</td>
			<td style="width:457px;">For further information, visit http://optitrack.com/products/prime-17w/</td>
		</tr>
		<tr height="42">
			<td height="42" style="height:42px;width:216px;">OptiTrack (Motion capture software)</td>
			<td style="width:161px;">NaturalPoint OptiTrack</td>
			<td style="width:127px;">Motive 2.0</td>
			<td style="width:457px;">For further information, visit https://optitrack.com/downloads/motive.html</td>
		</tr>
		<tr height="42">
			<td height="42" style="height:42px;width:216px;">MiddleVR (Middleware software)</td>
			<td style="width:161px;">MiddleVR</td>
			<td style="width:127px;">MiddleVR For Unity</td>
			<td style="width:457px;">For further information, visit http://www.middlevr.com/middlevr-for-unity/</td>
		</tr>
		<tr height="42">
			<td height="42" style="height:42px;width:216px;">VRDaemon (Middleware software)</td>
			<td style="width:161px;">MiddleVR</td>
			<td style="width:127px;">MiddleVR For Unity</td>
			<td style="width:457px;">For further information, visit http://www.middlevr.com/middlevr-for-unity/</td>
		</tr>
		<tr height="21">
			<td height="21" style="height:21px;width:216px;">Unity3D (Game engine)</td>
			<td style="width:161px;">Unity Technologies</td>
			<td style="width:127px;">Personal</td>
			<td>For further information, visit https://unity3d.com/unity</td>
		</tr>
	</tbody>
</table>

measuring the kinematics of daily living movements with motion capture systems in virtual reality

The inability to complete instrumental activities of daily living (IADL) is a precursor to various neuropsychological diseases. Questionnaire-based assessments of IADL are easy to use but prone to subjective bias. Here, we describe a novel virtual reality (VR) test to assess two complex IADL tasks: handling financial transactions and using public transportation. While a participant performs the tasks in a VR setting, a motion capture system traces the position and orientation of the dominant hand and head in a three-dimensional Cartesian coordinate system. Kinematic raw data are collected and converted into &#39;kinematic performance measures,&#39; i.e., motion trajectory, moving distance, and time to completion. Motion trajectory is the path of a particular body part (e.g., dominant hand or head) in space. Moving distance refers to the total distance of the trajectory, and time to completion is how long it took to complete an IADL task. These kinematic measures could discriminate patients with cognitive impairment from healthy controls. The development of this kinematic measuring protocol allows detection of early IADL-related cognitive impairments.

CSV files from &#34;Task 1: Withdraw money&#34; were analyzed using the statistical software R to calculate the dominant hand trajectory, moving distance, and time to completion. The trajectory of the dominant hand movement is visualized (Figure 6). The moving distance of the dominant hand is calculated by summing the total distances between sequential hand positions while performing Task 1. The distance between positions is the Euclidian distance. Time to co...

Watch this Scientific Journal Video about Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality at JoVE.com

Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality

We designed a virtual reality test to assess instrumental activities of daily living (IADL) with a motion capture system. We propose a detailed kinematic analysis to interpret the participant&#39;s various movements, including trajectory, moving distance, and time to completion to evaluate IADL capabilities.

The inability to complete instrumental activities of daily living (IADL) is a precursor to various neuropsychological diseases. Questionnaire-based ...

Research

JoVE Journal

Behavior

Two-photon Calcium Imaging in Mice Navigating a Virtual Reality Environment

In recent years, two-photon imaging has become an invaluable tool in neuroscience, as it allows for chronic measurement of the activity of genetically ...

Development of a Virtual Reality Assessment of Everyday Living Skills

Cognitive impairments affect the majority of patients with schizophrenia and these impairments predict poor long term psychosocial outcomes.&#160; ...

Multifunctional Setup for Studying Human Motor Control Using Transcranial Magnetic Stimulation, Electromyography, Motion Capture, and Virtual Reality

The study of neuromuscular control of movement in humans is accomplished with numerous technologies. Non-invasive methods for investigating neuromuscular ...

Evaluation of a Smartphone-based Human Activity Recognition System in a Daily Living Environment

An evaluation method that includes continuous activities in a daily-living environment was developed for Wearable Mobility Monitoring Systems (WMMS) that ...

Substantiating Appropriate Motion Capture Techniques for the Assessment of Nordic Walking Gait and Posture in Older Adults

Nordic walking (NW) has become a safe and simple form of exercise in recent years, and in studying this gait pattern, various data collection techniques ...

Using Virtual Reality to Transfer Motor Skill Knowledge from One Hand to Another

As far as acquiring motor skills is concerned, training by voluntary physical movement is superior to all other forms of training (e.g. training by ...

Mobile Game-based Virtual Reality Program for Upper Extremity Stroke Rehabilitation

Stroke rehabilitation requires repetitive, intensive, goal-oriented therapy. Virtual reality (VR) has the potential to satisfy these requirements. ...

A Networked Desktop Virtual Reality Setup for Decision Science and Navigation Experiments with Multiple Participants

Investigating the interactions among multiple participants is a challenge for researchers from various disciplines, including the decision sciences and ...

Virtual Reality Experiments with Physiological Measures

Virtual reality (VR) experiments are increasingly employed because of their internal and external validity compared to real-world observation and ...

Online Explorative Study on the Learning Uses of Virtual Reality Among Early Adopters

Virtual reality (VR) has shown great educational potential because it makes it possible to simulate any desired situation or event, thus playing an ...