Name: measuring the kinematics of daily living movements with motion capture systems in virtual reality
Uploaded: 2018-04-05T14:00:00.000+00:00
Duration: 8 min 45 s
Description: Watch this Scientific Journal Video about Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality at JoVE.com

K.S. and A.L. contribute equally. This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT &#38; Future Planning (NRF-2016R1D1A1B03931389).

All experimental procedures described here were approved by the Institutional Review Board of Hanyang University, according to the Declaration of Helsinki (HYI-15-029-2). 6 healthy controls (4 males and 2 females) and 6 MCI patients (3 males and 3 females) were recruited from a tertiary medical center, Hanyang University Hospital.
1. Recruit Participants
<ol>
	<li>Recruit MCI patients (i.e., impaired IADL capabilities) and healthy controls (i.e., normal IADL capabilities) aged between 70 - 80 years.</li>
	<li>With the help of a neurologist with more than 10 years of clinical experience, review the patients&#39; medical histories, and exclude patients with a history of neurological/psychiatric diseases or brain surgery. 
	NOTE: Use the following neuropsychological tests: Mini Mental State Examination-Dementia Screening, Korean Instrumental Activities of Daily Living, Free and Cued Selective Reminding Test, Digit Span Test-Forward/Backward, Trail Making Test-A/B13, and the criteria of Albert et al.14 to diagnose MCI.</li>
</ol>
2. Install VR Software and Connect Computers
<ol>
	<li>Setup the hardware in the dedicated room similar to Figure 1. Perform this protocol in a room-sized immersive virtual environment (4 x 2.5 x 2.5 m3) containing 4 computers, 4 stereoscopic three-dimensional (3D) projectors, and 8 motion tracking cameras to track the position and orientation of the dominant hand and head during the two IADL tasks. 
	NOTE: The VR technologies used in this article are computer hardware and software that offer immersive and interactive 3D experiences, by which realistic objects and events can be presented in a virtual environment. The details of the hardware and software are described in the Materials Table.</li>
	<li>Ensure all the computers are equipped with the required software (Visual Studio 2012 redistributable package (x86), DirectX, and MiddleVR, or equivalent). For MiddleVR, i.e., middleware software, check the website15 to obtain the latest versions of the libraries for the input devices, stereoscopy, clustering, and interactions.</li>
	<li>Connect computers to stereoscopic 3D projectors. Graphical settings are 1920 x 1080 pixel resolution.</li>
	<li>Create a Windows 10 HomeGroup to connect the 4 computers to a home network. On the primary computer, create a folder and share it with other HomeGroup computers.</li>
	<li>On the primary computer, initiate the middleware software. Click &#34;Cluster&#34; button. Set the primary computer as a server and other computers as clients. This will synchronize the state of all devices. Click &#34;3D Nodes&#34; button. Specify the position, orientation, and size of the virtual environment screen.</li>
	<li>Complete the settings based on the website15 and save the configuration file.</li>
</ol>
3. Set Up Motion Capture Systems in a Virtual Environment
<ol>
	<li>Mount 8 motion tracking cameras in a virtual environment to fully cover the capture volume. Fix cameras securely so that they remain stationary during capture. Ensure objects in a virtual environment will be visible by at least 2 cameras at all times.</li>
	<li>Install OptiTrack Motive software, i.e., motion capture software, on the primary computer using the installation manual16. Connect the primary computer with the motion capture systems with Category 6 Ethernet cables.</li>
	<li>Calibrate the motion capture systems with the following steps, as detailed in the software manual16.
	<ol>
		<li>Remove all extraneous reflections or unnecessary markers from the capture volume.</li>
		<li>Click the &#34;Mask Visible&#34; button to mask unwanted reflections or ambient interference.</li>
		<li>Click the &#34;Start Wanding&#34; button. Use the calibration wand to support the capture of sample frames in order to compute respective positions and orientations in 3D space.</li>
		<li>Click the &#34;Calculate&#34; button to calibrate the system using collected samples.</li>
		<li>Check the calibration results (in order from worst to best): Poor, Fair, Good, Great, Excellent, and Exceptional. If the result is better than Great, click the &#34;Apply&#34; button. If not, click the &#34;Cancel&#34; button and repeat the wanding process.</li>
		<li>Place the calibration square inside the 3D space where you want the origin to be located. Click &#34;Set Ground Plane&#34; button to establish a tracked 3D coordinate system origin.</li>
		<li>Select associated reflective markers for the dominant hand and head. Click the &#34;Rigid Body&#34; button and then click the &#34;Create From Selected Markers&#34; button.</li>
	</ol>
	</li>
	<li>On the motion capture software, open the &#34;Streaming&#34; menu. Verify that the port number listed is 3883, and select the &#34;Broadcast frame data&#34; box in the &#34;VRPN Streaming Engine&#34; category. Click &#34;Ctrl&#34; + &#34;S&#34; to save the calibration file.</li>
	<li>On the primary computer, initiate the middleware software. Click the &#34;Devices&#34; button. Add a VRPN Tracker to obtain tracking data from the motion capture system, and then save the configuration file.</li>
</ol>
4. Prepare a Virtual Environment for Use
<ol>
	<li>Remove all reflective objects (i.e., watches, rings, earrings, metals, etc.) from the virtual environment.</li>
	<li>Turn on computers, stereoscopic 3D projectors, and motion capture systems (360 frames per second).</li>
	<li>Once 4 computers are running, launch the VRDaemon software. For example, double click on &#34;VRDaemon.exe&#34; which located in &#34;C:\Program Files (x86)\MiddleVR\bin.&#34;</li>
	<li>On the primary computer, initiate the motion capture software. Click the button near the top menu labeled &#34;Open Existing Project.&#34; Load the camera calibration file.</li>
	<li>On the primary computer, initiate the middleware software. Click the &#34;Simulations&#34; button. Load the appropriate simulation and configuration files from a shared folder.</li>
	<li>On the middleware software, press the &#34;Run&#34; button to execute an immersive virtual application with the selected simulation and configuration files.</li>
</ol>
5. Familiarize the Participant with the Virtual Environment
<ol>
	<li>Provide the participant with stereoscopic glasses weighing around 50 g. The display frequency of the stereoscopic glasses is 192 Hz. Ensure that the stereoscopic glasses are comfortably placed over the eyes and ears; see Figure 2A.</li>
	<li>Attach reflective markers weighing less than 1 g to the participant&#39;s dominant hand and head. Be careful to attach the reflective markers tightly; see Figure 2B. Inform the participant that they can freely move or rotate in the virtual environment using head movement and can click virtual objects with the dominant hand. A virtual hand appears in the virtual environment to mimic the position of the participant&#39;s index finger; see Figure 3.</li>
	<li>Ask the participant to freely move (i.e., stand up, sit down, go left, and go right) in the virtual environment for 5 min to familiarize themselves with the VR environment. Then ask the participant to click virtual buttons for 5 min in order to become familiar with how to interact with virtual objects with the dominant hand. Provide another 10 min training session if the participant asks for one.</li>
	<li>Check whether the participant is immune to VR sickness with a simulator sickness questionnaire17. 
	CAUTION: The synchronized motion tracking on the stereoscopic display can cause VR sickness, which can result in discomfort, headache, stomach awareness, nausea, vomiting, pallor, sweating, fatigue, drowsiness, disorientation, and apathy. If the participant complains of fatigue or the simulator sickness score is too high, stop the protocol.</li>
</ol>
6. Perform &#34;Task 1: Withdraw money&#34;
CAUTION: Counterbalance the sequences of Task 1 and Task 2 to remove the carry-over effect.
<ol>
	<li>Explain to the participant the details of the task and provide the 8 action steps to complete the task in the virtual environment. The steps are (1) insert the card into the ATM, (2) select the &#39;withdraw&#39; menu, (3) select the amount to withdraw, (4) select the bill type, (5) enter the PIN (personal identification number), (6) select the receipt option, (7) remove the card, and (8) take the money from the ATM (see Figure 4).</li>
	<li>On the primary computer, initiate the middleware software. On the &#34;Simulations&#34; tab, select a simulation file for Task 1 and a configuration file. Press the &#34;Run&#34; button; &#34;Task 1: Withdraw money&#34; will run in the virtual environment. 
	NOTE: For the Task 1 file, see the attached &#34;Task 1 Withdraw Money.zip&#34; file in Supplemental File 1. Note that the virtual task was developed with the Unity 3D engine.</li>
	<li>If the &#34;Task 1: Withdraw money&#34; runs in the virtual environment, instruct the participant to perform as follows: &#34;Please withdraw 70,000 KRW (equivalent to around 60 USD) from the ATM for shopping. Select two different types of notes, one 50,000 KRW note for 50,000 KRW and two 10,000 KRW notes for 20,000 KRW. The password for your transaction is today&#39;s date. For instance, if the experiment is carried out on the 11th of November, then the PIN is 1111. Please keep the receipt for further reference.&#34;</li>
	<li>Once the task is finished, check the saved kinematic data in CSV files (comma-separated values) for further analysis from a shared folder. 
	NOTE: Using the motion capture systems, during &#34;Task 1: Withdraw money&#34; record the position and orientation of the dominant hand when conducting a task with a recording frequency of 1 ms.</li>
	<li>Give about 5 min break to the participant before starting &#34;Task 2: Take a bus.&#34;</li>
</ol>
7. Perform &#34;Task 2: Take a bus&#34;
<ol>
	<li>Explain to the participant the details of the task and provide instructions on how to complete &#34;Task 2: Take a bus&#34; as follows: &#34;Please wait at the bus stop and take the target bus. The target bus information will be given on the VR screen by a specific line number, color, and destination. When the target bus arrives, be sure to walk out of the bus stop and to the front door of the target bus. 8 different target buses will be randomly generated and presented.&#34; See Figure 5.</li>
	<li>On the primary computer, initiate the middleware software. On the &#34;Simulations&#34; tab, select a simulation file for Task 2 and a configuration file. Press the &#34;Run&#34; button, then &#34;Task 2: Take a bus&#34; will run in the virtual environment. 
	NOTE: For the Task 2 file, see the attached &#34;Task 2 Take a Bus.zip&#34; file in Supplemental File 2. Note that the virtual task was developed with the Unity 3D engine.</li>
	<li>If the &#34;Task 2: Take a bus&#34; runs in the virtual environment, instruct the participant to wait in the bus stop. Click the &#34;Spacebar&#34; key on the keyboard to make buses arrive at the bus stop.</li>
	<li>Once the task is finished, check the saved kinematic data in CSV files for further analysis from a shared folder. 
	NOTE: Using the motion capture systems, during &#34;Task 2: Take a bus&#34; record the position and orientation of the head when conducting the task with a recording frequency of 1 ms.</li>
	<li>The protocol is complete. Help the participant remove the stereoscopic glasses and detach the reflective markers from the dominant hand and head.</li>
</ol>

<ol>
	<li>Reppermund, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Impairment+in+instrumental+activities+of+daily+living+with+high+cognitive+demand+is+an+early+marker+of+mild+cognitive+impairment:+the+Sydney+Memory+and+Ageing+Study.">Impairment in instrumental activities of daily living with high cognitive demand is an early marker of mild cognitive impairment: the Sydney Memory and Ageing Study.</a> Psychol. Med. 43 (11), 2437-2445 (2013).</li><li>Graf, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+Lawton+instrumental+activities+of+daily+living+scale.">The Lawton instrumental activities of daily living scale.</a> Am. J. Nurs. 108 (4), 52-62 (2008).</li><li>Gold, D. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+examination+of+instrumental+activities+of+daily+living+assessment+in+older+adults+and+mild+cognitive+impairment.">An examination of instrumental activities of daily living assessment in older adults and mild cognitive impairment.</a> J. Clin. Exp. Neuropsychol. 34 (1), 11-34 (2012).</li><li>Jekel, K., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mild+cognitive+impairment+and+deficits+in+instrumental+activities+of+daily+living:+a+systematic+review.">Mild cognitive impairment and deficits in instrumental activities of daily living: a systematic review.</a> Alzheimers. Res. Ther. 7 (1), 17 (2015).</li><li>Suchy, Y., Kraybill, M. L., Franchow, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Instrumental+activities+of+daily+living+among+community-dwelling+older+adults:+discrepancies+between+self-report+and+performance+are+mediated+by+cognitive+reserve.">Instrumental activities of daily living among community-dwelling older adults: discrepancies between self-report and performance are mediated by cognitive reserve.</a> J. Clin. Exp. Neuropsychol. 33 (1), 92-100 (2011).</li><li>Desai, A. K., Grossberg, G. T., Sheth, D. N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Activities+of+Daily+Living+in+patients+with+Dementia.">Activities of Daily Living in patients with Dementia.</a> CNS drugs. 18 (13), 853-875 (2004).</li><li>Ma, M., Jain, L. C., Anderson, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Virtual, augmented reality and serious games for healthcare 1. 68, (2014).</li><li>Allain, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Detecting+everyday+action+deficits+in+Alzheimer's+disease+using+a+nonimmersive+virtual+reality+kitchen.">Detecting everyday action deficits in Alzheimer's disease using a nonimmersive virtual reality kitchen.</a> J. Int. Neuropsychol. Soc. 20 (5), 468-477 (2014).</li><li>Klinger, E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=AGATHE:+A+tool+for+personalized+rehabilitation+of+cognitive+functions+based+on+simulated+activities+of+daily+living.">AGATHE: A tool for personalized rehabilitation of cognitive functions based on simulated activities of daily living.</a> IRBM. 34 (2), 113-118 (2013).</li><li>White, D., Burdick, K., Fulk, G., Searleman, J., Carroll, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+virtual+reality+application+for+stroke+patient+rehabilitation.">A virtual reality application for stroke patient rehabilitation.</a> ICMA. 2, 1081-1086 (2005).</li><li>Dimbwadyo-Terrer, I., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Activities+of+daily+living+assessment+in+spinal+cord+injury+using+the+virtual+reality+system+Toyra:+functional+and+kinematic+correlations.">Activities of daily living assessment in spinal cord injury using the virtual reality system Toyra: functional and kinematic correlations.</a> Virtual Real. 20 (1), 17-26 (2016).</li><li>Preische, O., Heymann, P., Elbing, U., Laske, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Diagnostic+value+of+a+tablet-based+drawing+task+for+discrimination+of+patients+in+the+early+course+of+Alzheimer's+disease+from+healthy+individuals.">Diagnostic value of a tablet-based drawing task for discrimination of patients in the early course of Alzheimer's disease from healthy individuals.</a> J. Alzheimers. Dis. 55 (4), 1463-1469 (2017).</li><li>Seo, K., Kim, J. K., Oh, D. H., Ryu, H., Choi, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Virtual+daily+living+test+to+screen+for+mild+cognitive+impairment+using+kinematic+movement+analysis.">Virtual daily living test to screen for mild cognitive impairment using kinematic movement analysis.</a> PLOS ONE. 12 (7), e0181883 (2017).</li><li>Albert, M. S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+diagnosis+of+mild+cognitive+impairment+due+to+Alzheimer's+disease:+Recommendations+from+the+National+Institute+on+Aging-Alzheimer's+Association+workgroups+on+diagnostic+guidelines+for+Alzheimer's+disease.">The diagnosis of mild cognitive impairment due to Alzheimer's disease: Recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease.</a> Alzheimers. Dement. 7 (3), 270-279 (2011).</li><li>. Motive Quick Start Guide Available from: <a target="_blank" href="https://optitrack.com/public/documents/motive-quick-start-guide-v1.10.0.pdf">https://optitrack.com/public/documents/motive-quick-start-guide-v1.10.0.pdf</a> (2017)</li><li>Kennedy, R. S., Lane, N. E., Berbaum, K. S., Lilienthal, M. G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Simulator+sickness+questionnaire:+An+enhanced+method+for+quantifying+simulator+sickness.">Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness.</a> Int. J. Aviat. Psychol. 3 (3), 203-220 (1993).</li><li>Singh, N. B., Baumann, C. R., Taylor, W. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Can+Gait+Signatures+Provide+Quantitative+Measures+for+Aiding+Clinical+Decision-Making?+A+Systematic+Meta-Analysis+of+Gait+Variability+Behavior+in+Patients+with+Parkinson's+Disease.">Can Gait Signatures Provide Quantitative Measures for Aiding Clinical Decision-Making? A Systematic Meta-Analysis of Gait Variability Behavior in Patients with Parkinson's Disease.</a> Front. Hum. Neurosci. 10, 319 (2016).</li><li>Hernandez, F., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Six+degree-of-freedom+measurements+of+human+mild+traumatic+brain+injury.">Six degree-of-freedom measurements of human mild traumatic brain injury.</a> Ann. Biomed. Eng. 43 (8), 1918-1934 (2015).</li><li>Gamito, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cognitive+training+on+stroke+patients+via+virtual+reality-based+serious+games.">Cognitive training on stroke patients via virtual reality-based serious games.</a> Disabil. Rehabil. 39 (4), 385-388 (2017).</li></ol>

The authors declare no conflicts of interest.

We detailed a kinematic measuring protocol of daily living movements with motion capture systems in an immersive VR environment. First, the experimental setting guided to how to set up, prepare, and familiarize participants with the immersive VR environment. Second, we developed two standardized IADL tasks in VR. Third, Step 3 and Step 5 in the Protocol section are the most critical steps to minimize VR sickness. When setting up the motion capture systems in the virtual environment (Step 3), it is important to mount the ...

Instrumental activities of daily living (IADL), such as handling financial transactions, using public transportation, and cooking, are medical markers since they require multiple neuropsychological functions1. Impaired IADL capabilities are thus considered precursors to neurological diseases, such as mild cognitive impairment (MCI) and dementia2. Gold&#39;s comprehensive review of IADL tasks3 indicated that more cognitively demanding tasks, such as managing finances and using public transportation, were the earliest predictor of MCI and dementia.
To date, the most commonly used assessments of IADL are self-reported questionnaires, informant-based questionnaires, and performance-based assessments4. Questionnaire-based assessments of IADL are cost-effective and easy to use, but are prone to subjective bias. For instance, when self-reporting, patients tend to over- or under-estimate their IADL capabilities5. Similarly, informants misjudge IADL capabilities due to the observer&#39;s misperceptions or knowledge gaps4. Thus, performance-based assessments that ask patients to carry out specific IADL tasks have been preferred, although many of the tasks are inappropriate for a general clinical setting6.
Recently, virtual reality (VR) studies have shown that this technology could have significant applications in medicine and healthcare, which includes everything from training to rehabilitation to medical assessment7. All participants can be tested under the same VR conditions, which mimic the real world. For instance, Allain et al.8 developed a virtual coffee-making task and showed that patients with cognitive impairment performed the task poorly. Klinger et al.9 developed another VR environment for mailing and shopping tasks and found a meaningful relationship between task completion time in VR and neuropsychological test results. Previous VR studies of IADL assessment have mostly focused on simple performance measures such as reaction time or accuracy when using conventional input devices such as a mouse and keyboard8,9. More detailed performance data about IADL is thus needed to efficiently screen for patients with MCI4.
Kinematic analysis of real-time motion capture data is a powerful approach to quantitatively document detailed performance data associated with IADL tasks. For example, White et al.10 developed a virtual kitchen that captures the participant&#39;s joint angle data during daily living tasks and used captured data to quantitatively assess the effectiveness of physical therapy. Dimbwadyo-Terrer et al.11 developed an immersive VR environment to assess upper limb performance when conducting basic daily living tasks and showed that kinematic data recorded in a VR environment highly correlated with functional scales of the upper limb. These kinematic analyses with motion capture systems could provide further opportunity to quickly assess a patient&#39;s cognitive impairment12. Inclusion of the detailed kinematic data in screening for patients with MCI significantly improved the classification of patients compared to healthy controls13.
Here, we describe a protocol to assess the kinematics of daily living movements with motion capture systems in an immersive VR environment. The protocol comprised two complex IADL tasks: &#34;Task 1: Withdraw money&#34; (handling financial transactions) and &#34;Task 2: Take a bus&#34; (using public transportation). While the tasks were performed, a motion capture system traced the position and orientation of the dominant hand and head. After completing Task 1, dominant hand trajectory, moving distance, and time to completion were collected. In Task 2, head trajectory, moving distance, and time to completion were collected. The Representative Results section in this article details the preliminary test of patients with MCI (i.e., IADL capabilities are impaired) compared to healthy controls (i.e., IADL capabilities are intact).

<table><tbody><tr><td>Computer</td><td>N/A</td><td>N/A</td><td>Computer requirements:&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; - Single socket H3 (LGA 1150) supports&lt;br/&gt; - Intel&amp;reg; Xeon&amp;reg; E3-1200 v3, 4th gen. Core i7/i5/i3 processors&lt;br/&gt; - Intel&amp;reg; C226 Express PCH&lt;br/&gt; - Up to 32GB DDR3 ECC/non-ECC 1600MHz UDIMM in 4 sockets&lt;br/&gt; - Dual Gigabit Ethernet LAN ports&lt;br/&gt; - 8x SATA3 (6Gbps)&lt;br/&gt; - 2x PCI-E 3.0 x16, 3x PCI-E 2.0 x1, and 2x PCI 5V 32-bit slots&lt;br/&gt; - 6x USB 3.0 (2 rear + 4 via headers)&lt;br/&gt; - 10x USB 2.0 (4 rear + 6 via headers)&lt;br/&gt; - HD Audio 7.1 channel connector by Realtek ALC1150&lt;br/&gt; - 1x DOM power connector and 1x SPDIF Out Header&lt;br/&gt; - 800W High Efficiency Power Supply&lt;br/&gt; - Intel Xeon E3-1230v3&lt;br/&gt; - DDR3 PC12800 8GB ECC&lt;br/&gt; - WD 1TB BLUE WD 10EZEX&amp;nbsp; 3.5&quot;&lt;br/&gt; - NVIDIA QUADRO K5000 &amp;amp; SYNC</td></tr><tr><td>Stereoscopic 3D Projector</td><td>Barco</td><td>F35 AS3D WUXGA</td><td>Resolution: &lt;br /&gt;- WQXGA (2,560 x 1,600) &lt;br /&gt;- Panorama (2,560 x 1,080) &lt;br /&gt;- WUXGA (1,920 x 1,200), 1080p (1,920 x 1,080)</td></tr><tr><td>Stereoscopic Glasses</td><td>Volfoni</td><td>Edge 1.2</td><td>For further information, visit http://volfoni.com/en/edge-1-2/</td></tr><tr><td>Motion Capture Systems</td><td>NaturalPoint OptiTrack</td><td>17W</td><td>For further information, visit http://optitrack.com/products/prime-17w/</td></tr><tr><td>OptiTrack (Motion capture software)</td><td>NaturalPoint OptiTrack</td><td>Motive 2.0</td><td>For further information, visit https://optitrack.com/downloads/motive.html</td></tr><tr><td>MiddleVR (Middleware software)</td><td>MiddleVR</td><td>MiddleVR For Unity</td><td>For further information, visit http://www.middlevr.com/middlevr-for-unity/</td></tr><tr><td>VRDaemon (Middleware software)</td><td>MiddleVR</td><td>MiddleVR For Unity</td><td>For further information, visit http://www.middlevr.com/middlevr-for-unity/</td></tr><tr><td>Unity3D (Game engine)</td><td>Unity Technologies</td><td>Personal</td><td>For further information, visit https://unity3d.com/unity</td></tr></tbody></table>

measuring the kinematics of daily living movements with motion capture systems in virtual reality

The inability to complete instrumental activities of daily living (IADL) is a precursor to various neuropsychological diseases. Questionnaire-based assessments of IADL are easy to use but prone to subjective bias. Here, we describe a novel virtual reality (VR) test to assess two complex IADL tasks: handling financial transactions and using public transportation. While a participant performs the tasks in a VR setting, a motion capture system traces the position and orientation of the dominant hand and head in a three-dimensional Cartesian coordinate system. Kinematic raw data are collected and converted into &#39;kinematic performance measures,&#39; i.e., motion trajectory, moving distance, and time to completion. Motion trajectory is the path of a particular body part (e.g., dominant hand or head) in space. Moving distance refers to the total distance of the trajectory, and time to completion is how long it took to complete an IADL task. These kinematic measures could discriminate patients with cognitive impairment from healthy controls. The development of this kinematic measuring protocol allows detection of early IADL-related cognitive impairments.

CSV files from &#34;Task 1: Withdraw money&#34; were analyzed using the statistical software R to calculate the dominant hand trajectory, moving distance, and time to completion. The trajectory of the dominant hand movement is visualized (Figure 6). The moving distance of the dominant hand is calculated by summing the total distances between sequential hand positions while performing Task 1. The distance between positions is the Euclidian distance. Time to co...

Watch this Scientific Journal Video about Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality at JoVE.com

Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality

We designed a virtual reality test to assess instrumental activities of daily living (IADL) with a motion capture system. We propose a detailed kinematic analysis to interpret the participant&#39;s various movements, including trajectory, moving distance, and time to completion to evaluate IADL capabilities.

The inability to complete instrumental activities of daily living (IADL) is a precursor to various neuropsychological diseases. Questionnaire-based ...

measuring-kinematics-daily-living-movements-with-motion-capture

Research

JoVE Journal

Behavior

7.6K Views.  Hanyang University. We designed a virtual reality test to assess instrumental activities of daily living (IADL) with a motion capture system. We propose a detailed kinematic analysis to interpret the participant's various movements, including trajectory, moving distance, and time to completion to evaluate IADL capabilities.

Video: Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality

Development of a Virtual Reality Assessment of Everyday Living Skills

Cognitive impairments affect the majority of patients with schizophrenia and these impairments predict poor long term psychosocial outcomes.&#160; Treatment studies aimed at cognitive impairment in patients with schizophrenia not only require demonstration of improvements on cognitive tests, but also evidence that any cognitive changes lead to clinically meaningful improvements.&#160; Measures of &#8220;functional capacity&#8221; index the extent to which individuals have the potential to perform skills required for real world functioning. &#160;Current data do not support the recommendation of any single instrument for measurement of functional capacity.&#160; The Virtual Reality Functional Capacity Assessment Tool (VRFCAT) is a novel, interactive gaming based measure of functional capacity that uses a realistic simulated environment to recreate routine activities of daily living. Studies are currently underway to evaluate and establish the VRFCAT&#8217;s sensitivity, reliability, validity, and practicality.&#160;This new measure of functional capacity is practical, relevant, easy to use, and has several features that improve validity and sensitivity of measurement of function in clinical trials of patients with CNS disorders.

A challenge for proving treatment efficacy for cognitive impairments in schizophrenia is finding the optimizing measurement of skills related to everyday functioning.&#160; The Virtual Reality Functional Capacity Assessment Tool (VRFCAT) is an interactive gaming based computerized measure aimed at skills associated with everyday functioning, including baseline impairments and treatment related changes.

Cognitive impairments affect the majority of patients with schizophrenia and these impairments predict poor long term psychosocial outcomes.&#160; ...

Multifunctional Setup for Studying Human Motor Control Using Transcranial Magnetic Stimulation, Electromyography, Motion Capture, and Virtual Reality

The study of neuromuscular control of movement in humans is accomplished with numerous technologies. Non-invasive methods for investigating neuromuscular function include transcranial magnetic stimulation, electromyography, and three-dimensional motion capture. The advent of readily available and cost-effective virtual reality solutions has expanded the capabilities of researchers in recreating &#8220;real-world&#8221; environments and movements in a laboratory setting. Naturalistic movement analysis will not only garner a greater understanding of motor control in healthy individuals, but also permit the design of experiments and rehabilitation strategies that target specific motor impairments (e.g. stroke). The combined use of these tools will lead to increasingly deeper understanding of neural mechanisms of motor control. A key requirement when combining these data acquisition systems is fine temporal correspondence between the various data streams. This protocol describes a multifunctional system&#8217;s overall connectivity, intersystem signaling, and the temporal synchronization of recorded data. Synchronization of the component systems is primarily accomplished through the use of a customizable circuit, readily made with off the shelf components and minimal electronics assembly skills.

Transcranial magnetic stimulation, electromyography, and 3D motion capture are commonly used non-invasive techniques for investigating neuromuscular function in humans. In this paper, we describe a protocol that synchronously samples data generated by all three of these tools along with the unique addition of virtual reality stimulus presentation and feedback.

The study of neuromuscular control of movement in humans is accomplished with numerous technologies. Non-invasive methods for investigating neuromuscular ...

Movement Retraining using Real-time Feedback of Performance

Any modification of movement - especially movement patterns that have been honed over a number of years - requires re-organization of the neuromuscular patterns responsible for governing the movement performance. This motor learning can be enhanced through a number of methods that are utilized in research and clinical settings alike. In general, verbal feedback of performance in real-time or knowledge of results following movement is commonly used clinically as a preliminary means of instilling motor learning. Depending on patient preference and learning style, visual feedback (e.g. through use of a mirror or different types of video) or proprioceptive guidance utilizing therapist touch, are used to supplement verbal instructions from the therapist. Indeed, a combination of these forms of feedback is commonplace in the clinical setting to facilitate motor learning and optimize outcomes.
Laboratory-based, quantitative motion analysis has been a mainstay in research settings to provide accurate and objective analysis of a variety of movements in healthy and injured populations. While the actual mechanisms of capturing the movements may differ, all current motion analysis systems rely on the ability to track the movement of body segments and joints and to use established equations of motion to quantify key movement patterns. Due to limitations in acquisition and processing speed, analysis and description of the movements has traditionally occurred offline after completion of a given testing session.
This paper will highlight a new supplement to standard motion analysis techniques that relies on the near instantaneous assessment and quantification of movement patterns and the display of specific movement characteristics to the patient during a movement analysis session. As a result, this novel technique can provide a new method of feedback delivery that has advantages over currently used feedback methods.

Retraining abnormal movement patterns following injury or disease is a key component of physical rehabilitation. Recent advances in technology have permitted accurate assessment of movement during a variety of tasks, with near instantaneous quantification of results. This provides new opportunities for modification of faulty movement patterns in real time.

Any modification of movement - especially movement patterns that have been honed over a number of years - requires re-organization of the neuromuscular ...

Medicine

Virtual Reality Experiments with Physiological Measures

Virtual reality (VR) experiments are increasingly employed because of their internal and external validity compared to real-world observation and laboratory experiments, respectively. VR is especially useful for geographic visualizations and investigations of spatial behavior. In spatial behavior research, VR provides a platform for studying the relationship between navigation and physiological measures (e.g., skin conductance, heart rate, blood pressure). Specifically, physiological measures allow researchers to address novel questions and constrain previous theories of spatial abilities, strategies, and performance. For example, individual differences in navigation performance may be explained by the extent to which changes in arousal mediate the effects of task difficulty. However, the complexities in the design and implementation of VR experiments can distract experimenters from their primary research goals and introduce irregularities in data collection and analysis. To address these challenges, the Experiments in Virtual Environments (EVE) framework includes standardized modules such as participant training with the control interface, data collection using questionnaires, the synchronization of physiological measurements, and data storage. EVE also provides the necessary infrastructure for data management, visualization, and evaluation. The present paper describes a protocol that employs the EVE framework to conduct navigation experiments in VR with physiological sensors. The protocol lists the steps necessary for recruiting participants, attaching the physiological sensors, administering the experiment using EVE, and assessing the collected data with EVE evaluation tools. Overall, this protocol will facilitate future research by streamlining the design and implementation of VR experiments with physiological sensors.

Virtual reality (VR) experiments can be difficult to implement and require meticulous planning. This protocol describes a method for the design and implementation of VR experiments that collect physiological data from human participants. The Experiments in Virtual Environments (EVE) framework is employed to accelerate this process.

Virtual reality (VR) experiments are increasingly employed because of their internal and external validity compared to real-world observation and ...

The Immersive Cleveland Clinic Virtual Reality Shopping Platform for the Assessment of Instrumental Activities of Daily Living

A decline in the performance of instrumental activities of daily living (IADLs) has been proposed as a prodromal marker of neurological disease. Existing clinical and performance-based IADL assessments are not feasible for integration into clinical medicine. Virtual reality (VR) is a powerful yet underutilized tool that could advance the diagnosis and treatment of neurological disease. An impediment to the adoption and scaling of VR in clinical neurology is VR-related sickness resulting from sensory inconsistencies between the visual and vestibular systems (i.e., locomotion problem).
The Cleveland Clinic Virtual Reality Shopping (CC-VRS) platform attempts to solve the locomotion problem by coupling an omnidirectional treadmill with high-resolution VR content, enabling the user to physically navigate a virtual grocery store to simulate shopping. The CC-VRS consists of Basic and Complex shopping experiences; both require walking 150 m and retrieving five items. The Complex experience has additional scenarios that increase the cognitive and motor demands of the task to better represent the continuum of activities associated with real-world shopping. The CC-VRS platform provides objective and quantitative biomechanical and cognitive outcomes related to the user&#39;s IADL performance. Initial data indicate that the CC-VRS results in minimal VR-sickness and is feasible and tolerable for older adults and patients with Parkinson&#39;s disease (PD). The considerations underlying the development, design, and hardware and software technology are reviewed, and initial models of integration into primary care and neurology are provided.

Virtual reality (VR) is a powerful yet underutilized approach to advance the diagnosis and treatment of neurological disease. The Cleveland Clinic Virtual Reality Shopping platform combines state-of-the-art VR content with an omnidirectional treadmill to quantify instrumental activities of daily living-a proposed prodromal marker of neurological disease.

A decline in the performance of instrumental activities of daily living (IADLs) has been proposed as a prodromal marker of neurological disease. Existing ...

Engineering

Integrating VR and Biosensors for Quantitative Motion-Muscle Activity Assessment

Setup for the Quantitative Assessment of Motion and Muscle Activity During a Virtual Modified Box and Block Test

The ability to move allows us to interact with the world. When this ability is impaired, it can significantly reduce one's quality of life and independence and may lead to complications. The importance of remote patient evaluation and rehabilitation has recently grown due to limited access to in-person services. For example, the COVID-19 pandemic unexpectedly resulted in strict regulations, reducing access to non-emergent healthcare services. Additionally, remote care offers an opportunity to address healthcare disparities in rural, underserved, and low-income areas where access to services remains limited. 
Improving accessibility through remote care options would limit the number of hospital or specialist visits and render routine care more affordable. Finally, the use of readily available commercial consumer electronics for at-home care can enhance patient outcomes due to improved quantitative observation of symptoms, treatment efficacy, and therapy dosage. While remote care is a promising means to address these issues, there is a crucial need to quantitatively characterize motor impairment for such applications. The following protocol seeks to address this knowledge gap to enable clinicians and researchers to obtain high-resolution data on complex movement and underlying muscle activity. The ultimate goal is to develop a protocol for remote administration of functional clinical tests. 
Here, participants were instructed to perform a medically-inspired Box and Block task (BBT), which is frequently used to assess hand function. This task requires subjects to transport standardized cubes between two compartments separated by a barrier. We implemented a modified BBT in virtual reality to demonstrate the potential of developing remote assessment protocols. Muscle activation was captured for each subject using surface electromyography. This protocol allowed for the acquisition of high-quality data to better characterize movement impairment in a detailed and quantitative manner. Ultimately, these data have the potential to be used to develop protocols for virtual rehabilitation and remote patient monitoring.

Author Spotlight: Enhancing Remote Rehabilitation with Virtual Reality and Electromyography

The protocol described here aims to enhance the quantitative evaluation of upper limb deficits, with the goal of developing additional technology for remote assessment both in the clinic and at home. Virtual reality and biosensor technologies are combined with standard clinical techniques to provide insights into the functioning of the neuromuscular system.

The ability to move allows us to interact with the world. When this ability is impaired, it can significantly reduce one's quality of life and ...

Video Movement Analysis Using Smartphones (ViMAS): A Pilot Study

The use of smartphones in clinical practice is steadily increasing with the availability of low cost/freely available &#34;apps&#34; that could be used to assess human gait. The primary aim of this manuscript is to test the concurrent validity of kinematic measures recorded by a smartphone application in comparison to a 3D motion capture system in the sagittal plane. The secondary aim was to develop a protocol for clinicians on the set up of the smartphone camera for video movement analysis.
The sagittal plane knee angle was measured during heel strike and toe off events using the smart phone app and a 3D motion-capture system in 32 healthy subjects. Three trials were performed at near (2-m) and far (4-m) smartphone camera distances. The order of the distances was randomized. Regression analysis was performed to estimate the height of the camera based on either the subject&#39;s height or leg length.
Absolute measurement errors were least during toe off (3.12 &#177; 5.44 degrees) compared to heel strike (5.81 &#177; 5.26 degrees). There were significant (p &#60; 0.05) but moderate agreements between the application and 3D motion capture measures of knee angles. There were also no significant (p &#62; 0.05) differences between the absolute measurement errors between the two camera positions. The measurement errors averaged between 3 - 5 degrees during toe off and heel strike events of the gait cycle.
The use of smartphone apps can be a useful tool in the clinic for performing gait or human movement analysis. Further studies are needed to establish the accuracy in measuring movements of the upper extremity and trunk.

Video Movement Analysis Using Smartphones ViMAS: A Pilot Study

This manuscript describes the method to test the concurrent validity of kinematic measures recorded by the smartphone application in comparison to a 3D motion capture system in the sagittal plane. This protocol will enable clinicians to set up smartphones for video capture of human movement.

The use of smartphones in clinical practice is steadily increasing with the availability of low cost/freely available &#34;apps&#34; that could be used to ...

Biplanar Videoradiography to Study the Wrist and Distal Radioulnar Joints

Accurate measurement of skeletal kinematics in vivo is essential for understanding normal joint function, the influence of pathology, disease progression, and the effects of treatments. Measurement systems that use skin surface markers to infer skeletal motion have provided important insight into normal and pathological kinematics, however, accurate arthrokinematics cannot be attained using these systems, especially during dynamic activities. In the past two decades, biplanar videoradiography (BVR) systems have enabled many researchers to directly study the skeletal kinematics of the joints during activities of daily living. To implement BVR systems for the distal upper extremity, videoradiographs of the distal radius and the hand are acquired from two calibrated X-ray sources while a subject performs a designated task. Three-dimensional (3D) rigid-body positions are computed from the videoradiographs via a best-fit registrations of 3D model projections onto&#160;to each BVR view. The 3D models are density-based image volumes of the specific bone derived from independently acquired computed-tomography data. Utilizing graphics processor units and high-performance computing systems, this model-based tracking approach is shown to be fast and accurate in evaluating the wrist and distal radioulnar joint biomechanics. In this study, we first summarized the previous studies that have established the submillimeter and subdegree agreement of BVR with an in vitro&#160;optical motion capture system in evaluating the wrist and distal radioulnar joint kinematics. Furthermore, we used BVR to compute the center of rotation behavior of the wrist joint, to evaluate the articulation pattern of the components of the implant upon one another, and to assess the dynamic change of ulnar variance during pronosupination of the forearm. In the future, carpal bones may be captured in greater detail with the addition of flat panel X-ray detectors, more X-ray sources (i.e., multiplanar videoradiography), or advanced computer vision algorithms.

Biplanar videoradiography (BVR) is an advanced imaging technique for understanding the three-dimensional movement of skeletal bones and implants. Combining density-based image volumes and videoradiographs of the distal upper extremity, BVR is used to study the in vivo motion of the wrist and distal radioulnar joint, as well as joint arthroplasties.

Accurate measurement of skeletal kinematics in vivo is essential for understanding normal joint function, the influence of pathology, disease progression, ...

Bioengineering

Kinematic Analysis Using 3D Motion Capture of Drinking Task in People With and Without Upper-extremity Impairments

Kinematic analysis is a powerful method for objective assessment of upper extremity movements in a three-dimensional (3D) space. Three-dimensional motion capture with an optoelectronic camera system is considered as golden standard for kinematic movement analysis and is increasingly used as outcome measure to evaluate the movement performance and quality after an injury or disease involving upper extremity movements. This article describes a standardized protocol for kinematic analysis of drinking task applied in individuals with upper extremity impairments after stroke. The drinking task incorporates reaching, grasping and lifting a cup from a table to take a drink, placing the cup back, and moving the hand back to the edge of the table. The sitting position is standardized to the individual&#39;s body size and the task is performed in a comfortable self-paced speed and compensatory movements are not constrained. The intention is to keep the task natural and close to a real-life situation to improve the ecological validity of the protocol. A 5-camera motion capture system is used to gather 3D coordinate positions from 9 retroreflective markers positioned on anatomical landmarks of the arm, trunk, and face. A simple single marker placement is used to ensure the feasibility of the protocol in clinical settings. Custom-made Matlab software provides automated and fast analyses of movement data. Temporal kinematics of movement time, velocity, peak velocity, time of peak velocity, and smoothness (number of movement units) along with spatial angular kinematics of shoulder and elbow joint as well as trunk movements are calculated. The drinking task is a valid assessment for individuals with moderate and mild upper extremity impairment. The construct, discriminative and concurrent validity along with responsiveness (sensitivity to change) of the kinematic variables obtained from the drinking task have been established.

This protocol describes an objective method to evaluate the movement performance and sensorimotor function of the upper extremity applied to individuals with stroke and healthy controls. A standardized test procedure, kinematic analysis and outcome variables for three-dimensional motion capture of drinking task are provided.

Kinematic analysis is a powerful method for objective assessment of upper extremity movements in a three-dimensional (3D) space. Three-dimensional motion ...

Neuroscience

Comprehensive Understanding of Inactivity-Induced Gait Alteration in Rodents

It is well known that disuse affects neural systems and that joint motions become altered; however, which outcomes properly exhibit these characteristics is still unclear. The present study describes a motion analysis approach that utilizes three-dimensional (3D) reconstruction from video captures. Using this technology, disuse-evoked alterations of walking performances were observed in rodents exposed to a simulated microgravity environment by unloading their hindlimb by their tail. After 2 weeks of unloading, the rats walked on a treadmill, and their gait motions were captured with four charge-coupled device (CCD) cameras. 3D motion profiles were reconstructed and compared to those of control subjects using the image processing software. The reconstructed outcome measures successfully portrayed distinct aspects of distorted gait motion: hyperextension of the knee and ankle joints and higher position of the hip joints during the stance phase. Motion analysis is useful for several reasons. First, it enables quantitative behavioral evaluations instead of subjective observations (e.g., pass/fail in certain tasks). Second, multiple parameters can be extracted to fit specific needs once the fundamental datasets are obtained. Despite hurdles for broader application, the disadvantages of this method, including labor intensity and cost, may be alleviated by determining comprehensive measurements and experimental procedures.

The present protocol describes three-dimensional motion tracking/evaluation to depict gait motion alteration of rats after exposure to a simulated disuse environment.

It is well known that disuse affects neural systems and that joint motions become altered; however, which outcomes properly exhibit these characteristics ...