The studies using this protocol were funded by the Heart and Stroke Foundation (G-18-0020952), the Craig H. Neilsen Foundation (542675), the Natural Sciences and Engineering Research Council of Canada (RGPIN-2014-05498), and Ministry of Research, Innovation and Science, Ontario (ER16-12-013).

The study was approved by the Research Ethics Board of the University Health Network. Signed informed consent was obtained from each participant before enrollment in the study. Signed informed consent was also obtained from any caregivers or household members appearing in video recordings.
1. Verification of the protocol applicability to the individual
NOTE: This protocol is intended to be applied to individuals with impaired but not completely absent hand function (s...

<ol>
	<li>Nichols-Larsen, D. S., Clark, P., Zeringue, A., Greenspan, A., Blanton, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Factors+influencing+stroke+survivors'+quality+of+life+during+subacute+recovery.">Factors influencing stroke survivors' quality of life during subacute recovery.</a> Stroke. 36 (7), 1480-1484 (2005).</li><li>Anderson, K. D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Targeting+recovery:+priorities+of+the+spinal+cord-injured+population.">Targeting recovery: priorities of the spinal cord-injured population.</a> Journal of Neurotrauma. 21 (10), 1371-1383 (2004).</li><li>Gladstone, D. J., Danells, C. J., Black, S. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+Fugl-Meyer+assessment+of+motor+recovery+after+stroke:+a+critical+review+of+its+measurement+properties.">The Fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties.</a> Neurorehabilitation and Neural Repair. 16 (3), 232-240 (2002).</li><li>Barreca, S. R., Stratford, P. W., Lambert, C. L., Masters, L. M., Streiner, D. L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Test-retest+reliability,+validity,+and+sensitivity+of+the+Chedoke+arm+and+hand+activity+inventory:+a+new+measure+of+upper-limb+function+for+survivors+of+stroke.">Test-retest reliability, validity, and sensitivity of the Chedoke arm and hand activity inventory: a new measure of upper-limb function for survivors of stroke.</a> Archives of Physical Medicine and Rehabilitation. 86 (8), 1616-1622 (2005).</li><li>Uswatte, G., Taub, E., Morris, D., Vignolo, M., McCulloch, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Reliability+and+validity+of+the+upper-extremity+Motor+Activity+Log-14+for+measuring+real-world+arm+use.">Reliability and validity of the upper-extremity Motor Activity Log-14 for measuring real-world arm use.</a> Stroke. 36 (11), 2493-2496 (2005).</li><li>Duncan, P. W., Bode, R. K., Lai, S. M., Perera, S., Antagonist, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Rasch+analysis+of+a+new+stroke-specific+outcome+scale:+the+Stroke+Impact+Scale.">Rasch analysis of a new stroke-specific outcome scale: the Stroke Impact Scale.</a> Archives of Physical Medicine and Rehabilitation. 84 (7), 950-963 (2003).</li><li>Marino, R. J., Shea, J. A., Stineman, M. G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+capabilities+of+upper+extremity+instrument:+reliability+and+validity+of+a+measure+of+functional+limitation+in+tetraplegia.">The capabilities of upper extremity instrument: reliability and validity of a measure of functional limitation in tetraplegia.</a> Archives of Physical Medicine and Rehabilitation. 79 (12), 1512-1521 (1998).</li><li>Hayward, K. S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Exploring+the+role+of+accelerometers+in+the+measurement+of+real+world+upper-limb+use+after+stroke.">Exploring the role of accelerometers in the measurement of real world upper-limb use after stroke.</a> Brain Impairment. 17 (1), 16-33 (2016).</li><li>van der Pas, S. C., Verbunt, J. A., Breukelaar, D. E., van Woerden, R., Seelen, H. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Assessment+of+arm+activity+using+triaxial+accelerometry+in+patients+with+a+stroke.">Assessment of arm activity using triaxial accelerometry in patients with a stroke.</a> Archives of Physical Medicine and Rehabilitation. 92 (9), 1437-1442 (2011).</li><li>Oess, N. P., Wanek, J., Curt, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Design+and+evaluation+of+a+low-cost+instrumented+glove+for+hand+function+assessment.">Design and evaluation of a low-cost instrumented glove for hand function assessment.</a> Journal of Neuroengineering and Rehabilitation. 9 (1), 2 (2012).</li><li>Friedman, N., Rowe, J. B., Reinkensmeyer, D. J., Bachman, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+manumeter:+a+wearable+device+for+monitoring+daily+use+of+the+wrist+and+fingers.">The manumeter: a wearable device for monitoring daily use of the wrist and fingers.</a> IEEE Journal of Biomedical Health Informatics. 18 (6), 1804-1812 (2014).</li><li>Liu, X., Rajan, S., Ramasarma, N., Bonato, P., Lee, S. I. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+use+of+a+finger-worn+accelerometer+for+monitoring+of+hand+use+in+ambulatory+settings.">The use of a finger-worn accelerometer for monitoring of hand use in ambulatory settings.</a> IEEE Journal of Biomedical Health Informatics. 23 (2), 599-606 (2018).</li><li>Lee, S. I., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+novel+upper-limb+function+measure+derived+from+finger-worn+sensor+data+collected+in+a+free-living+setting.">A novel upper-limb function measure derived from finger-worn sensor data collected in a free-living setting.</a> PloS One. 14 (3), (2019).</li><li>Rowe, J. B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+variable+relationship+between+arm+and+hand+use:+a+rationale+for+using+finger+magnetometry+to+complement+wrist+accelerometry+when+measuring+daily+use+of+the+upper+extremity.">The variable relationship between arm and hand use: a rationale for using finger magnetometry to complement wrist accelerometry when measuring daily use of the upper extremity.</a> 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. , 4087-4090 (2014).</li><li>Dousty, M., Zariffa, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Tenodesis+Grasp+Detection+in+Egocentric+Video.">Tenodesis Grasp Detection in Egocentric Video.</a> IEEE Journal of Biomedical and health. , (2020).</li><li>Likitlersuang, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Egocentric+video:+a+new+tool+for+capturing+hand+use+of+individuals+with+spinal+cord+injury+at+home.">Egocentric video: a new tool for capturing hand use of individuals with spinal cord injury at home.</a> Journal of Neuroengineering and Rehabilitation. 16 (1), 83 (2019).</li><li>Tsai, M. -. F., Wang, R. H., Zariffa, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Generalizability+of+Hand-Object+Interaction+Detection+in+Egocentric+Video+across+Populations+with+Hand+Impairment.">Generalizability of Hand-Object Interaction Detection in Egocentric Video across Populations with Hand Impairment.</a> 2020 42nd Annual International Conference of the IEEE Engineering in Medicine &amp; Biology Society (EMBC). , 3228-3231 (2020).</li><li>Bandini, A., Dousty, M., Zariffa, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+wearable+vision-based+system+for+detecting+hand-object+interactions+in+individuals+with+cervical+spinal+cord+injury:+First+results+in+the+home+environment.">A wearable vision-based system for detecting hand-object interactions in individuals with cervical spinal cord injury: First results in the home environment.</a> 2020 42nd Annual International Conference of the IEEE Engineering in Medicine &amp; Biology Society (EMBC). , 2159-2162 (2020).</li><li>Dousty, M., Zariffa, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Towards+Clustering+Hand+Grasps+of+Individuals+with+Spinal+Cord+Injury+in+Egocentric+Video.">Towards Clustering Hand Grasps of Individuals with Spinal Cord Injury in Egocentric Video.</a> 2020 42nd Annual International Conference of the IEEE Engineering in Medicine &amp; Biology Society (EMBC). , 2151-2154 (2020).</li><li>Bandini, A., Zariffa, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Analysis+of+the+hands+in+egocentric+vision:+A+survey.">Analysis of the hands in egocentric vision: A survey.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. , (2020).</li><li>Likitlersuang, J., Sumitro, E. R., Theventhiran, P., Kalsi-Ryan, S., Zariffa, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Views+of+individuals+with+spinal+cord+injury+on+the+use+of+wearable+cameras+to+monitor+upper+limb+function+in+the+home+and+community.">Views of individuals with spinal cord injury on the use of wearable cameras to monitor upper limb function in the home and community.</a> Journal of Spinal Cord Medicine. 40 (6), 706-714 (2017).</li></ol>

We presented a protocol for recording videos of ADLs at home using wearable cameras in individuals with upper limb impairments, such as cSCI and stroke. The protocol is flexible and can be goal-directed to capture hand function performance in specific ADLs or to track the progress of rehabilitation remotely in people living at home. The egocentric vision paradigm has great potential for remote monitoring of hand function in individuals living in the community, and for optimizing rehabilitation once people are discharged ...

Hand function is a determinant of independence and quality of life across clinical populations with upper limb impairments1,2. Capturing the hand function of individuals with hand impairments at home is vital to evaluating the progress of their ability to carry out activities of daily living (ADLs) during and after rehabilitation. Most clinical hand function assessments are conducted in a clinical or laboratory environment, rather than at home3,4. Existing clinical hand function assessments that seek to capture the impact on ADLs at home are questionnaires and rely on subjective self-reported ratings5,6,7. An objective evaluation to assess the ultimate impact of rehabilitation on hand function at home is still unavailable.
In recent years, many wearable technologies have been developed and implemented to capture upper limb function in real-world environments. Wearable sensors such as accelerometers and inertial measurement units (IMUs) have been commonly used to measure upper limb movements in daily life. However, these devices typically do not distinguish whether the detected epochs belong to functional upper limb movements8,9, defined as purposeful movements intended to complete a desired task. For example, some wearable sensors are sensitive to the presence of upper limb swings during walking, which is not a functional movement of the upper limb. Furthermore, although wrist-worn accelerometers capture upper limb movements, they cannot capture the details of hand function in real-world environments. Sensorized gloves allow capturing more detailed information about hand manipulations10, but they may be cumbersome for people whose hand function and sensation are already impaired. Wearable approaches have also been proposed to capture finger movements through magnetometry or finger-worn accelerometers11,12,13, but the functional interpretation of those movements remains challenging14. Thus, although previously proposed wearable devices are small and convenient to use, they are insufficient to describe the details and functional context of hand use.
Wearable cameras have been proposed to fill these gaps and capture details of hand function during ADLs at home for neurorehabilitation applications15,16,17,18,19. Automated analysis of egocentric videos using computer vision has considerable potential to quantify hand function in context, by providing information both about the hands themselves and about the tasks carried out in real ADLs20. On the other hand, the duration of continuous recordings is typically limited to approximately 1 to 1.5 h by battery, storage, and comfort considerations. Here, within these constraints, we present an egocentric video collection protocol intended to obtain data that is both representative of an individual&#39;s daily life as well as informative for hand function evaluation.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Egocentric camera</td>
			<td>GoPro Inc., CA, USA</td>
			<td>GoPro Hero 4 and 5</td>
			<td>A camera that records from a first-person angle.</td>
		</tr>
		<tr>
			<td>Battery chager and batteries</td>
			<td>GoPro Inc., CA, USA</td>
			<td>MAX Dual Battery Charger + Battery</td>
			<td>Extra batteries for the camera and battery charger</td>
		</tr>
		<tr>
			<td>Camera charger</td>
			<td>GoPro Inc., CA, USA</td>
			<td>Supercharger</td>
			<td>This charger is connected to the camera directly without disassembling the camera frame.</td>
		</tr>
		<tr>
			<td>Camera frame</td>
			<td>GoPro Inc., CA, USA</td>
			<td>The Frame</td>
			<td>The hinge of the camera frame can be used to adjust the angle of view of the camera.</td>
		</tr>
		<tr>
			<td>Headband for the camera</td>
			<td>GoPro Inc., CA, USA</td>
			<td>Head Strap + QuickClip</td>
			<td></td>
		</tr>
		<tr>
			<td>SD card</td>
			<td>SanDisk, CA, USA</td>
			<td>32GB microSD</td>
			<td></td>
		</tr>
		<tr>
			<td>Tablet</td>
			<td>ASUSTeK Computer Inc., Taiwan</td>
			<td>ZenPad 8.0 Z380M</td>
			<td>The tablet is installed with the GoPro App in order to connect with the camera.</td>
		</tr>
	</tbody>
</table>

capturing representative hand use at home using egocentric video in individuals with upper limb impairment

Impaired hand function after neurological injuries can have a major impact on independence and quality of life. Most existing upper limb assessments are carried out in person, which is not always indicative of hand use in the community. Novel approaches to capture hand function in daily life are required to measure the true impact of rehabilitation interventions. Egocentric video combined with computer vision for automated analysis has been proposed to evaluate hand use at home. However, there are limitations to the duration of continuous recordings. We present a protocol designed to ensure that the videos obtained are representative of daily routines while respecting participant privacy. 
A representative recording schedule is selected through a collaborative process between the researchers and participants, to ensure that the videos capture natural tasks and performance, while being useful for hand assessment. Use of the equipment and procedures is demonstrated to the participants. A total of 3 h of video recordings are scheduled over two weeks. To reduce privacy concerns, participants have full control to start and stop recordings, and the opportunity to edit the videos before returning them to the research team. Reminders are provided, as well as help calls and home visits if necessary. 
The protocol was tested with 9 stroke survivors and 14 individuals with cervical spinal cord injury. The videos obtained contained a variety of activities, such as meal preparation, dishwashing, and knitting. An average of 3.11 &plusmn; 0.98 h of video were obtained. The recording periods varied from 12-69 d, due to illness or unexpected events in some cases. Data was successfully obtained from twenty-two out of 23 participants, with 6 participants requiring assistance from the investigators during the home recording period. The protocol was effective for collecting videos that contained valuable information about hand function at home after neurological injuries.

Participant demographics and inclusion criteria 
A sample of 23 participants was recruited for these studies: 9 stroke survivors (6 men, 3 women) and 14 individuals with cSCI (12 men, 2 women). Summary demographic and clinical information for the recruited sample are reported in Table 1.
<table border="1" fo:keep-together.within-page="always">
	<tbody>
		<tr>
			<td rowspan="2"></td>
			<td rowspan="2">Age (years)</td>
			<...

Watch this Scientific Journal Video about Capturing Representative Hand Use at Home Using Egocentric Video in Individuals with Upper Limb Impairment at JoVE.com

Capturing Representative Hand Use at Home Using Egocentric Video in Individuals with Upper Limb Impairment

A protocol is proposed to capture natural hand function of individuals with hand impairments during their daily routines using an egocentric camera. The goal of the protocol is to ensure that the recordings are representative of an individual&#39;s typical hand use during activities of daily living at home.

Impaired hand function after neurological injuries can have a major impact on independence and quality of life. Most existing upper limb assessments are ...

This protocol captures a representative daily routine of individuals with upper limb impairments using wearable cameras or egocentric videos. Its purpose is to evaluate and use at home for rehabilitation applications. The imaged hand use can help to evaluate the impact of new therapies in the home environments.The protocol is designed to efficiently capture data representative of the user's daily life. The information obtained using this protocol can be used to evaluate the impact of rehabilitation, track progress over time or guide remote care. Good communications with participant is essential.Ensure that they understand the importance of not changing their usual routine or the manner in which they perform tasks. Begin by asking the participant whether their affected hands impact their ability to carry out activities of daily living, or ADLs. Ask the participant to give some examples of tasks that they can and cannot perform independently with their affected hands.Ask participants to recall their daily routines over the past two weeks. Document which daily tasks are performed, for how long, and at approximately what time. In collaboration with the participant, select three time slots of 90 minutes each, during which to record videos.Select time slots that are spread over different days of the week and take place when ADLs involving the hands are typically carried out in sequence. Obtain agreement of the participant on the recording schedules after discussing any concerns that they may have. Set a goal of three hours of videos over two weeks and inform the participant that insufficient videos may lead to extending their recording periods.Instruct the participant to focus on capturing realistic routines. Ensure that the participant understands that all recordings should take place inside their homes, not in public places. Give some examples that may raise privacy concerns, such as bathing, dressing and checking confidential information.Remind the participant to be aware of mirrors, which may show their face in the recordings. Suggest that the participant avoids the presence of other people, such as family members or caregivers, as much as possible in the videos. Demonstrate how to use an egocentric camera.Specifically, show the participant how to turn the camera on and off and how to control recordings. Ask the participant to practice turning the camera on and off, then to try to record from the camera. If applicable, demonstrate how to use a tablet with the pre-installed camera app.Demonstrate how to turn the tablet on and off, how to connect the tablet to the camera through the camera app, and how to control the recordings. Ask the participant to practice the demonstrated steps. Then, show them how to review the recorded videos and how to trim or delete them.Have the participant practice reviewing, trimming and deleting a recording from the tablet. Next, demonstrate how to don or doff the camera using an elastic headband adjustable to the participant's head. Place the camera on the participant's forehead and adjust the headband to wear the camera comfortably and steadily.Ensure an optimal angle of the camera with respect to the forehead. Ask the participant to record a short segment of video while moving their hands in front of them and manipulating an object. Review the recorded video and ensure that the two hands are clearly visible in the central region of the scene while conducting manipulation tasks.Practice the use of the camera and tablet with the participant and their caregivers until they demonstrate proficiency. Give the participant the kit with the equipment to record their ADLs at home. In addition to the camera and tablet, the kit should include extra camera batteries, battery chargers for both camera and tablet, charging cables, headband for the camera and a printed set of guidelines for using the camera.A sample of 23 participants was recruited, including nine stroke survivors and 14 individuals with spinal cord injury. The proposed protocol was successful for 95.7%of the participants. Recorded activities included meal preparation, eating, dishwashing, physical activity and knitting.The daily average video recording length per participant was 60 minutes, with an average of three hours of video obtained per participant. Some participants reported feeling tired and uncomfortable wearing the camera for over one hour, due to its weight and heat against the forehead. 27%of participants required assistance from the research team during the recording periods at home.On average, each help call took five to 10 minutes. In addition, 15 caregivers of participants were involved in helping to record videos at home. Selecting the activities needs to be collaborative process between the researcher and participants to ensure that realistic, representative and meaningful routines are captured.The data obtained from these videos can be used for automatic analysis, through computer vision and machine learning techniques to create new tools for tracking rehabilitation progress and outcomes.

capturing-representative-hand-use-at-home-using-egocentric-video

Research

JoVE Journal

Bioengineering

2.4K vistas.  University of Toronto. This protocol captures a representative daily routine of individuals with upper limb impairments using wearable cameras or egocentric videos. Its purpose is to evaluate and use at home for rehabilitation applications. The imaged hand use can help to evaluate the impact of new therapies in the home environments.The protocol is designed to efficiently capture data representative of the user's daily life. The information obtained using this protocol can be used to evaluate the impact of r...