JoVE Logo

Zaloguj się

Aby wyświetlić tę treść, wymagana jest subskrypcja JoVE. Zaloguj się lub rozpocznij bezpłatny okres próbny.

W tym Artykule

  • Podsumowanie
  • Streszczenie
  • Wprowadzenie
  • Protokół
  • Wyniki
  • Dyskusje
  • Ujawnienia
  • Podziękowania
  • Materiały
  • Odniesienia
  • Przedruki i uprawnienia

Podsumowanie

A protocol is proposed to capture natural hand function of individuals with hand impairments during their daily routines using an egocentric camera. The goal of the protocol is to ensure that the recordings are representative of an individual's typical hand use during activities of daily living at home.

Streszczenie

Impaired hand function after neurological injuries can have a major impact on independence and quality of life. Most existing upper limb assessments are carried out in person, which is not always indicative of hand use in the community. Novel approaches to capture hand function in daily life are required to measure the true impact of rehabilitation interventions. Egocentric video combined with computer vision for automated analysis has been proposed to evaluate hand use at home. However, there are limitations to the duration of continuous recordings. We present a protocol designed to ensure that the videos obtained are representative of daily routines while respecting participant privacy.

A representative recording schedule is selected through a collaborative process between the researchers and participants, to ensure that the videos capture natural tasks and performance, while being useful for hand assessment. Use of the equipment and procedures is demonstrated to the participants. A total of 3 h of video recordings are scheduled over two weeks. To reduce privacy concerns, participants have full control to start and stop recordings, and the opportunity to edit the videos before returning them to the research team. Reminders are provided, as well as help calls and home visits if necessary.

The protocol was tested with 9 stroke survivors and 14 individuals with cervical spinal cord injury. The videos obtained contained a variety of activities, such as meal preparation, dishwashing, and knitting. An average of 3.11 ± 0.98 h of video were obtained. The recording periods varied from 12-69 d, due to illness or unexpected events in some cases. Data was successfully obtained from twenty-two out of 23 participants, with 6 participants requiring assistance from the investigators during the home recording period. The protocol was effective for collecting videos that contained valuable information about hand function at home after neurological injuries.

Wprowadzenie

Hand function is a determinant of independence and quality of life across clinical populations with upper limb impairments1,2. Capturing the hand function of individuals with hand impairments at home is vital to evaluating the progress of their ability to carry out activities of daily living (ADLs) during and after rehabilitation. Most clinical hand function assessments are conducted in a clinical or laboratory environment, rather than at home3,4. Existing clinical hand function assessments that seek to capture the impact on ADLs at home are questionnaires and rely on subjective self-reported ratings5,6,7. An objective evaluation to assess the ultimate impact of rehabilitation on hand function at home is still unavailable.

In recent years, many wearable technologies have been developed and implemented to capture upper limb function in real-world environments. Wearable sensors such as accelerometers and inertial measurement units (IMUs) have been commonly used to measure upper limb movements in daily life. However, these devices typically do not distinguish whether the detected epochs belong to functional upper limb movements8,9, defined as purposeful movements intended to complete a desired task. For example, some wearable sensors are sensitive to the presence of upper limb swings during walking, which is not a functional movement of the upper limb. Furthermore, although wrist-worn accelerometers capture upper limb movements, they cannot capture the details of hand function in real-world environments. Sensorized gloves allow capturing more detailed information about hand manipulations10, but they may be cumbersome for people whose hand function and sensation are already impaired. Wearable approaches have also been proposed to capture finger movements through magnetometry or finger-worn accelerometers11,12,13, but the functional interpretation of those movements remains challenging14. Thus, although previously proposed wearable devices are small and convenient to use, they are insufficient to describe the details and functional context of hand use.

Wearable cameras have been proposed to fill these gaps and capture details of hand function during ADLs at home for neurorehabilitation applications15,16,17,18,19. Automated analysis of egocentric videos using computer vision has considerable potential to quantify hand function in context, by providing information both about the hands themselves and about the tasks carried out in real ADLs20. On the other hand, the duration of continuous recordings is typically limited to approximately 1 to 1.5 h by battery, storage, and comfort considerations. Here, within these constraints, we present an egocentric video collection protocol intended to obtain data that is both representative of an individual's daily life as well as informative for hand function evaluation.

Protokół

The study was approved by the Research Ethics Board of the University Health Network. Signed informed consent was obtained from each participant before enrollment in the study. Signed informed consent was also obtained from any caregivers or household members appearing in video recordings.

1. Verification of the protocol applicability to the individual

NOTE: This protocol is intended to be applied to individuals with impaired but not completely absent hand function (specific criteria can be adapted to the population and/or question of interest).

  1. Ask participants whether their affected hands impact their ability to carry out ADLs.
    NOTE: It is recommended to ask participants to give some examples of tasks that they can and cannot perform independently with their affected hands.
  2. Verify that the total score on the Montreal Cognitive Assessment (MoCA) is above 21, in order to avoid potential difficulties understanding and following protocol procedures.

2. Determination of the daily routine of participants

  1. Ask participants to recall their daily routines over the past two weeks. Document which daily tasks are performed, for how long, and at approximately what time.
  2. In collaboration with participants, select 3 timeslots of 1.5 h each during which to record videos. Select timeslots that are spread over different days of the week, and take place when ADLs involving the hands are typically carried out in sequence.
    NOTE: The selected ADLs must be representative of each participant's typical activities, and be perceived by them as meaningful. Scheduling recording periods on different days is intended to increase the variety of recorded ADLs and to promote the capture of useful and meaningful data.
    ​NOTE: Recording timeslots are scheduled for recording efficiency, but participants should understand that they have full control of when to start and to stop recordings.

3. Agreement on recording schedules and target video content with participants

  1. Obtain agreement of each participant on the recording schedules, after discussing any concerns that they may have.
  2. Set a goal of 3 h of videos over two weeks. Inform participants that insufficient videos may lead to extending their recording periods.

4. Emphasis of the importance of performing ADLs naturally

  1. Instruct participants to focus on capturing realistic routines, instead of specifying particular activities to record. The intent of the instruction is to discourage participants from artificially recording specific activities in greater amounts than is typical for them.

5. Notification of potential privacy issues during recordings at home

  1. Ensure that participants understand that all recordings should take place inside their homes, not in public places to avoid privacy issues.
  2. Give some examples that may raise privacy concerns, such as bathing, dressing/undressing, and checking confidential information. Remind participants to be aware of mirrors, which may show their faces in the recordings.
  3. Suggest that participants avoid the presence of other people such as family members or caregivers as much as possible in the videos.
    ​NOTE: In the context of research studies, in cases where the presence of other people is unavoidable, informed consent should be obtained from those individuals.

6. Camera and tablet instruction

NOTE: If participants indicate during the initial contact that they require caregiver assistance for many of their daily needs, the caregiver is encouraged to also attend the study visit and be trained on the use of the equipment, so that they can later assist the participant.

  1. Demonstrate how to use an egocentric camera (Table of Materials) to participants.
    1. Demonstrate how to turn the camera on and off.
    2. Demonstrate how to control recordings (start, pause, stop) using the camera.
  2. Demonstrate how to use a tablet (Table of Materials) with the preinstalled camera app to control the recordings, if applicable.
    ​NOTE: The demonstration includes controlling the recordings from the camera app, as well as replaying and editing (e.g., trim or delete) the recorded videos. A camera remote was initially considered (Supplemental Files), but in practice was not used because participants were comfortable using the camera or tablet to start and stop recordings.
    1. Demonstrate how to turn on and off the tablet.
    2. Demonstrate how to connect the tablet to the camera through the camera app.
    3. Demonstrate how to control the recordings from the camera app.
    4. Demonstrate how to review recorded videos from the camera app.
    5. Demonstrate how to trim or delete the videos from the camera app.
  3. Demonstrate how to don and doff the camera using an elastic headband adjustable to the participant's head.
    NOTE: See Figure 1.
    figure-protocol-5393
    Figure 1. Wearable camera setup. (A) Positioning of the egocentric camera. (B) Viewing angle from the camera. Please click here to view a larger version of this figure.
    1. Place the camera on the participant's forehead. Adjust the headband to wear the camera comfortably and steadily.
    2. Ensure an optimal angle of the camera with respect to the forehead.
    3. Ask participants to record a short segment of video while moving the hands in front of them and manipulating an object (e.g., the tablet).
    4. Review the recorded video and ensure the two hands were clearly visible in the central region of the scene while conducting manipulation tasks.
    5. Practice the use of the camera and tablet with participants and their caregivers, until they demonstrate proficiency.

7. Giving the equipment

  1. Give the kit with all the equipment to participants to record their ADLs at home. In addition to the camera and tablet, the kit includes extra camera batteries, battery chargers for both camera and tablet, charging cables, headband for the camera, and a printed set of guidelines for using the camera (See the Supplemental Material).

8. Experimental troubleshooting and followup

  1. Provide contact information of the researchers to help solve obstacles during the actual recordings at home. After one week, researchers call the participants to document the recording progress and solve any potential technical issues.

9. Retrieval of equipment and videos

  1. Retreive all the equipment and videos from participants in person or through pre-paid mail parcels.
  2. Ensure that participants agree to share all the videos returned. Participants are encouraged to review all of the collected videos before returning them to the research team, and to delete any portions that they do not wish to share.
  3. For research studies, review the returned videos and check if anyone appears in the video without having given their consent. If so, send consent forms or call the individuals who appear in the videos to gain their consents for use of the videos. If the individuals are not reachable, the portions of the videos in which they appear are deleted by the researchers.

Wyniki

Participant demographics and inclusion criteria
A sample of 23 participants was recruited for these studies: 9 stroke survivors (6 men, 3 women) and 14 individuals with cSCI (12 men, 2 women). Summary demographic and clinical information for the recruited sample are reported in Table 1.

<...
Age (years)

Dyskusje

We presented a protocol for recording videos of ADLs at home using wearable cameras in individuals with upper limb impairments, such as cSCI and stroke. The protocol is flexible and can be goal-directed to capture hand function performance in specific ADLs or to track the progress of rehabilitation remotely in people living at home. The egocentric vision paradigm has great potential for remote monitoring of hand function in individuals living in the community, and for optimizing rehabilitation once people are discharged ...

Ujawnienia

The authors have nothing to disclose.

Podziękowania

The studies using this protocol were funded by the Heart and Stroke Foundation (G-18-0020952), the Craig H. Neilsen Foundation (542675), the Natural Sciences and Engineering Research Council of Canada (RGPIN-2014-05498), and Ministry of Research, Innovation and Science, Ontario (ER16-12-013).

Materiały

NameCompanyCatalog NumberComments
Egocentric cameraGoPro Inc., CA, USAGoPro Hero 4 and 5A camera that records from a first-person angle.
Battery chager and batteriesGoPro Inc., CA, USAMAX Dual Battery Charger + BatteryExtra batteries for the camera and battery charger
Camera chargerGoPro Inc., CA, USASuperchargerThis charger is connected to the camera directly without disassembling the camera frame.
Camera frameGoPro Inc., CA, USAThe FrameThe hinge of the camera frame can be used to adjust the angle of view of the camera.
Headband for the cameraGoPro Inc., CA, USAHead Strap + QuickClip
SD cardSanDisk, CA, USA32GB microSD
TabletASUSTeK Computer Inc., TaiwanZenPad 8.0 Z380MThe tablet is installed with the GoPro App in order to connect with the camera.

Odniesienia

  1. Nichols-Larsen, D. S., Clark, P., Zeringue, A., Greenspan, A., Blanton, S. Factors influencing stroke survivors' quality of life during subacute recovery. Stroke. 36 (7), 1480-1484 (2005).
  2. Anderson, K. D. Targeting recovery: priorities of the spinal cord-injured population. Journal of Neurotrauma. 21 (10), 1371-1383 (2004).
  3. Gladstone, D. J., Danells, C. J., Black, S. E. The Fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties. Neurorehabilitation and Neural Repair. 16 (3), 232-240 (2002).
  4. Barreca, S. R., Stratford, P. W., Lambert, C. L., Masters, L. M., Streiner, D. L. Test-retest reliability, validity, and sensitivity of the Chedoke arm and hand activity inventory: a new measure of upper-limb function for survivors of stroke. Archives of Physical Medicine and Rehabilitation. 86 (8), 1616-1622 (2005).
  5. Uswatte, G., Taub, E., Morris, D., Vignolo, M., McCulloch, K. Reliability and validity of the upper-extremity Motor Activity Log-14 for measuring real-world arm use. Stroke. 36 (11), 2493-2496 (2005).
  6. Duncan, P. W., Bode, R. K., Lai, S. M., Perera, S., Antagonist, G. Rasch analysis of a new stroke-specific outcome scale: the Stroke Impact Scale. Archives of Physical Medicine and Rehabilitation. 84 (7), 950-963 (2003).
  7. Marino, R. J., Shea, J. A., Stineman, M. G. The capabilities of upper extremity instrument: reliability and validity of a measure of functional limitation in tetraplegia. Archives of Physical Medicine and Rehabilitation. 79 (12), 1512-1521 (1998).
  8. Hayward, K. S., et al. Exploring the role of accelerometers in the measurement of real world upper-limb use after stroke. Brain Impairment. 17 (1), 16-33 (2016).
  9. van der Pas, S. C., Verbunt, J. A., Breukelaar, D. E., van Woerden, R., Seelen, H. A. Assessment of arm activity using triaxial accelerometry in patients with a stroke. Archives of Physical Medicine and Rehabilitation. 92 (9), 1437-1442 (2011).
  10. Oess, N. P., Wanek, J., Curt, A. Design and evaluation of a low-cost instrumented glove for hand function assessment. Journal of Neuroengineering and Rehabilitation. 9 (1), 2 (2012).
  11. Friedman, N., Rowe, J. B., Reinkensmeyer, D. J., Bachman, M. The manumeter: a wearable device for monitoring daily use of the wrist and fingers. IEEE Journal of Biomedical Health Informatics. 18 (6), 1804-1812 (2014).
  12. Liu, X., Rajan, S., Ramasarma, N., Bonato, P., Lee, S. I. The use of a finger-worn accelerometer for monitoring of hand use in ambulatory settings. IEEE Journal of Biomedical Health Informatics. 23 (2), 599-606 (2018).
  13. Lee, S. I., et al. A novel upper-limb function measure derived from finger-worn sensor data collected in a free-living setting. PloS One. 14 (3), (2019).
  14. Rowe, J. B., et al. The variable relationship between arm and hand use: a rationale for using finger magnetometry to complement wrist accelerometry when measuring daily use of the upper extremity. 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. , 4087-4090 (2014).
  15. Dousty, M., Zariffa, J. Tenodesis Grasp Detection in Egocentric Video. IEEE Journal of Biomedical and health. , (2020).
  16. Likitlersuang, J., et al. Egocentric video: a new tool for capturing hand use of individuals with spinal cord injury at home. Journal of Neuroengineering and Rehabilitation. 16 (1), 83 (2019).
  17. Tsai, M. -. F., Wang, R. H., Zariffa, J. Generalizability of Hand-Object Interaction Detection in Egocentric Video across Populations with Hand Impairment. 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). , 3228-3231 (2020).
  18. Bandini, A., Dousty, M., Zariffa, J. A wearable vision-based system for detecting hand-object interactions in individuals with cervical spinal cord injury: First results in the home environment. 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). , 2159-2162 (2020).
  19. Dousty, M., Zariffa, J. Towards Clustering Hand Grasps of Individuals with Spinal Cord Injury in Egocentric Video. 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). , 2151-2154 (2020).
  20. Bandini, A., Zariffa, J. Analysis of the hands in egocentric vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. , (2020).
  21. Likitlersuang, J., Sumitro, E. R., Theventhiran, P., Kalsi-Ryan, S., Zariffa, J. Views of individuals with spinal cord injury on the use of wearable cameras to monitor upper limb function in the home and community. Journal of Spinal Cord Medicine. 40 (6), 706-714 (2017).

Przedruki i uprawnienia

Zapytaj o uprawnienia na użycie tekstu lub obrazów z tego artykułu JoVE

Zapytaj o uprawnienia

Przeglądaj więcej artyków

Egocentric VideoUpper Limb ImpairmentRehabilitation ProtocolDaily RoutineActivities Of Daily LivingADLsWearable CamerasParticipant CommunicationVideo RecordingTherapeutic EvaluationData CaptureHome EnvironmentRealistic RoutinesPrivacy ConcernsInstructional Demonstration

This article has been published

Video Coming Soon

JoVE Logo

Prywatność

Warunki Korzystania

Zasady

Badania

Edukacja

O JoVE

Copyright © 2025 MyJoVE Corporation. Wszelkie prawa zastrzeżone