A subscription to JoVE is required to view this content. Sign in or start your free trial.
Here we present a protocol which characterizes the sense of agency developed over the control of sensate virtual or robotic prosthetic hands. Psychophysical questionnaires are employed to capture the explicit experience of agency, and time interval estimates (intentional binding) are employed to implicitly measure the sense of agency.
This work describes a methodological framework that can be used to explicitly and implicitly characterize the sense of agency developed over the neural-machine interface (NMI) control of sensate virtual or robotic prosthetic hands. The formation of agency is fundamental in distinguishing the actions that we perform with our limbs as being our own. By striving to incorporate advanced upper-limb prostheses into these same perceptual mechanisms, we can begin to integrate an artificial limb more closely into the user's existing cognitive framework for limb control. This has important implications in promoting user acceptance, use, and effective control of advanced upper-limb prostheses. In this protocol, participants control a virtual prosthetic hand and receive kinesthetic sensory feedback through their preexisting NMIs. A series of virtual grasping tasks are performed and perturbations are systematically introduced to the kinesthetic feedback and virtual hand movements. Two separate measures of agency are employed: established psychophysical questionnaires (to capture the explicit experience of agency) and a time interval estimate task to capture the implicit sense of agency (intentional binding). Results of this protocol (questionnaire scores and time interval estimates) can be analyzed to quantify the extent of agency formation.
As robotic prostheses become increasingly advanced, the importance of relevant sensory feedback will continue to grow. Sensory feedback affects how humans perceive, interact with, and even integrate machines into their body schema. Recent NMI techniques can now provide prosthetic limb users with intuitive control and achieve sensations associated with touch1,2,3,4,5,6,7 and kinesthesia (movement sense)8,9 in missing limbs. When this sensory information is paired with the visual information provided by watching the artificial limb during operation, we have access to key elements that inform the distinction of self-versus-other. Leveraging this access may help bring prosthetic limb users a step closer to operating an artificial limb as a part of their body, rather than just a tool.
Body awareness and the sense of being embodied arises from the establishment of agency (the experience of authorship over a limb's actions) and ownership (the feeling that a limb is a part of the body)10,11. Ownership is primarily mediated through the integration of touch and visual information12. Agency emerges from the integration of intent, movement sensation (kinesthesia), visual information, and predictive cognitive models11. During the performance of a voluntary action, agency is formed when the sensory consequences of that action align with the performer's intent and predictions from the performer's internal models13. Agency is separate and distinct from ownership. The concept of limb ownership has been studied frequently in prosthesis literature14. A sense of limb ownership forms in NMI participants when touch feedback is spatially and temporally appropriate, as measured explicitly through questionnaires or implicitly through changes in residual limb temperature, or temporal order judgments15. However, fewer opportunities have existed to explore agency in the context of NMI16. Recent work with NMI participants has demonstrated that agency can be purposefully promoted and is separate from the experience of ownership8.
Agency is particularly significant in the operation of robotic prostheses as it is a cognitive link to the control of the artificial limb's physical actions through experiences of causality, the feeling of controlling the artificial limb or causing something to happen17. Robotic prostheses are advanced computerized machines that the user must cooperate with to effectively complete tasks. Some prosthetic limbs have incorporated autonomous functions, such as grip-slip detection and correction; yet these systems have seen limited adoption as functionality running outside the user's control can be viewed as frustrating if not appropriately implemented8,18. This presents a fundamental challenge that is echoed throughout applications of human cooperation with autonomous machines. That is, humans often trust their own actions over those resulting from a collaboration with computers or machines, and this trust directly influences an operator's likelihood to use the autonomous functions19,20. As humans, we innately trust ourselves and our bodies to perform the actions we intend; when this is achieved, we establish an intrinsic sense of agency. Interestingly, the formation of agency is impacted in cooperative human-computer actions. During human-human cooperative tasks, a shared sense of agency may be formed over movement21; yet, the literature suggests that shared agency is encumbered during human-computer cooperation22,23. These challenges are reflected in prosthetic upper-limb use, and the rejection rates of robotic devices remain high, with 23% - 39% of users discontinuing their use24. In fact, many prosthesis users still prefer body-powered systems25. These systems remove the computerized machine from the control loop and more intimately couple the user's body movement to the prosthesis movement via wire cables. This further reinforces the importance of cognitive integration in the use of advanced prosthetic devices. We suggest that NMI systems can provide a number of the necessary sensory and motor pieces to help move artificial limbs closer to establishing a cooperative sense of agency, and this will be instrumental in promoting the acceptance and the true integration of these computerized machines with their users.
Agency may be measured in a number of ways. The simplest measures use psychophysical questionnaires or scales that explicitly ask participants to whom or what they attribute an event17,26,27. This relies on an individual's existing perception of "self" by requiring participants to make inferential judgments of self-attribution (i.e., to explicitly judge whether "I" or another entity was responsible for an action or event). Implicit measures provide insight into the background cognitive processes that occur during motor action and sensory events. This view of agency attempts to measure that which is not explicitly perceived by an individual. Typically this is achieved by having participants characterize a perceived difference in self- and externally-generated actions, for example having participants report the duration of time they perceived to occur between a self- and externally-generated event17,28. During the performance of self-generated actions, agency implicitly manifests as a perceptual compression in time between actions and their sensory consequences, known as intentional binding28. When individuals report the time they perceived to occur between an action and its outcome, a shorter perceived duration of time corresponds to a more strongly formed sense of agency29,30. Interestingly, it has been demonstrated that explicit and implicit measures may not directly correlate as they are likely characterizing different perceptual mechanisms17 that together inform the sense of agency. As such, establishing a more comprehensive understanding of agency formation during prosthesis use will likely require experimental protocols employing both explicit and implicit measures.
This work describes a methodological framework that can be used to explicitly and implicitly characterize the sense of agency developed over the NMI control of sensate virtual or robotic prosthetic hands. Two techniques to measure agency during the performance of a sensorimotor object-grasping task are highlighted. Established psychophysical questionnaires are employed to capture the explicit experience of agency, while time interval estimates (intentional binding) are employed to implicitly measure the sense of agency.
The scope of this protocol is to evaluate the sense of agency in the context of an NMI that provides physiologically relevant active motor control and kinesthetic feedback. These techniques are generalizable to virtual or physical prosthetic NMI systems. There are minimal restrictions on the populations that may be recruited to perform this protocol. For instance, the mobility of the participant's upper limbs cannot be bilaterally affected (they must have one sound limb), and they must possess the cognitive ability to make time-based judgments and articulate experienced sensations.
This protocol has been previously approved and follows the guidelines of the Cleveland Clinic’s human research ethics committee.
1. Hardware and Software of the NMI
2. Experimental Setup
3. Experimental Conditions
4. Performance of the Experiment
The experimental protocol was performed with three amputee participants operating a sensate virtual prosthesis via their NMI8 (Figure 1). The setup used a participant-controllable virtual hand moving through preprogrammed kinematic profiles using the MuJoCo HAPTIX physics engine31. The virtual hand was displayed on a horizontal monitor in front of the participants at a location spatially congruent with ...
Here a methodological framework is presented to characterize the experience of agency formed while operating sensate prostheses via NMIs. In this context, agency is particularly relevant as it bridges physical action to the background cognitive processes that shape perception. Through a participant's prosthesis and NMI, we have direct access to a number of key elements that establish the sense of agency: intent, motor output, and movement sensation. Of importance to advanced prosthetic limb control, the tool...
The authors have nothing to disclose.
The authors would like to thank Madeline Newcomb for her contributions to the figure generation. This work was funded by the U.S. taxpayers through an NIH, Office of the Director, Common Fund, Transformative R01 Research Award (grant #1R01NS081710-01) and the Defense Advanced Research Projects Agency (contract number N66001-15-C-4015 under the auspices of Biology Technology Office program manager D. Weber).
Name | Company | Catalog Number | Comments |
LabVIEW 2015, Service Pack 1, Version 15.0.1f2 64-bit | National Instruments, Austin, TX, USA | Full or Pro Version | We wrote custom software in LabVIEW to coordinate virtual prosthesis control with kinesthetic feedback as well as to present experimental conditions and record data. |
8-Slot, USB CompactDAQ Chassis | National Instruments, Austin, TX, USA | cDAQ-9178 | |
±60 V, 800 kS/s, 12-Bit, 8-Channel C Series Voltage Input Module | National Instruments, Austin, TX, USA | NI-9221 | |
100 kS/s/ch Simultaneous, ±10 V, 4-Channel C Series Voltage Output Module | National Instruments, Austin, TX, USA | NI-9263 | |
Custom Wearable Kinesthetic Tactor | HDT Global, Solon, OH, USA | N/A | This item was custom made. Other methods of delivering kinesthetic feedback are acceptable as long as the participant feels the sensation of the hand moving in real-time with the movements of the virtual hand. |
MuJoCo Physics Engine, HAPTIX Version | Roboti LLC, Redmond, WA, USA | mjhaptix150 | Newer versions of MuJoCo should be acceptable as well. We used the MPL Gripper Model. |
Myobock Electrodes, powered by Otto Bock EnergyPack in MyoBoy Battery Receptacle | Ottobock, Duderstadt, Germany | electrodes: 13E200=60 battery: 757B21 battery receptacle: 757Z191=2 | Any setup that provides an amplified, filtered, and rectified EMG or neural control signal could be used. |
Request permission to reuse the text or figures of this JoVE article
Request PermissionExplore More Articles
This article has been published
Video Coming Soon
Copyright © 2025 MyJoVE Corporation. All rights reserved