Virtual Hand with Ambiguous Movement between the Self and Other Origin: Sense of Ownership and 'Other-Produced' Agency

Summary

While previous research on bodily self-consciousness assumed that self- and other-origin movements were perceptually distinguishable, this protocol allows them to be ambiguous on a virtual hand with unintentional slight movements. This enables us to observe one’s experience formed by SoO and other-produced SoA, rather than the absence of SoA.

Abstract

The feeling that a body part is one’s own body (sense of ownership; SoO) and the feeling based on the causal relationship between one’s will and action (sense of agency; SoA) have been recognized as the basis of our bodily self-consciousness. Previously, the illusory SoO over a fake body part (e.g., rubber hand) was introduced as the rubber hand illusion (RHI). Furthermore, it was determined that one could also evoke a SoA over an object with movements linked to the one’s prior intention. On the other hand, the postdictivity of our spontaneity implies that it is essentially inseparable whether actions originate from self or others. In other words, our SoA or daily experiences are obtained in such as inseparable scenario. Previous research, however, has maintained the premise that self- and other-origin movements are perceptually distinguishable. Here, we implement a protocol to make these aspects ambiguous for the participants and to estimate whether they can feel SoO and/or SoA and how. To this end, we employ an experiment using virtual reality, under which participants observe virtual fingers moving very slowly (or quickly or not moving) while their own fingers do not move. For evaluation of the illusory SoO, measurements of skin conductance responses against a knife threat are adopted. Additionally, we introduce face-to-face interviews to determine whether the feelings regarding the slow movement match the conventional SoA definition. Our representative results suggest that the SoO is evoked over the hand, and various attitudes to accept its movement as the participant’s own with awareness that they did not originate it are reported by the majority. As the results show, the novelty of this protocol is discovering that in such a situation, the SoO cooperates with an externally produced SoA to establish one’s own bodily experience rather than the independence of the SoO and SoA.

Introduction

How does one perceive one’s body or the actions one has performed as one’s own? These sensations constitute our bodily self-consciousness and our own experience. Such an issue, particularly in terms of “minimal self”, has been considered to comprise a sense of ownership (SoO), the sense to feel the body as one’s own, and a sense of agency (SoA), the sense of authorship over bodily movement¹. Many studies on SoO/SoA have been based on the comparator model², in which one’s actions are subjected to internal comparison processes^3,4,5,6. The comparator model describes a feed-forward loop linking intention with a prediction of one’s motion and a feedback loop relating sensory feedback to the prediction; these loops have been recognized as the SoA and SoO, respectively^1,6.

As a means of experimental investigation, body transfer illusions have been employed. A typical approach is the rubber hand illusion (RHI), in which synchronous stroking of participants’ hidden hands and fake hands located in front of them evokes illusory ownership of the fake hand⁷. While the original RHI refers only to the SoO, recent implementations of the RHI equipped with a movable hand have allowed assessment of the mechanism of SoA as giving experiences of feed-forward loop matching to the participants. Although the SoO and SoA coincide in normal behavior¹, they can be independent of each other, and both can be evoked separately. For example, if participants are shown an avatar, they feel only a weak SoO but not an SoA, or if participants see their hand hiding their thumb from a third person’s perspective, they feel only an SoA but not an SoO⁸. An SoO elicited by an SoA has also been reported⁹, and the relationship between the two and how both together produce bodily consciousness are still open questions.

Although SoA is a broad term, it is commonly regarded as a sensation of a generated action while self-detached from the external world. In this sense, in order to evaluate the SoA experimentally, it is assumed that the participants themselves manipulate the real body or the fake body, for example, using a hand image projected on a screen¹⁰, a fake hand located on a wooden pole¹¹, or a hand located on a cover above the participant’s hand^12,13. In these studies, when the visible fake hand was synchronized with participant-generated movement, the emergence of the SoA was confirmed. Additionally, when the movements were externally generated not by a participant but by an experimenter using a physical connection to the hand, it was confirmed that the absence of the SoA could occur, while the SoO was retained. In those studies, a pair of absence and presence of the agency was implemented by a pair of other- and self-origin movements for the participant.

The question arises: does the absence of the SoA imply that body movement is made by others except for the one? Is there a clear separation between me and others? However, myself could involve unconsciousness which could be others in myself. Concerning unconsciousness, one cannot determine whether a movement originates from the one’s self or others. Although the sensation of autonomous convulsion is destined to be relevant for unconsciousness, the previous experimental setups have not paid attention to the ambiguity of the self and others. To involve unconscious movement in an experiment, one has to implement a virtual move that cannot be readily determined as originating from the participant or others. Such ambiguity of the participant and others is also observed in the discussion of “readiness potential”¹⁴. The readiness potential preceding free will and intentional consciousness can be compared to others and me in a brain, respectively¹⁵. In other words, the self and others in one’s self are inseparable from each other. In fact, behaviorally, it has been pointed out that SoA could be acquired in a postdictive manner^16,17. Considering that our SoA or bodily self-consciousness could be acquired in such a self/other ambiguity, while previous research on SoO/SoA has been carried out after making them perceptually distinguishable, it is necessary to investigate what kind of subjective experience occurs when both parties are ambiguous.

Here, we introduce a protocol that enables us to evaluate SoO/SoA effects on a virtual hand with slight movements such that it is unclear whether the movement is self- or other-originated. Specifically, using virtual reality (VR), our participants were made to see a virtual hand from the first person’s perspective through a head-mounted display (HMD) without telling who the hand belongs to. Their hand was given a tactile stimulus (stroking with a paintbrush) in sync with the brush movement on the hand shown in HMD, and then, after one minute, the fingers in the video suddenly start moving slowly. For comparison, we conducted six trials for each participant, following three conditions (in duplicate): no hand movement, slow movement (ambiguity between self- and other-origin), and fast movement (clearly not self- but other-origin). To evaluate the SoO, skin conductance response (SCR) measurements were adopted. The change in SCR during a presentation of a threat was used as an indicator of bodily self^18,19,20, as it provides an online and objective estimation. In this study, we defined SCR arising from a knife threat following hand movement as the intensity of the SoO^18,19. Moreover, for evaluating the SoA, subjective reports were collected using face-to-face interviews after all RHI procedures. Here, it could be supposed that such an ambiguous movement between self- and other-origin would cause the bodily movement originating from another to be perceived as the participant’s own. Such a rewriting of the causal relationship in SoA cannot be captured by the pre-established method to simply evaluate the consistency between one's intention and one’s action. Therefore, while in conventional RHI studies, the participants were required to score given statements describing the sensation¹³, this protocol allows participants to describe their sensations in their own words. After the subjective reports are collected in such a manner, the steps for classifying and evaluating them are prepared. These series enable us not only to determine whether SoA was present or absent but also to observe how their sensations, which were accepted as their own experience, differed from the stereotypical SoA.

Protocol

All methods described here were approved to follow the Code of Ethics of the World Medical Association (Declaration of Helsinki) and approved by the Independent Ethics Committee of Nagaoka University of Technology. No identifying information was collected from the undergraduates. Participants were given detailed information about the experiments before participating and were required to sign a written acceptance statement. Written informed consent was obtained from all subjects and was stored at Nagaoka University of Technology.

1. Preparing seven kinds of videos

NOTE: In total, create seven videos for a pre-experimental session (one video) and for experimental sessions in the “stable”, “slow” and “fast” conditions (Figure 1) for each gender (three times two videos).

1. In an experiment room, set a table and two chairs, a hand rest (approximately 35 x 60 cm white cushion tilted approximately 25 degrees) and a display on the table (Figure 2). Mark these positions to be able to reproduce accurately in the experimental session.
2. Place a 360° camera at eye level when the participants are sitting in the chair (Figure 2).
3. Start to record the camera with nobody in the video for approximately 20–30 s for a pre-experimental session to ensure that there is reasonable visibility for each participant.
4. For the video to show male participants, let a male experimenter (Experimenter A) wear a lab coat, sit in the chair and put on his left hand on the hand rest with the palm up, taking care that the fingers do not touch each other nor touch the hand rest.
5. Play a 60 bpm 4-beat reference tone using a metronome application and a smartphone.
6. Let another experimenter (Experimenter B) sit facing Experimenter A, with a paint brush in Experimenter B’s hand (Figure 3).
7. Start to record the camera for the video to use the “stable” condition.
8. Using the reference tone, let Experimenter B stroke all of Experimenter A’s fingers for 80 s. At the same time, let Experimenter B memorize the trajectory to reproduce it later.
9. By the third experimenter, show an action that an approximately 30 cm kitchen knife (20 cm blade) appears, slashes Experimenter A’s left wrist and disappears, taking 5 s (Figure 4).
10. After 5 s of margin duration, stop the camera.
11. Start to record the camera for the video to use the “slow” condition.
12. Repeat step 1.8 while changing the brush trajectory to avoid monotony.
13. 60 s after the start, let Experimenter A close and open the five fingers at a consistent slow speed for 20 s (20 s/close and open cycle, approximately 1.3 mm/s; Figure 5(A)).
14. Repeat steps 1.9 and 1.10.
15. Start to record the camera for the video to use the “fast” condition.
16. Repeat steps 1.12 to 1.14 while changing the finger speed from slow to fast (2 s/open and close cycle, approximately 35 mm/s; Figure 5(B)).
17. Repeat steps 1.4 to 1.16 while changing from a male Experimenter A to a female Experimenter A to produce another three videos for the female participants.
18. Remove the camera. Check that the videos do not diverge from the real world, using the HMD and sitting the chair and putting left hand on the hand rest.
19. If necessary, practice reproducing the trajectory of the brush accurately.

2. Experimental procedure

1. Use more than 18 participants who are healthy adults (approximate ages of 18–25 and a gender ratio of 50% are better) without history of major illness and naïve to the experiment and speaking the same native language to collect subjective reports. Experiment with each participant separately.
2. RHI with/without finger movement sessions and SCR measurements
   1. Conduct the following steps that include the role of interacting with the participant by the same experimenter with the same looks as Experimenter B in the videos to use. Use assistance if necessary.
   2. Ask a participant to wear a lab coat and sit the chair. Attach Ag–AgCl electrodes connected with an SCR data acquisition device to the index and ring fingers of the participant’s right hand after polishing those fingers. Keep the hand down while not touching anything. Set the sampling rate at 500 samples/s, and start to record the SCR data.
   3. Give to the participant the following three instructions:
      1. “Your task is put your left hand on the hand rest with the palm facing up and keeping the hand stable”
      2. “Please try to relax.”
      3. “When wearing the HMD, please look at your left hand”.
   4. Have the participant wear an HMD with a black screen. Mirror the screen on the HMD to the display on the table until the end of the experiment.
   5. Show the video for the pre-experimental session on the HMD after announcing that. Ask the participant to look around and ensure that the view is natural. If something is wrong, adjust the HMD or the seat height until the participant is comfortable.
   6. Show a black screen on the HMD, and ask the participant to put his/her left hand on the hand rest with the palm up. Help the participant, and take care that the fingers are not touching each other or the hand rest.
   7. Sit facing the participant with headphones and a paintbrush. Prepare for one video for one of the three conditions with the left hand of the same gender as the participant.
   8. Begin the first trial with one condition randomly selected from the three. After announcing that the session is beginning, start the video. Brush the participant’s left hand with the same timing and position as s/he is watching, checking the mirroring display on the desk and the reference tone, until the knife cuts in (Figure 3(B)).
   9. Show a black screen on the HMD 2 minutes to allow SCR to return to normal.
   10. Repeat steps 2.2.8 to 2.2.10 for the remaining five times (a total of six times; two trials of each of the three conditions) in random order. Ask the participant after every two trials whether anything is wrong with the physical condition.
   11. Stop recording the SCR data. Remove the HMD and SCR data acquisition device.
3. Face-to-face interviews: Pose the following questions to the participant and collect the answers orally while recording them.
   1. Ask “Did you see that the hand was moved?” [Q1: For confirmation]
      If the answer is not “Yes”, go to step 2.3.4.
   2. Ask “Did you see the movement of the two speeds? [Q2: For confirmation]
      If the answer is not “Yes”, go to step 2.3.4.
   3. Ask “What did you think about the slower hand movement?” [Q3: For SoA evaluation]
   4. Ask “Could you tell me if you felt anything during the experiment.” [Q4: Free report]

3. Data analysis

1. SCR data
   1. Identify the SCRs for each trial in the following manner^19,21,22: the difference in amplitude between the maximal and minimal value that occurred during all of 2–10 s in the last 5 s (knife threat; for evaluation of SoO) and immediately after 5 s.
   2. Calculated the magnitude of amplitudes according to a pre-established method²³ as:
      
   3. Conduct the Shapiro–Wilk test on all data. If it is confirmed that some data were not normally distributed, use nonparametric statistical tests to process the data; otherwise, use parametric statistical tests. In all statistical tests, set alpha to 5%.
   4. Evaluate the difference among the three conditions with a multiple comparison.
2. Face-to-face interviews
   1. Divide the data of those who answered “No” to Q1 and/or 2.
   2. Compare the answers to Q3 with the following three statements that have been used to measure participants’ SoA in the RHI study¹³.
      Statement 1: The hand moved just like I wanted it to, as if it was obeying my will.
      Statement 2: I felt as if I could control the movements of the hand.
      Statement 3: I felt as if I could cause the movement I saw.
   3. Classify the answers to Q3 from a summary of each, and analyze the overall trend.

Discussion

The purpose of this protocol is to highlight the inherent inseparability of whether the origin of action is the participant’s self or another. It could be supposed that such inseparability causes the participants to accept the movement of alien origin as their own experiences. Here, we proposed a method to collect the sensations in such a situation, referring to the conventional discussion of the SoO/SoA. As a means of achieving this, step 1.13 (preparing the video) and step 2.2.8 (showing it under plausible circumstances) are achieved by appropriately shifting the behavior of the observed body part from that of the actual. Some other steps support this aim from the following two points. The first point is to make the virtual image closer to reality. Thus, the world in the video is visually aligned to the real world regarding the hand features (steps 1.4, 1.17 and 2.2.3), laboratory landscapes (step 1.1), and the facing experimenter (step 2.2.1). In addition, establishing cross-modal interaction would also be effective, as it is considered to be essential, especially in the full body-transfer illusion^24,25. Therefore, it would be effective to use a 360-degree image (step 1.2; matching the visual information and head movement control) and to accurately trace the trajectories (step 2.2.8; matching of visual and tactile sensation). The second point is to broaden the interpretation of (virtual) visual information. This point was also emphasized in another study that made participants uncertain of whether their view in HMD was virtual or real²⁶. While the authors achieved this goal by reducing the virtual and real visual resolution equally, in this protocol, we applied the following two methods in step 2.2.3, to give no information about what would be shown and to make themselves relax. These cares seemed to work considering the reaction to a question “Guess what hand you were looking at?” that we asked to all participants after Q3 to confirm the validity of this protocol (this is an extra question that does not need to appear in the protocol). Then, rather than having no one correctly answer that it was a prerecorded video, it was found that the majority of them were experimenting with uncertainties and allowing various possibilities.

Additionally, step 2.3 is considered to be important in evaluating such feelings that the bodily movement originating from others is perceived as one’s own and that may also contain contradictions when viewed objectively. While in the typical method using a questionnaire about SoO/SoA, participants are presented the preprepared expressions about the feelings first, in this protocol, they are requested to describe their feelings in words of their own. This way enables us to avoid reinterpretation of their experience by the given expressions by the experimenter and to collect contradictory answers. Actually, as shown above, the majority of our participants reported that the observed move was compatible with what was relevant to them. Moreover, we could collect the answers “it was like mine and not mine” from three participants in step 2.3.4. In addition, it is also notable that illusory sensory feedback was reported in Type 3, although the number was small. Those points that had not been observed in previous RHI studies might offer us new insight.

Regarding possibilities to improve and troubleshoot this method, a common problem is that accurately tracing the paintbrush trajectory is important but difficult. Indeed, the illusion disappears as soon as the movement of the brush is asynchronous^7,27. Our subjects also reported that the sense of ownership suddenly disappeared at the moment when the experimenter failed to trace by accident. While it has been reported that the illusion intensity increases with increasing complexity and randomness of the brush movement, we recommend taking a simple trajectory that is not easily mistaken rather than taking the risk. In addition, regarding SCR measurement, one could be concerned that the participants may get used to the repeated knife threat. In some previous studies^19,22, however, the experiment was conducted many more times (for example, thirty times¹⁹). This protocol proposes two trials for each condition, for a total of six times per participant to reduce their burden, as the minimum number of times that would reveal the difference between the three conditions. Furthermore, it is possible to consider increasing the number of trials for more statistically reliable data collection and to study the effect of iteration of the trial. Another point, our materials table shows the set of equipment that we used to obtain our representative results. However, especially since the head-mounted display manufacturers are frequently updated to new products, the catalog number (Oculus DK2) in the table is now discontinued. Currently, the same experiment can be performed by using the latest model (Oculus Rift S or Quest 2; in this case the corresponding display is a PC with Windows OS instead of Mac OS), and we recommend carrying out with the product available at that time.

A considerable limitation of this protocol is that the estimation of SoA depends on the retrospective method. The reason that the face-to-face interview is set as the last step of the experimental procedure part and not just after the target trials (the trail in the “slow” condition), such as general methods¹³, is to prevent the participants from focusing their attention on the question in the subsequent trials. Considering that this risk had a greater impact on the entire experiment, we set the current order. On the other hand, this case in which other trials are inserted between the target trial and the interview arises the risks that the acquired feeling would be mixed or forgotten. Here, that issue was mitigated by using questions 1 and 2. Furthermore, the use of general questionnaires is still retrospective. Therefore, developing a measurement method that can be performed in parallel with the experiment using physiological indexes such as SCR could be said as a common issue. Moreover, it may be possible to observe some differences when comparing the trends of the subjective reports against slow and fast conditions. While repeating questions has a risk of affecting participants’ retrospective subjective reports, a possible approach is a comparison between groups, to ask about only one condition to each.

In summary, this protocol has implemented a situation in which the SoO coexists with the SoA that is inconsistent on one body part, creating a good balance of difference and consistency between the shown state and the actual state. This time, for the sake of simplification, the participants’ real hand is made to be stable and the shown hand is made to move slightly, but the opposite case is also applicable. In that case, it is important for each participant to move their fingers at a uniform speed and distance; thus, it is necessary to confirm and practice this process well in advance instructions. Moreover, a wide range of applications can be imagined, such as the case with a change in the shape and/or the number of shown fingers or the case with intervention and/or interaction with the obvious hands of others.

Materials

Name	Company	Catalog Number	Comments
display	Apple Inc.	MacBookAir	for mirroring the screen which is shown to the participants in the HMD
head mounted display	Oculus VR Inc.	Oculus DK2	for dispaying the 360° videos
metronome apptication	ihara-product	Smart Metronome	for generate reference tones
SCR data acquisition device	Biopac Systems inc.	Biopac System MP150	for SCR data mesurement
SCR data processing software	Biopac Systems inc.	AcqKnowledge 4.1	for SCR data processing
smart phone	Apple Inc.	iPhone SE	for playing reference tones
statistical analyzing software	The R Foundation	R 3.4.2.	for statistical analyzing
360° camera	RICOH Company Ltd.	RICOH THETA S	for preparing the videos