Name: applying incongruent visual-tactile stimuli during object transfer with vibro-tactile feedback
Uploaded: 2019-05-23T14:30:00
Description: Watch this Scientific Journal Video about Applying Incongruent Visual-Tactile Stimuli during Object Transfer with Vibro-Tactile Feedback at JoVE.com

This study was not funded.

The following protocol follows the guidelines of human research ethics committee of the university. See Table of Materials for reference to the commercial products.
NOTE: After receiving approval of the university ethics committee, 20 healthy individuals (7 males and 13 females, mean and standard deviation [SD] of age 30.2 &#177; 16.3 years) were recruited. Each subject read and signed an informed consent form pretrial. Inclusion criteria were right-handed individuals aged 18 ...

<ol>
	<li>Aggelopoulos, N. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Perceptual+inference.">Perceptual inference.</a> Neuroscience and Biobehavioral Reviews. 55, 375-392 (2015).</li><li>van der Groen, O., van der Burg, E., Lunghi, C., Alais, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Touch+influences+visual+perception+with+a+tight+orientation-tuning.">Touch influences visual perception with a tight orientation-tuning.</a> PloS One. 8 (11), e79558 (2013).</li><li>Pei, Y. C., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cross-modal+sensory+integration+of+visual-tactile+motion+information:+instrument+design+and+human+psychophysics.">Cross-modal sensory integration of visual-tactile motion information: instrument design and human psychophysics.</a> Sensors. 13 (6), 7212-7223 (2013).</li><li>Kording, K. P., Wolpert, D. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Bayesian+integration+in+sensorimotor+learning.">Bayesian integration in sensorimotor learning.</a> Nature. 427 (6971), 244-247 (2004).</li><li>Tajadura-Jimenez, A., Banakou, D., Bianchi-Berthouze, N., Slater, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Embodiment+in+a+Child-Like+Talking+Virtual+Body+Influences+Object+Size+Perception,+Self-Identification,+and+Subsequent+Real+Speaking.">Embodiment in a Child-Like Talking Virtual Body Influences Object Size Perception, Self-Identification, and Subsequent Real Speaking.</a> Scientific Reports. 7 (1), (2017).</li><li>Campos, J. L., Butler, J. S., Bulthoff, H. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+integration+in+the+estimation+of+walked+distances.">Multisensory integration in the estimation of walked distances.</a> Experimental Brain Research. 218 (4), 551-565 (2012).</li><li>Sun, H. J., Campos, J. L., Chan, G. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+integration+in+the+estimation+of+relative+path+length.">Multisensory integration in the estimation of relative path length.</a> Experimental Brain Research. 154 (2), 246-254 (2004).</li><li>Parmentier, F. B., Ljungberg, J. K., Elsley, J. V., Lindkvist, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+behavioral+study+of+distraction+by+vibrotactile+novelty.">A behavioral study of distraction by vibrotactile novelty.</a> Journal of Experimental Psychology, Human Perception, and Performance. 37 (4), 1134-1139 (2011).</li><li>Sigrist, R., Rauter, G., Riener, R., Wolf, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Augmented+visual,+auditory,+haptic,+and+multimodal+feedback+in+motor+learning:+a+review.">Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review.</a> Psychonomic Bulletin &amp; Review. 20 (1), 21-53 (2013).</li><li>Hebert, J. S., Lewicke, J., Williams, T. R., Vette, A. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Normative+data+for+modified+Box+and+Blocks+test+measuring+upper-limb+function+via+motion+capture.">Normative data for modified Box and Blocks test measuring upper-limb function via motion capture.</a> Journal of Rehabilitation Research and Development. 51 (6), 918-932 (2014).</li><li>Raveh, E., Portnoy, S., Friedman, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Adding+vibrotactile+feedback+to+a+myoelectric-controlled+hand+improves+performance+when+online+visual+feedback+is+disturbed.">Adding vibrotactile feedback to a myoelectric-controlled hand improves performance when online visual feedback is disturbed.</a> Human Movement Science. 58, 32-40 (2018).</li><li>Raveh, E., Friedman, J., Portnoy, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Evaluation+of+the+effects+of+adding+vibrotactile+feedback+to+myoelectric+prosthesis+users+on+performance+and+visual+attention+in+a+dual-task+paradigm.">Evaluation of the effects of adding vibrotactile feedback to myoelectric prosthesis users on performance and visual attention in a dual-task paradigm.</a> Clinical Rehabilitation. 32 (10), 1308-1316 (2018).</li><li>Raveh, E., Portnoy, S., Friedman, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Myoelectric+Prosthesis+Users+Improve+Performance+Time+and+Accuracy+Using+Vibrotactile+Feedback+When+Visual+Feedback+Is+Disturbed.">Myoelectric Prosthesis Users Improve Performance Time and Accuracy Using Vibrotactile Feedback When Visual Feedback Is Disturbed.</a> Archives of Physical Medicine and Rehabilitation. , (2018).</li><li>Raveh, E., Friedman, J., Portnoy, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visuomotor+behaviors+and+performance+in+a+dual-task+paradigm+with+and+without+vibrotactile+feedback+when+using+a+myoelectric+controlled+hand.">Visuomotor behaviors and performance in a dual-task paradigm with and without vibrotactile feedback when using a myoelectric controlled hand.</a> Assistive Technology: The Official Journal of RESNA. , 1-7 (2017).</li><li>Dienes, Z. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Using+Bayes+to+get+the+most+out+of+non-significant+results.">Using Bayes to get the most out of non-significant results.</a> Frontiers in Psychology. 5, 781 (2014).</li><li>Shams, L., Murray, M. M., Wallace, M. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Early+Integration+and+Bayesian+Causal+Inference+in+Multisensory+Perception.">Early Integration and Bayesian Causal Inference in Multisensory Perception.</a> The Neural Bases of Multisensory Processes. , (2012).</li><li>D'Amour, S., Pritchett, L. M., Harris, L. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Bodily+illusions+disrupt+tactile+sensations.">Bodily illusions disrupt tactile sensations.</a> Journal of Experimental Psychology, Human Perception, and Performance. 41 (1), 42-49 (2015).</li><li>Tidoni, E., Fusco, G., Leonardis, D., Frisoli, A., Bergamasco, M., Aglioti, S. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Illusory+movements+induced+by+tendon+vibration+in+right-+and+left-handed+people.">Illusory movements induced by tendon vibration in right- and left-handed people.</a> Experimental Brain Research. 233 (2), 375-383 (2015).</li><li>Fuentes, C. T., Gomi, H., Haggard, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Temporal+features+of+human+tendon+vibration+illusions.">Temporal features of human tendon vibration illusions.</a> The European Journal of Neuroscience. 36 (12), 3709-3717 (2012).</li><li>de Vignemont, F., Ehrsson, H. H., Haggard, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Bodily+illusions+modulate+tactile+perception.">Bodily illusions modulate tactile perception.</a> Current Biology. 15 (14), 1286-1290 (2005).</li><li>Marotta, A., Tinazzi, M., Cavedini, C., Zampini, M., Fiorio, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Individual+Differences+in+the+Rubber+Hand+Illusion+Are+Related+to+Sensory+Suggestibility.">Individual Differences in the Rubber Hand Illusion Are Related to Sensory Suggestibility.</a> PloS One. 11 (12), e0168489 (2016).</li><li>Stevenson, R. A., Zemtsov, R. K., Wallace, M. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Individual+differences+in+the+multisensory+temporal+binding+window+predict+susceptibility+to+audiovisual+illusions.">Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions.</a> Journal of Experimental Psychology, Human Perception, and Performance. 38 (6), 1517-1529 (2012).</li><li>Maravita, A., Spence, C., Driver, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+integration+and+the+body+schema:+close+to+hand+and+within+reach.">Multisensory integration and the body schema: close to hand and within reach.</a> Current Biology. 13 (13), R531-R539 (2003).</li><li>Carey, D. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+integration:+attending+to+seen+and+felt+hands.">Multisensory integration: attending to seen and felt hands.</a> Current Biology. 10 (23), R863-R865 (2000).</li><li>Tsakiris, M., Haggard, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+rubber+hand+illusion+revisited:+visuotactile+integration+and+self-attribution.">The rubber hand illusion revisited: visuotactile integration and self-attribution.</a> Journal of Experimental Psychology, Human Perception, and Performance. 31 (1), 80-91 (2005).</li></ol>

In this study, a protocol that quantifies the effect of adding VTF on the object transfer kinematics in the presence of incongruent visual-tactile stimuli was presented. To the best of our knowledge, this is the only protocol available to test the effect of VTF on the response to incongruent visual-tactile stimuli. The several critical steps involved in the application of incongruent visual-tactile stimuli during object transfer with VTF include the following: attaching the VTF system to the subject, activating the VTF, ...

Illusions are exploitations of the limitations of our senses, as we mistakenly perceive information that deviates from objective reality. Our perceptual inference is based on our experience in interpreting sensory data and on the calculation of our brain of the most reliable estimate of reality in the presence of ambiguous sensory input1.
A sub-category in the research of illusions is one that combines incongruent sensory signals. The illusion that results from incongruent sensory signals originates from the constant multisensory integration performed by our brain. While there are numerous studies concerning incongruence in visual-auditory signals, incongruence in other sensory pairs is less reported. This difference in the number of reports might be attributed to the higher simplicity in designing a setup that incorporates visual-auditory incongruence. However, studies that report results relating to other sensory pairs modalities, are interesting. For example, the effect of incongruent visual-haptic signals on visual sensitivity2 was studied using a system where the visual and haptic stimuli were matched in spatial frequency; however, the haptic and visual orientation was identical (congruent) or orthogonal (incongruent). In another study, the effect of incongruent visual-tactile motion stimuli on the perceived visual direction of motion was investigated using a visual-tactile cross-modal integration stimulator with a lighted panel that presents visual stimuli and a tactile stimulator that presents tactile motion stimuli with arbitrary motion direction, speed, and indentation depth in the skin3. It was suggested that we internally represent both the statistical distribution of the task and our sensory uncertainty, combining them in a manner consistent with a performance-optimizing Bayesian process4.
Virtual reality has made the ability to deceive the visual feedback to the subject an easy task. Several studies used multisensory virtual reality to misalign visual and somatosensory information. For example, virtual reality was recently used to induce embodiment in a child&#8217;s body, with or without activation of a child-like voice distortion5. In another example, the visual presentation of the walking distance during self-motion was extended and was therefore incongruent with the travel distance felt by body-based cues6. A similar virtual reality setup was designed for a cycling activity7. All of the aforementioned literature, however, did not combine an interference to one of the senses, in addition to the incongruent signal. We chose the tactile sense to receive such a disturbance.
Our tactile sensory system provides direct evidence as to whether an object is being grasped. We therefore expect that when direct visual feedback is distorted or unavailable, the role of the tactile sensory system in object manipulation tasks will be prominent. However, what would happen if the tactile sensory channel was also disturbed? This is a possible outcome of using vibrotactile feedback (VTF) for sensory augmentation, as it captures the attention of the individual8. Today, augmented feedback of different modalities is used as an external tool, meant to enhance our internal sensory feedback and improve performance during motor learning, in sport and in rehabilitation settings9.
The study of incongruent visual-tactile stimuli may enhance our understanding regarding perception of the sensory input. Particularly, quantification of the additive or reductive value of VTF during motor function that involves incongruent visual-tactile stimuli, can assist in future prosthetics design, smart sport-wear, or any other garments that incorporate VTF. Since amputees are deprived of tactile stimuli at the distal aspect of their residuum, their daily usage of the VTF, embedded in the prosthetic to convey knowledge of grasping, for example, might influence how they perceive visual feedback. Understanding of the mechanism of perception under these conditions, will allow engineers to perfect VTF modalities to reduce the negative effect on VTF users.
We aimed to test the effect of VTF on the response to incongruent visual-tactile stimuli. In the presented setup, the tactile feedback is acquired by grasping a block and moving it across a partition; the visual feedback is a real-time virtual presentation of the moving block and the partition (acquired using a motion capture system). Since the subject is prevented from seeing the actual hand movement, the only visual feedback is the virtual one. The congruent feedback is the reliable presentation of the movement of the block, so that the subject feels that the block is grasped and sees it move along with the path of the hand. The incongruent feedback appears as the movement of the block diverts from the actual movement path, so that it seems to drop from the hand when it is actually still held by the subject, thereby contradicting the tactile feedback. Three hypotheses were tested: when moving an object from one place to another using virtual visual feedback, (i) the path and duration of the object&#8217;s transfer motion will increase when incongruent visual-tactile stimuli is presented, (ii) this change will increase when incongruent visual-tactile stimuli is presented and VTF is activated on the moving arm, and (iii) a positive correlation will be found between the perceived difficulty level of performing the task with the VTF activated and the path and duration of the object&#8217;s transfer motion. The first hypothesis originates from aforementioned literature that report that various modalities of incongruent feedback affect our responses. The second hypothesis relates to the previous findings that VTF captures the attention of the individual. For the third hypothesis, we assumed that subjects who were more disturbed by the VTF, will trust the virtual visual feedback more than their tactile sense.

<table>
	<tbody>
		<tr>
			<td>3D printer</td>
			<td>Makerbot</td>
			<td></td>
			<td>https://www.makerbot.com/</td>
		</tr>
		<tr>
			<td>Box and Blocks test</td>
			<td>Sammons Preston</td>
			<td></td>
			<td>https://www.performancehealth.com/box-and-blocks-test</td>
		</tr>
		<tr>
			<td>Flexiforce sensors (1lb)</td>
			<td>Tekscan Inc.</td>
			<td></td>
			<td>https://www.tekscan.com/force-sensors</td>
		</tr>
		<tr>
			<td>JASP</td>
			<td>JASP Team</td>
			<td></td>
			<td>https://jasp-stats.org/</td>
		</tr>
		<tr>
			<td>Labview</td>
			<td>National Instruments</td>
			<td></td>
			<td>http://www.ni.com/en-us/shop/labview/labview-details.html</td>
		</tr>
		<tr>
			<td>Micro Arduino</td>
			<td>Arduino LLC</td>
			<td></td>
			<td>https://store.arduino.cc/arduino-micro</td>
		</tr>
		<tr>
			<td>Motion capture system</td>
			<td>Qualisys</td>
			<td></td>
			<td>https://www.qualisys.com</td>
		</tr>
		<tr>
			<td>Shaftless vibration motor</td>
			<td>Pololu</td>
			<td></td>
			<td>https://www.pololu.com/product/1638</td>
		</tr>
		<tr>
			<td>SPSS</td>
			<td>IBM</td>
			<td></td>
			<td>https://www.ibm.com/analytics/spss-statistics-software</td>
		</tr>
	</tbody>
</table>

applying incongruent visual-tactile stimuli during object transfer with vibro-tactile feedback

The application of incongruent sensory signals that involves disrupted tactile feedback is rarely explored, specifically with the presence of vibrotactile feedback (VTF). This protocol aims to test the effect of VTF on the response to incongruent visual-tactile stimuli. The tactile feedback is acquired by grasping a block and moving it across a partition. The visual feedback is a real-time virtual presentation of the moving block, acquired using a motion capture system. The congruent feedback is the reliable presentation of the movement of the block, so that the subject feels that the block is grasped and see it move along with the path of the hand. The incongruent feedback appears as the movement of the block diverts from the actual movement path, so that it seems to drop from the hand when it is actually still held by the subject, thereby contradicting the tactile feedback. Twenty subjects (age 30.2 &#177; 16.3) repeated 16 block transfers, while their hand was hidden. These were repeated with VTF and without VTF (total of 32 block transfers). Incongruent stimuli were presented randomly twice within the 16 repetitions in each condition (with and without VTF). Each subject was asked to rate the difficulty level of performing the task with and without the VTF. There were no statistically significant differences in the length of the hand paths and durations between transfers recorded with congruent and incongruent visual-tactile signals &#8211; with and without the VTF. The perceived difficulty level of performing the task with the VTF significantly correlated with the normalized path length of the block with VTF (r = 0.675, p = 0.002). This setup is used to quantify the additive or reductive value of VTF during motor function that involves incongruent visual-tactile stimuli. Possible applications are prosthetics design, smart sport-wear, or any other garments that incorporate VTF.

We used the described technique to test the three hypotheses that when moving an object from one place to another using virtual visual feedback: (i) the path and duration of the object&#8217;s transfer motion will increase when incongruent visual-tactile stimuli is presented; (ii) this change will increase when incongruent visual-tactile stimuli is presented and VTF is activated on the moving arm; and (iii) a positive correlation will be found between the perceived difficulty level of performing the task with the VTF act...

Watch this Scientific Journal Video about Applying Incongruent Visual-Tactile Stimuli during Object Transfer with Vibro-Tactile Feedback at JoVE.com

Applying Incongruent Visual-Tactile Stimuli during Object Transfer with Vibro-Tactile Feedback

We present a protocol to apply incongruent visual-tactile stimuli during an object transfer task. Specifically, during block transfers, performed while the hand is hidden, a virtual presentation of the block shows random occurrences of false block drops. The protocol also describes adding vibrotactile feedback while performing the motor task.

The application of incongruent sensory signals that involves disrupted tactile feedback is rarely explored, specifically with the presence of vibrotactile ...

Research

JoVE Journal

Behavior

Irrelevant Stimuli and Action Control: Analyzing the Influence of Ignored Stimuli via the Distractor-Response Binding Paradigm

Selection tasks in which simple stimuli (e.g. letters) are presented and a target stimulus has to be selected against one or more distractor stimuli are ...

Applying Stereotactic Injection Technique to Study Genetic Effects on Animal Behaviors

Stereotactic injection is a useful technique to deliver high titer lentiviruses to targeted brain areas in mice. Lentiviruses can either overexpress or ...

Force and Position Control in Humans - The Role of Augmented Feedback

During motor behaviour, humans interact with the environment by for example manipulating objects and this is only possible because sensory feedback is ...

A Method for Manipulating Blood Glucose and Measuring Resulting Changes in Cognitive Accessibility of Target Stimuli

Much research in social psychology has investigated the impact of bodily energy need on cognition and decision-making.  As such, blood glucose, the body's ...

Novel Object Exploration as a Potential Assay for Higher Order Repetitive Behaviors in Mice

Restricted, repetitive behaviors (RRBs) are a core feature of autism spectrum disorder (ASD) and disrupt the lives of affected individuals.  RRBs are ...

Novel Object Recognition Test for the Investigation of Learning and Memory in Mice

The object recognition test (ORT) is a commonly used behavioral assay for the investigation of various aspects of learning and memory in mice. The ORT is ...

The (Spatial) Memory Game: Testing the Relationship Between Spatial Language, Object Knowledge, and Spatial Cognition

The Spatial Memory Game: Testing the Relationship Between Spatial Language, Object Knowledge, and Spatial Cognition

The memory game paradigm is a behavioral procedure to explore the relationship between language, spatial memory, and object knowledge. Using two different ...

Control of Eating Behavior Using a Novel Feedback System

Subjects eat food from a plate that sits on a scale connected to a computer that records the weight loss of the plate during the meal and makes up a curve ...

Novel Object Recognition and Object Location Behavioral Testing in Mice on a Budget

Ethologically relevant behavioral testing is a critical component of any study that uses mouse models to study the cognitive effects of various ...

A Vibrotactile Feedback Device for Seated Balance Assessment and Training

Postural perturbations, motion tracking, and sensory feedback are modern techniques used to challenge, assess, and train upright sitting, respectively. ...