The current body of work is supported by the National Institute on Aging at the National Institute of Health (K01AG049813 to JRM). Supplementary funding was provided by the Resnick Gerontology Center of the Albert Einstein College of Medicine. Special thanks to all the volunteers and research staff for exceptional support with this project.

All participants provided written informed consent to the experimental procedures, which were approved by the institutional review board of the Albert Einstein College of Medicine.
1. Participant Recruitment, Inclusion Criteria, and Consent
<ol>
	<li>Recruit a relatively large cohort of English-speaking individuals who can ambulate independently and are free of significant sensory loss; active neurological or psychiatric disorders that interfere with experimental evaluations; and current/future medical procedures that affect mobility.</li>
	<li>Ensure that each participant can successfully complete a sensory screening exam, where visual, auditory, and somatosensory acuity are formally tested to confirm study appropriateness.
	<ol>
		<li>Use the Snellen eye chart to ensure that bilateral visual acuity is better than or equal to 20/100.</li>
		<li>Use a tone-emitting otoscope to ensure that participants are at a minimum able to hear a 2,000 Hz tone at 25 dB30.</li>
		<li>Determine whether participants maintain a diagnosis of clinical neuropathy and whether it interferes with the ability to feel the experimental somatosensory stimulation21,28,29.</li>
		<li>If the participant is unable to meet these minimum sensory requirements, do not include them in the study.</li>
	</ol>
	</li>
	<li>Exclude older adults with dementia by implementing cut-scores from reliable screening instruments such as the AD8 Dementia Screening Interview cutoff score &#8805;2 31,32; and the Memory Impairment Screen MIS; cutoff score &#60; 533.</li>
	<li>Have participants provide written informed consent to the experimental procedures (approved by a local institutional review board) if willing to participate.</li>
</ol>
2. Experimental Design
<ol>
	<li>Use stimulus presentation software to program a simple reaction time experiment with three experimental conditions: visual (V) alone, somatosensory (S) alone, and simultaneous visual-somatosensory (VS). Inform participants to respond to each sensory stimulus, regardless of the condition, as quickly as possible. See supplementary files for an example of a VS simple RT task (Supplementary File 1).
	<ol>
		<li>Use a stimulus generator with three control boxes (30.48 mm &#215; 20.32 mm &#215; 12.70 mm) and plastic housing for stimulators. The left and right control boxes contain bilateral blue light emitting diodes (LEDs; 15.88 cm diameter) that illuminate for visual stimulation and bilateral motors with 0.8 G vibration amplitude that vibrate for somatosensory stimulation (equivalent to a cell-phone vibration)22,23,28.</li>
		<li>Ensure that stimulus generators provide both unisensory (visual OR somatosensory alone), as well as multisensory (simultaneous visual AND somatosensory) stimulation. Place a center dummy control box equidistant (28 cm) from the left and right control boxes described in 2.1.1. and affix a visual target sticker (central circle of 0.4 cm diameter) to serve as the fixation point.</li>
		<li>Connect the stimulus generator to the experimental computer via the parallel port which allows the direct control for each stimulator.</li>
		<li>Program the stimulus presentation software to send transistor-transistor-logic (TTL, 5 V) pulses to the trigger stimulus generators on and off directly via the parallel port. Set the stimulus presentation time to 100 ms in duration.</li>
	</ol>
	</li>
	<li>In the stimulus presentation software, program a minimum of 3 experimental blocks each consisting of 45 trials (15 trials of each stimulus condition presented in random order) for a total of 135 stimulus presentations for this simple reaction time experiment.</li>
	<li>Vary the inter-stimulus-interval randomly between 1 and 3 s to prevent anticipatory effects. Alternatively, insert catch trials where the stimulus parameters are the same as above, but the TTL pulse is not sent, thus no visual or somatosensory stimulation occurs and, therefore, no response is expected.</li>
	<li>Allow participants up to 2,000 ms to respond to any given stimulus condition. If no response is detected within the 2,000 ms response period, ensure that the stimulus presentation software advances to the next trial automatically. 
	NOTE: This response window cut-off is arbitrary but necessary to keep the total experimental time to a minimum; note that longer RTs will be set to infinity regardless.</li>
	<li>Separate the three experimental blocks by programming 20-s breaks in the stimulus presentation software to reduce potential fatigue and increase concentration. Ensure each subsequent block starts immediately after the 20-s break concludes.</li>
	<li>Program written instructions to appear on the visual display (monitor of the experimental computer). The exact instructions are provided in the supplemental material. Ask the participant to start the experiment by pushing the response pad with their right foot when ready to commence. Once the stimulus parameters are programmed, the stimulus presentation software creates a script that is to be run on each participant.</li>
	<li>Provide participant ID and session number in order to run the experimental script. Once the experiment is completed, a unique behavioral data log is produced for each participant (see Supplementary File 2 for a sample Eprime 2.0 output file).</li>
</ol>
3. Apparatus &#38; Task
<ol>
	<li>Have participants sit upright and comfortably rest hands upon the left and right control boxes.
	<ol>
		<li>Strategically place index fingers over the vibratory motors mounted to the back of the control box, and thumbs on the front of the control box, under the LED to not block the light&#160;(see Figure 1).</li>
		<li>Ensure that the somatosensory stimuli are inaudible by providing participants with headphones over which continuous white noise is played at a comfortable level (typically 65-75 dBs).</li>
	</ol>
	</li>
	<li>Instruct participants to respond to all stimuli as quickly as possible.
	<ol>
		<li>Ask participants to use a foot-pedal located under the right foot as the response pad since fingers will be accepting somatosensory stimulation&#160;(see Figure 1).</li>
	</ol>
	</li>
	<li>Calculate performance accuracy by stimulus condition.
	<ol>
		<li>Instruct participants to respond to each of the experimental stimuli (45 per condition) as quickly as possible.</li>
		<li>Divide the number of accurately detected stimuli per condition over 45 (total number of trials per condition) to obtain measures of performance accuracy for visual, somatosensory, and VS conditions, respectively.</li>
	</ol>
	</li>
</ol>
4. Race Model Inequality Data Preparation (Individual Level)
<ol>
	<li>Determine whether an individual&#8217;s behavioral performance is valid.
	<ol>
		<li>Exclude participants that are not able to attain an accuracy of 70% correct or greater on any one stimulus condition As the participant&#8217;s performance accuracy on a simple reaction time task decreases, so does the reliability of the individual&#8217;s data.</li>
		<li>Consider trials inaccurate (omitted) if a participant fails to respond to a stimulus within the set response time period and set corresponding RT to infinity rather than excluding the trial from the analysis25,28. 
		NOTE: In previous studies, the group-averaged (n=289) stimulus detection was 96% across all conditions, and over 90 % of the population had detection rates above 90% for all conditions28.</li>
		<li>Do not employ data-trimming procedures that delete very slow RTs as this will bias the distribution of RT data.25 Ensure RTs that are clearly outliers are set to infinity. See supplementary file depicting alterations in CDFs based on data-trimming procedures and inclusion of slow RTs (Supplementary File 3).</li>
	</ol>
	</li>
	<li>Organize the RT Data.
	<ol>
		<li>Sort RT data in ascending order by the experimental condition. Place visual, somatosensory, and VS conditions in separate columns of sorted RT data. Ensure each row represents one trial and each cell represents the actual RT (or infinity in the case of omitted or slow trials).</li>
	</ol>
	</li>
	<li>Bin the RT Data.
	<ol>
		<li>Identify the fastest RT (to whichever condition- orange ellipse) and the slowest RT (to whichever condition-red ellipse). Subtract the slowest RT from the fastest (e.g., 740 ms &#8211; 237 ms) in order to calculate the individual&#8217;s RT range (503ms; blue ellipse) across ALL test conditions. Table 1&#160;demonstrates how to calculate an individual&#39;s RT Range and depicts the various color elipses.</li>
		<li>Bin RT data from the 0% (fastest RT = 237 in this example) to the 100% (or slowest RT = 740 in this example) in 5% increments by taking the fastest RT and gradually adding 5% of the RT range identified in 4.3.1 until 100% of the RT data is accounted for (see Table 2). This will result in 21-time bins. 
		NOTE: In Table 2 - 1%ile is only included in worksheet just for illustrative purposes.</li>
	</ol>
	</li>
	<li>Calculate the Cumulative Distribution Frequency (CDF) for the experimental conditions.
	<ol>
		<li>Using spreadsheet software, use a &#8220;FREQUENCY&#8221; function where array1 equals the actual RTs for one of the experimental conditions and array2 equals the 21 quantized RTs bins calculated in Step 4.3, divided by the total number of trials (45) per condition. This is illustrated in Figure 2a.</li>
		<li>Repeat this function for the other two experimental conditions (Figure&#160;2b-2c) so as to populate frequencies (or probability (P)) of an RT occurring within each of the 21 quantized time bins, for each of the three experimental conditions.</li>
		<li>Next, create the cumulative distribution frequency (CDF) by summing the running total of probabilities across the quantized bins (0%, 0+5%, 0+5+10%, 0+5+10+15%, etc.) for each of three experimental conditions. For example, in the cumulative probability column for the Soma condition (column AE), the cumulative probability for the 95%ile range (cell AE22) is the summation of the probability values in cells Z3:Z23 (see Figure 3).</li>
	</ol>
	</li>
	<li>Actual vs. Predicted CDFs.
	<ol>
		<li>Ensure that the CDF of the multisensory condition represents the actual CDF (see Figure 4 column AF and plotted purple trace). To calculate the predicted CDF (column AG), sum the two unisensory CDFs (with an upper limit set to 1)&#160;across each of the 21 quantized time bins (see Figure 5). Start at the 0th percentile (bin 1) and continue all the way down to the 100th percentile (bin 21).</li>
	</ol>
	</li>
	<li>Conduct the Test of the Race Model Inequality (RMI).
	<ol>
		<li>Subtract the predicted CDF (calculated in 4.5.2.) from the actual CDF for each of the 21 quantized time bins to obtain the difference values (column AH; see Figure 6).</li>
		<li>Plot these 21 values as a line graph, where the x-axis represents each one of the quantized time bins (column AC) and the y-axis represents the probability difference between the actual and predicted CDFs (column AH; Figure 7 (black trace).</li>
		<li>Check for positive values at any latency (i.e., quantiles) which indicate the integration of the unisensory stimuli and reflect a violation of the RMI (see the green highlighted portion of difference wave from 0.00 &#8211; 0.10 in Figure 7).</li>
	</ol>
	</li>
</ol>
5. Quantification of the Multisensory Effect (Group Level).
<ol>
	<li>Group-average the individual RMI data (differences between predicted CDF and the actual CDF for each of the 21-time bins; step 4.6.1- column AH) across all participants. Use a spreadsheet software to assign&#160;individuals to rows and time bins as columns. In a new spreadsheet, place the 21 values calculated in 4.6.1 in individual rows (1 row per participant), and average values within time bins to create one group-averaged difference waveform.</li>
	<li>Plot the group average 21 values as a line graph, where the x-axis represents each one of the quantized time bins and the y-axis represents the probability difference between CDFs.</li>
	<li>Visually inspect and document the violated portion of the group-averaged difference wave (i.e., positive values).</li>
	<li>Run Gondan&#8217;s RMI permutation test (R script available as a free download)26 to determine whether there is a statistically significant violation of the RMI over the positive values identified in step 5.3.
	<ol>
		<li>Organize the data in one text file where the first column is named &#8220;Obs&#8221; for Observer (e.g., participant ID), the second column is named &#8220;Cond&#8221; for stimulus condition (V, S, or VS) and the third column is named &#8220;RT&#8221; for actual RT or &#8220;Inf&#8221; if set to infinity.</li>
		<li>Open the software, identify which time bins are to be tested (based on the positive time bins identified in 5.3), and enter the text file name created in 5.4.1.</li>
		<li>Run the test by calling up the script. The results will provide a tmax value, 95% criterion, and p-value which will be instrumental in determining whether a significant violation of the Race Model exists across the entire study sample.</li>
	</ol>
	</li>
	<li>Calculate the area-under-the-curve (AUC) for each individual after establishing the significantly violated percentile bins in step 5.3. AUC will serve as the magnitude of multisensory integration (or the independent variable). To calculate AUC use participant 1&#8217;s data as an example, for percentile bins 0.00 - 0.15 depicted in Figure&#160;8a-d).
	<ol>
		<li>Sum the CDF difference value at time bin 1 (1st time positive value) with the CDF difference value of time bin 2 (next positive value) and then divide by two (see Figure 8a). Repeat step 5.3.1. for each consecutive pair of time bins containing positive values (see Figure 8b-8c).</li>
		<li>Sum the results obtained from steps 5.5.1 - 5.5.2. to generate the total AUC of the CDF difference wave during the violated percentile range (e.g., 0.00 &#8211; 0.15 in Figure 8d). 
		NOTE: AUC is a continuous measure and one AUC value is present for each individual for the violated portion of the RMI (Figure 8d red ellipse = participant 1&#8217;s AUC = 0.13). AUC can be used as an independent variable representing &#8216;magnitude of VS integration&#8217; which can later be tested to predict important clinical outcome measures (see also28,29).</li>
	</ol>
	</li>
	<li>Assign multisensory integration classification groups based on the number of violated percentile bins (values greater than zero highlighted in gray in Table 3) during the significantly violated percentile range identified above in step 5.3. Looking at Table 3 (percentile bins 0.00 &#8211; 0.15): Participant 1 has positive values for 2 out of 4 bins; Participant 2 has positive values for 4 out of 4 bins; and Participant 3 has positive values for 0 out of 4 bins.
	<ol>
		<li>Operationalize a classification system based on the number of violated percentile bins (values greater than zero for 0, 1, 2, or 3 bins) during the 0-10th percentile.&#160;</li>
		<li>Figure 9&#160;depicts one potential classification definition which is adapted from recently published data presented by Mahoney and Verghese29.</li>
	</ol>
	</li>
</ol>

<ol>
	<li>Foxe, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Auditory-somatosensory+multisensory+processing+in+auditory+association+cortex:+an+fMRI+study.">Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study.</a> Journal of Neurophysiology. 88 (1), 540-543 (2002).</li><li>Molholm, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+auditory-visual+interactions+during+early+sensory+processing+in+humans:+a+high-density+electrical+mapping+study.">Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study.</a> Brain Research: Cognitive Brain Research. 14 (1), 115-128 (2002).</li><li>Murray, M. M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Grabbing+your+ear:+rapid+auditory-somatosensory+multisensory+interactions+in+low-level+sensory+cortices+are+not+constrained+by+stimulus+alignment.">Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment.</a> Cerebral Cortex. 15 (7), 963-974 (2005).</li><li>Molholm, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Audio-visual+multisensory+integration+in+superior+parietal+lobule+revealed+by+human+intracranial+recordings.">Audio-visual multisensory integration in superior parietal lobule revealed by human intracranial recordings.</a> Journal of Neurophysiology. 96 (2), 721-729 (2006).</li><li>Peiffer, A. M., Mozolic, J. L., Hugenschmidt, C. E., Laurienti, P. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Age-related+multisensory+enhancement+in+a+simple+audiovisual+detection+task.">Age-related multisensory enhancement in a simple audiovisual detection task.</a> Neuroreport. 18 (10), 1077-1081 (2007).</li><li>Brandwein, A. B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+development+of+audiovisual+multisensory+integration+across+childhood+and+early+adolescence:+a+high-density+electrical+mapping+study.">The development of audiovisual multisensory integration across childhood and early adolescence: a high-density electrical mapping study.</a> Cerebral Cortex. 21 (5), 1042-1055 (2011).</li><li>Girard, S., Collignon, O., Lepore, F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+gain+within+and+across+hemispaces+in+simple+and+choice+reaction+time+paradigms.">Multisensory gain within and across hemispaces in simple and choice reaction time paradigms.</a> Experimental Brain Research. 214 (1), 1-8 (2011).</li><li>Mahoney, J. R., Li, P. C., Oh-Park, M., Verghese, J., Holtzer, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+integration+across+the+senses+in+young+and+old+adults.">Multisensory integration across the senses in young and old adults.</a> Brain Research. 1426, 43-53 (2011).</li><li>Foxe, J. J., Ross, L. A., Molholm, S., Stein, B. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Ch.+38.">Ch. 38.</a> The New Handbook of Multisensory Processing. , 691-706 (2012).</li><li>Kinchla, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Detecting+target+elements+in+multielement+arrays:+A+confusability+model.">Detecting target elements in multielement arrays: A confusability model.</a> Perception and Psychophysics. 15, 149-158 (1974).</li><li>Miller, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Divided+attention:+Evidence+for+coactivation+with+redundant+signals.">Divided attention: Evidence for coactivation with redundant signals.</a> Cognitive Psychology. 14 (2), 247-279 (1982).</li><li>Eriksen, C. W., Goettl, B., St James, J. D., Fournier, L. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Processing+redundant+signals:+coactivation,+divided+attention,+or+what?.">Processing redundant signals: coactivation, divided attention, or what?.</a> Perception and Psychophysics. 45 (4), 356-370 (1989).</li><li>Mordkoff, J. T., Yantis, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+interactive+race+model+of+divided+attention.">An interactive race model of divided attention.</a> Journal of Experimental Psychology: Human Perception and Performance. 17 (2), 520-538 (1991).</li><li>Miller, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Timecourse+of+coactivation+in+bimodal+divided+attention.">Timecourse of coactivation in bimodal divided attention.</a> Perception and Psychophysics. 40 (5), 331-343 (1986).</li><li>Gondan, M., Lange, K., Rosler, F., Roder, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+redundant+target+effect+is+affected+by+modality+switch+costs.">The redundant target effect is affected by modality switch costs.</a> Psychonomic Bulletin Review. 11 (2), 307-313 (2004).</li><li>Colonius, H., Diederich, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+race+model+inequality:+interpreting+a+geometric+measure+of+the+amount+of+violation.">The race model inequality: interpreting a geometric measure of the amount of violation.</a> Psychological Review. 113 (1), 148-154 (2006).</li><li>Maris, G., Maris, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Testing+the+race+model+inequality:+A+nonparametric+approach.">Testing the race model inequality: A nonparametric approach.</a> Journal of Mathematical Psychology. 47 (5-6), 507-514 (2003).</li><li>Clark, J. J., Yuille, A. L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Data Fusion for Sensory Information Processing Systems. , (1990).</li><li>Ernst, M. O., Banks, M. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Humans+integrate+visual+and+haptic+information+in+a+statistically+optimal+fashion.">Humans integrate visual and haptic information in a statistically optimal fashion.</a> Nature. 415 (6870), 429-433 (2002).</li><li>Mahoney, J. R., Verghese, J., Dumas, K., Wang, C., Holtzer, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+effect+of+multisensory+cues+on+attention+in+aging.">The effect of multisensory cues on attention in aging.</a> Brain Research. 1472, 63-73 (2012).</li><li>Mahoney, J. R., Holtzer, R., Verghese, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual-somatosensory+integration+and+balance:+evidence+for+psychophysical+integrative+differences+in+aging.">Visual-somatosensory integration and balance: evidence for psychophysical integrative differences in aging.</a> Multisensory Research. 27 (1), 17-42 (2014).</li><li>Mahoney, J. R., Dumas, K., Holtzer, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual-Somatosensory+Integration+is+linked+to+Physical+Activity+Level+in+Older+Adults.">Visual-Somatosensory Integration is linked to Physical Activity Level in Older Adults.</a> Multisensory Research. 28 (1-2), 11-29 (2015).</li><li>Dumas, K., Holtzer, R., Mahoney, J. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual-Somatosensory+Integration+in+Older+Adults:+Links+to+Sensory+Functioning.">Visual-Somatosensory Integration in Older Adults: Links to Sensory Functioning.</a> Multisensory Research. 29 (4-5), 397-420 (2016).</li><li>Couth, S., Gowen, E., Poliakoff, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Using+race+model+violation+to+explore+multisensory+responses+in+older+adults:+Enhanced+multisensory+integration+or+slower+unisensory+processing.">Using race model violation to explore multisensory responses in older adults: Enhanced multisensory integration or slower unisensory processing.</a> Multisensory Research. 31 (3-4), 151-174 (2017).</li><li>Gondan, M., Minakata, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+tutorial+on+testing+the+race+model+inequality.">A tutorial on testing the race model inequality.</a> Attention, Perception &amp; Psychophysics. 78 (3), 723-735 (2016).</li><li>Gondan, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+permutation+test+for+the+race+model+inequality.">A permutation test for the race model inequality.</a> Behavior Research Methods. 42 (1), 23-28 (2010).</li><li>Kiesel, A., Miller, J., Ulrich, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Systematic+biases+and+Type+I+error+accumulation+in+tests+of+the+race+model+inequality.">Systematic biases and Type I error accumulation in tests of the race model inequality.</a> Behavior Research Methods. 39 (3), 539-551 (2007).</li><li>Mahoney, J., Cotton, K., Verghese, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+Integration+Predicts+Balance+and+Falls+in+Older+Adults.">Multisensory Integration Predicts Balance and Falls in Older Adults.</a> Journal of Gerontology: Medical Sciences. , (2018).</li><li>Mahoney, J. R., Verghese, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual-Somatosensory+Integration+and+Quantitative+Gait+Performance+in+Aging.">Visual-Somatosensory Integration and Quantitative Gait Performance in Aging.</a> Frontiers in Aging Neuroscience. 10, 377 (2018).</li><li>Yueh, B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Long-term+effectiveness+of+screening+for+hearing+loss:+the+screening+for+auditory+impairment--which+hearing+assessment+test+(SAI-WHAT)+randomized+trial.">Long-term effectiveness of screening for hearing loss: the screening for auditory impairment--which hearing assessment test (SAI-WHAT) randomized trial.</a> Journal of the American Geriatrics Society. 58 (3), 427-434 (2010).</li><li>Galvin, J. E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+AD8:+a+brief+informant+interview+to+detect+dementia.">The AD8: a brief informant interview to detect dementia.</a> Neurology. 65 (4), 559-564 (2005).</li><li>Galvin, J. E., Roe, C. M., Xiong, C., Morris, J. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Validity+and+reliability+of+the+AD8+informant+interview+in+dementia.">Validity and reliability of the AD8 informant interview in dementia.</a> Neurology. 67 (11), 1942-1948 (2006).</li><li>Buschke, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Screening+for+dementia+with+the+memory+impairment+screen.">Screening for dementia with the memory impairment screen.</a> Neurology. 52 (2), 231-238 (1999).</li></ol>

There are no conflicts of interest to report and the authors have nothing to disclose.

The goal of the current study was to detail the process behind the establishment of a robust multisensory integration phenotype. Here, we provide the necessary and critical steps required to acquire multisensory integration effects that can be utilized to predict important cognitive and motor outcomes relying on similar neural circuitry. Our overall objective was to provide a step-by-step tutorial for calculating the magnitude of multisensory integration in an effort to facilitate innovative and novel translational multi...

Interactions across sensory systems are essential for everyday functions. While multisensory integration effects are measured across a wide array of populations using assorted sensory combinations and different neuroscience approaches [including but not limited to the psychophysical, electrophysiological, and neuroimaging methodologies]1,2,3,4,5,6,7,8,9, currently a gold standard for quantifying multisensory integration is lacking. Given that multisensory experiments typically contain&#160;a behavioral component, reaction time (RT) data is often examined to determine the existence of a well-known phenomenon called the redundant signals effect10. As its name suggests, simultaneous sensory signals provide redundant information, which typically yield quicker RTs. Race and co-activation models are used to explain the above mentioned redundant signals effect11. Under race models, the unisensory signal that is processed the fastest is the winner of the race and is responsible for producing the behavioral response. However, evidence for co-activation occurs when responses to multisensory stimuli are quicker than what race models predict.
Earlier versions of the race model are inherently controversial12,13 as they are referred to by some as overly conservative14,15 and purportedly contain limitations regarding the independence between the constituent unisensory detection times inherent in the multisensory condition16. In an effort to address some of these limitations, Colonius &#38; Diederich16 developed a more conventional race model test:
<img alt="Equation 1" src="/files/ftp_upload/59575/59575eq1.jpg" />,
where the cumulative distribution frequencies (CDFs) of the unisensory conditions (e.g., A &#38; B; with an upper limit of one) are compared to the CDF of the simultaneous multisensory condition (e.g., AB) for any given latency (t)11,16,17. In general, a CDF determines how often an RT occurs, within a given range of RTs, divided by the total number of stimulus presentations (i.e., trials). If the CDF of the actual multisensory condition <img alt="Equation 2" src="/files/ftp_upload/59575/59575eq2.jpg" /> is less than or equal to the predicted CDF derived from the unisensory conditions
<img alt="Equation 3" src="/files/ftp_upload/59575/59575eq3.jpg" />,
then the race model is accepted and there is no evidence for sensory integration. However, when the multisensory CDF is greater than the predicted CDF derived from the unisensory conditions, the race model is rejected. Rejection of the race model indicates that multisensory interactions from redundant sensory sources combine&#160;in a non-linear manner, resulting in a speeding up of RTs (e.g., RT facilitation) to multisensory stimuli.
One main hurdle that multisensory researchers face is how to best quantify integration effects. For instance, in the case of the most basic behavioral multisensory paradigm, where participants are asked to perform a simple reaction time task, information regarding accuracy and speed is collected. Such multisensory data can be used at the face-value or manipulated using various mathematical applications including but not limited to Maximum Likelihood Estimation18,19, CDFs11, and various other statistical approaches. The majority of our previous multisensory studies employed both quantitative and probabilistic approaches where multisensory integrative effects were calculated by 1) subtracting the mean reaction time (RT) to a multisensory event from the mean reaction time (RT) to the shortest unisensory event, and 2) by employing CDFs to determine whether RT facilitation resulted from synergistic interactions facilitated by redundant sensory information8,20,21,22,23. However, the former methodology was likely not sensitive to the individual differences in integrative processes and researchers have since posited that the later methodology (i.e., CDFs) may provide a better proxy for quantifying multisensory integrative effects24.
Gondan and Minakata recently published a tutorial on how to accurately test the Race Model Inequality (RMI) since researchers all too often make countless errors during the acquisition and pre-processing stages of RT data collection and preparation25. First, the authors posit that is unfavorable to apply data trimming procedures where certain a priori minimum and maximum RT limits are set. They recommend that slow and omitted responses be set to infinity, rather than excluded. Second, given that the RMI may be violated at any latency, multiple t-tests are often used to test the RMI at different time points (i.e., quantiles); unfortunately, this practice leads to the increased Type I error and substantially reduced statistical power. To avoid these issues, it is recommended that the RMI be tested over one specific time range. Some researchers have suggested that it makes sense to test the fastest quartile of responses (0-25%)26 or some pre-identified windows (i.e., 10-25%)24,27 as multisensory integration effects are typically observed during that time interval; however, we argue that the percentile range to be tested must be dictated by the actual dataset (see Protocol Section 5). The problem with relying on published data from young adults or computer simulations is that older adults manifest very different RT distributions, likely due to the age-related declines in sensory systems. Race model significance testing should only be tested over violated portions (positive values) of group-averaged difference wave between actual and predicted CDFs from the study cohort.
To this end, a protective effect of multisensory integration in healthy older adults using the conventional test of the race model16 and the principles set forth by Gondan and colleagues25 has been demonstrated. In fact, greater magnitude of visual-somatosensory RMI (a proxy for multisensory integration) was found to be linked to better balance performance, lower probability of incident falls and increased spatial gait performance28,29.
The objective of the current experiment is to provide researchers with a step-by-step tutorial to calculate the magnitude of multisensory integration effects using the RMI, to facilitate the increased production of diverse translational research studies across many different clinical populations. Note that data presented in the current study are from recently published visual-somatosensory experiments conducted on healthy older adults28,29, but this methodology can be applied to various cohorts across many different experimental designs, utilizing a wide-array of multisensory combinations.

<table>
	<tbody>
		<tr>
			<td>stimulus generator</td>
			<td>Zenometrics, LLC; Peekskill, NY, USA</td>
			<td>n/a</td>
			<td>custom-built</td>
		</tr>
		<tr>
			<td>Excel</td>
			<td>Microsoft Corporation</td>
			<td></td>
			<td>spreadsheet program</td>
		</tr>
		<tr>
			<td>Eprime</td>
			<td>Psychology Software Tools (PST)</td>
			<td></td>
			<td>stimulus presentation software</td>
		</tr>
	</tbody>
</table>

using the race model inequality to quantify behavioral multisensory integration effects

Multisensory integration research investigates how the brain processes simultaneous sensory information. Research on animals (mainly cats and primates) and humans reveal that intact multisensory integration is crucial for functioning in the real world, including both cognitive and physical activities. Much of the research conducted over the past several decades documents multisensory integration effects using diverse psychophysical, electrophysiological, and neuroimaging techniques. While its presence has been reported, the methods used to determine the magnitude of multisensory integration effects varies and typically faces much criticism. In what follows, limitations of previous behavioral studies are outlined and a step-by-step tutorial for calculating the magnitude of multisensory integration effects using robust probability models is provided.

The purpose of this study was to provide a step-by-step tutorial of a methodical approach to quantify the magnitude of VS integration effects, to foster the publication of new multisensory studies using similar experimental designs and setups (see Figure 1). Screenshots of each step and calculation needed to derive magnitude of multisensory integration effects, as measured by RMI AUC, are delineated above and illustrated in Figures&#160;2-8.
<stro...

Watch this Scientific Journal Video about Using the Race Model Inequality to Quantify Behavioral Multisensory Integration Effects at JoVE.com

Using the Race Model Inequality to Quantify Behavioral Multisensory Integration Effects

The current study aims to provide a step-by-step tutorial for calculating the magnitude of multisensory integration effects in an effort to facilitate the production of translational research studies across diverse clinical populations.

Multisensory integration research investigates how the brain processes simultaneous sensory information. Research on animals (mainly cats and primates) ...

using-race-model-inequality-to-quantify-behavioral-multisensory

Research

JoVE Journal

Behavior

6.2K Views.  Albert Einstein College of Medicine. The current study aims to provide a step-by-step tutorial for calculating the magnitude of multisensory integration effects in an effort to facilitate the production of translational research studies across diverse clinical populations.

Video: Using the Race Model Inequality to Quantify Behavioral Multisensory Integration Effects

Behavioral Phenotyping of Murine Disease Models with the Integrated Behavioral Station (INBEST)

Due to rapid advances in genetic engineering, small rodents have become the preferred subjects in many disciplines of biomedical research. In studies of chronic CNS disorders, there is an increasing demand for murine models with high validity at the behavioral level. However, multiple pathogenic mechanisms and complex functional deficits often impose challenges to reliably measure and interpret behavior of chronically sick mice. Therefore, the assessment of peripheral pathology and a behavioral profile at several time points using a battery of tests are required. Video-tracking, behavioral spectroscopy, and remote acquisition of physiological measures are emerging technologies that allow for comprehensive, accurate, and unbiased behavioral analysis in a home-base-like setting. This report describes a refined phenotyping protocol, which includes a custom-made monitoring apparatus (Integrated Behavioral Station, INBEST) that focuses on prolonged measurements of basic functional outputs, such as spontaneous activity, food/water intake and motivated behavior in a relatively stress-free environment. Technical and conceptual improvements in INBEST design may further promote reproducibility and standardization of behavioral studies.

Behavioral Phenotyping of Murine Disease Models with the Integrated Behavioral Station INBEST

Prolonged and comprehensive monitoring of mice in a home-cage environment provides a deeper understanding of aberrant behavior in murine models of brain diseases. This paper describes the Integrated Behavioral Station (INBEST) as the key component of contemporary behavioral analysis.

Due to rapid advances in genetic engineering, small rodents have become the preferred subjects in many disciplines of biomedical research. In studies of ...

Applying Stereotactic Injection Technique to Study Genetic Effects on Animal Behaviors

Stereotactic injection is a useful technique to deliver high titer lentiviruses to targeted brain areas in mice. Lentiviruses can either overexpress or knockdown gene expression in a relatively focused region without significant damage to the brain tissue. After recovery, the injected mouse can be tested on various behavioral tasks such as the Open Field Test (OFT) and the Forced Swim Test (FST). The OFT is designed to assess locomotion and the anxious phenotype in mice by measuring the amount of time that a mouse spends in the center of a novel open field. A more anxious mouse will spend significantly less time in the center of the novel field compared to controls. The FST assesses the anti-depressive phenotype by quantifying the amount of time that mice spend immobile when placed into a bucket of water. A mouse with an anti-depressive phenotype will spend significantly less time immobile compared to control animals. The goal of this protocol is to use the stereotactic injection of a lentivirus in conjunction with behavioral tests to assess how genetic factors modulate animal behaviors.

Stereotactic injection of lentiviruses expressing cDNAs or shRNAs can modulate gene expression in specific brain areas of mice. Here, we present a protocol to combine stereotactic injections with behavioral tasks, such as the Open Field Test (OFT) and the Forced Swim Test (FST).

Stereotactic injection is a useful technique to deliver high titer lentiviruses to targeted brain areas in mice. Lentiviruses can either overexpress or ...

Vagus Nerve Stimulation as a Tool to Induce Plasticity in Pathways Relevant for Extinction Learning

Extinction describes the process of attenuating behavioral responses to neutral stimuli when they no longer provide the reinforcement that has been maintaining the behavior. There is close correspondence between fear and human anxiety, and therefore studies of extinction learning might provide insight into the biological nature of anxiety-related disorders such as post-traumatic stress disorder, and they might help to develop strategies to treat them. Preclinical research aims to aid extinction learning and to induce targeted plasticity in extinction circuits to consolidate the newly formed memory. Vagus nerve stimulation (VNS) is a powerful approach that provides tight temporal and circuit-specific release of neurotransmitters, resulting in modulation of neuronal networks engaged in an ongoing task. VNS enhances memory consolidation in both rats and humans, and pairing VNS with exposure to conditioned cues enhances the consolidation of extinction learning in rats. Here, we provide a detailed protocol for the preparation of custom-made parts and the surgical procedures required for VNS in rats. Using this protocol we show how VNS can facilitate the extinction of conditioned fear responses in an auditory fear conditioning task. In addition, we provide evidence that VNS modulates synaptic plasticity in the pathway between the infralimbic (IL) medial prefrontal cortex and the basolateral complex of the amygdala (BLA), which is involved in the expression and modulation of extinction memory.

Vagus nerve stimulation (VNS) has emerged as a tool to induce targeted synaptic plasticity in the forebrain to modify a range of behaviors. This protocol describes how to implement VNS to facilitate the consolidation of fear extinction memory.

Extinction describes the process of attenuating behavioral responses to neutral stimuli when they no longer provide the reinforcement that has been ...

Behavioral Disturbances: An Innovative Approach to Monitor the Modulatory Effects of a Nutraceutical Diet

In dogs, diets are often used to modulate behavioral disturbances related to chronic anxiety and stress caused by intense and restless activity. However, the traditional ways to monitor behavioral changes in dogs are complicated and not efficient. In the current clinical evaluation, a new, simple monitoring system was used to assess the effectiveness of a specific diet in positively modulating the intense and restless activity of 24 dogs of different ages and breeds. This protocol describes how to easily and rapidly evaluate improvement in a set of symptoms related to generalized anxiety by using a specific sensor, a mobile phone app, a wireless router, and a computer. The results showed that dogs treated with specific diets showed significant improvement in the times spent active and at rest after 10 days (p &lt; 0.01 and p &lt; 0.05, respectively). These dogs also showed an overall significant improvement in clinical and behavioral symptoms. A specific sensor, along with its related hardware, was demonstrated to successfully monitor behavioral changes relating to movement in dogs.

We report a simple approach to evaluate the effectiveness of a specific diet in positively modulating the daily activity and clinical and behavioral symptoms of dogs with evident behavioral disturbances.

In dogs, diets are often used to modulate behavioral disturbances related to chronic anxiety and stress caused by intense and restless activity. However, ...

Using the FishSim Animation Toolchain to Investigate Fish Behavior: A Case Study on Mate-Choice Copying In Sailfin Mollies

Over the last decade, employing computer animations for animal behavior research has increased due to its ability to non-invasively manipulate the appearance and behavior of visual stimuli, compared to manipulating live animals. Here, we present the FishSim Animation Toolchain, a software framework developed to provide researchers with an easy-to-use method for implementing 3D computer animations in behavioral experiments with fish. The toolchain offers templates to create virtual 3D stimuli of five different fish species. Stimuli are customizable in both appearance and size, based on photographs taken of live fish. Multiple stimuli can be animated by recording swimming paths in a virtual environment using a video game controller. To increase standardization of the simulated behavior, the prerecorded swimming path may be replayed with different stimuli. Multiple animations can later be organized into playlists and presented on monitors during experiments with live fish.
In a case study with sailfin mollies (Poecilia latipinna), we provide a protocol on how to conduct a mate-choice copying experiment with FishSim. We utilized this method to create and animate virtual males and virtual model females, and then presented these to live focal females in a binary choice experiment. Our results demonstrate that computer animation may be used to simulate virtual fish in a mate-choice copying experiment to investigate the role of female gravid spots as an indication of quality for a model female in mate-choice copying.
Applying this method is not limited to mate-choice copying experiments but can be used in various experimental designs. Still, its usability depends on the visual capabilities of the study species and first needs validation. Overall, computer animations offer a high degree of control and standardization in experiments and bear the potential to &#39;reduce&#39; and &#39;replace&#39; live stimulus animals as well as to &#39;refine&#39; experimental procedures.

Using the FishSim Animation Toolchain to Investigate Fish Behavior: A Case Study on Mate-Choice Copying In Sailfin Mollies

Using the novel FishSim Animation Toolchain, we present a protocol for non-invasive visual manipulation of public information in the context of mate-choice copying in sailfin mollies. FishSim Animation Toolchain provides an easy-to-use framework for the design, animation and presentation of computer-animated fish stimuli for behavioral experiments with live test fish.

Over the last decade, employing computer animations for animal behavior research has increased due to its ability to non-invasively manipulate the ...

Behavioral Assessments of Spontaneous Locomotion in a Murine MPTP-induced Parkinson's Disease Model

Parkinson&#8217;s disease (PD) is a common neurodegenerative disorder disease, causing the phenomenon of shaking, rigidity, slowness of movement and dementia. 1-Methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP) can lead to some Parkinson&#8217;s-like symptoms by destroying dopaminergic neurons in the substantia nigra of the brain. It has been thus used to establish PD models in various animal studies. Here, mice receive MPTP injections (20 mg/kg/day) for seven days and the behavioral tests are performed on the eighth day. This model is adapted efficiently in the study of PD. The behavioral tests here include the cylinder test and the open field test. The cylinder experiment is used to detect the animals&#8217; ability to lift their front paws when put into a different environment. As the PD model mice show arching&#8212;the mouse arches its back&#8212;the number of paw liftings decrease. This test is easy to execute. The open field test is used to detect the amount of time the mice spend on running, walking, and remaining immobile. We analyze animals&#8217; movements in open field using software and obtain data. Lastly, we use L-DOPA, one of the most commonly used PD drugs, as one example to show how to apply this model to the study of PD drugs. Our results indicate that MPTP neurotoxicity induces motor deficit which can be mitigated by L-DOPA.

We describe establishment of a murine model for Parkinson's disease using MPTP, and behavioral assessments using cylinder and open field tests to measure motor function. We then use L-DOPA as one example to show how to apply this model in the study of PD drugs.

Parkinson&#8217;s disease (PD) is a common neurodegenerative disorder disease, causing the phenomenon of shaking, rigidity, slowness of movement and ...

Assessing the Autonomic and Behavioral Effects of Passive Motion in Rats using Elevator Vertical Motion and Ferris-Wheel Rotation

The overall goal of this study is to assess the autonomic and behavioral effects of passive motion in rodents using the elevator vertical motion and Ferris-wheel rotation devices. These assays can help confirm the integrity and normal functioning of the autonomic nervous system. They are coupled to quantitative measures based on defecation counting, open-field examination, and balance beam crossing. The advantages of these assays are their simplicity, reproducibility, and quantitative behavioral measures. The limitations of these assays are that the autonomic reactions could be epiphenomena of non-vestibular disorders and that a functioning vestibular system is required. Examination of disorders such as motion sickness will be greatly aided by the detailed procedures of these assays.

Protocols are presented to assess the autonomic and behavioral effects of passive motion in rodents using elevator vertical motion and Ferris-wheel rotation.

The overall goal of this study is to assess the autonomic and behavioral effects of passive motion in rodents using the elevator vertical motion and ...

Using Practice Testing, Public Speaking, and Source Monitoring to Examine the Influences of Learning Strategies and Stress on Episodic Memory

Prior research demonstrated that learning information via retrieval practice, which entails studying and taking practice tests, resulted in less memory impairment under stress than learning information via repeated studying. The present experiment combined three experimental procedures to further examine the memory mechanisms underlying the efficacy of retrieval practice in the context of stress. A list-discrimination task was implemented, in which participants learned two distinct wordlists. This was combined with a retrieval-practice manipulation, as half of the participants engaged in practice testing and half engaged in conventional studying during learning. A week later, participants underwent stress induction, using the Trier Social Stress Test. Before and after stress induction, participants completed tests of item and source memory (i.e., list discrimination). The combination of these three procedures yielded informative results: retrieval practice, in the context of stress, improved item memory but not source memory relative to conventional studying. Limitations and future directions for the use of this methodology are discussed.

The present experiment combined three experimental procedures &#8212; a retrieval-practice learning manipulation, a list-discrimination task, and a stress-induction technique &#8212; to examine the influences of different learning strategies and acute stress on multiple measures of episodic memory.

Prior research demonstrated that learning information via retrieval practice, which entails studying and taking practice tests, resulted in less memory ...

The Three-Chamber Choice Behavioral Task using Zebrafish as a Model System

Neurodegenerative diseases are age-dependent, debilitating, and incurable. Recent reports have also correlated hyperglycemia with changes in memory and/or cognitive impairment. We have modified and developed a three-chamber choice cognitive task similar to that used with rodents for use with hyperglycemic zebrafish. The testing chamber consists of a centrally located starting chamber and two choice compartments on either side, with a shoal of conspecifics used as the reward. We provide data showing that once acquired, zebrafish remember the task at least 8 weeks later. Our data indicate that zebrafish respond robustly to this reward, and we have identified cognitive deficits in hyperglycemic fish after 4 weeks of treatment. This behavioral assay may also be applicable to other studies related to cognition and memory.

We present a behavioral chamber designed to assess cognitive performance. We provide data showing that once acquired, zebrafish remember the task 8 weeks later. We also show that hyperglycemic zebrafish have altered cognitive performance, indicating that this paradigm is applicable to studies assessing cognition and memory.

Neurodegenerative diseases are age-dependent, debilitating, and incurable. Recent reports have also correlated hyperglycemia with changes in memory and/or ...

Real-Time Proxy-Control of Re-Parameterized Peripheral Signals using a Close-Loop Interface

The fields that develop methods for sensory substitution and sensory augmentation have aimed to control external goals using signals from the central nervous systems (CNS). Less frequent however, are protocols that update external signals self-generated by interactive bodies in motion. There is a paucity of methods that combine the body-heart-brain biorhythms of one moving agent to steer those of another moving agent during dyadic exchange. Part of the challenge to accomplish such a feat has been the complexity of the setup using multimodal bio-signals with different physical units, disparate time scales and variable sampling frequencies.
In recent years, the advent of wearable bio-sensors that can non-invasively harness multiple signals in tandem, has opened the possibility to re-parameterize and update the peripheral signals of interacting dyads, in addition to improving brain- and/or body-machine interfaces. Here we present a co-adaptive interface that updates efferent somatic-motor output (including kinematics and heart rate) using biosensors; parameterizes the stochastic bio-signals, sonifies this output, and feeds it back in re-parameterized form as visuo/audio-kinesthetic reafferent input. We illustrate the methods using two types of interactions, one involving two humans and another involving a human and its avatar interacting in near real time. We discuss the new methods in the context of possible new ways to measure the influences of external input on internal somatic-sensory-motor control.

We present protocols and methods of analyses to build co-adaptive interfaces that stream, parameterize, analyze, and modify human body and heart signals in close-loop. This setup interfaces signals derived from the peripheral and central nervous systems of the person with external sensory inputs to help track biophysical change.

The fields that develop methods for sensory substitution and sensory augmentation have aimed to control external goals using signals from the central ...