Conducting Hyperscanning Experiments with Functional Near-Infrared Spectroscopy

Summary

The present protocol describes how to conduct fNIRS hyperscanning experiments and analyze brain-to-brain synchrony. Further, we discuss challenges and possible solutions.

Abstract

Concurrent brain recordings of two or more interacting persons, an approach termed hyperscanning, are gaining increasing importance for our understanding of the neurobiological underpinnings of social interactions, and possibly interpersonal relationships. Functional near-infrared spectroscopy (fNIRS) is well suited for conducting hyperscanning experiments because it measures local hemodynamic effects with a high sampling rate and, importantly, it can be applied in natural settings, not requiring strict motion restrictions. In this article, we present a protocol for conducting fNIRS hyperscanning experiments with parent-child dyads and for analyzing brain-to-brain synchrony. Furthermore, we discuss critical issues and future directions, regarding the experimental design, spatial registration of the fNIRS channels, physiological influences and data analysis methods. The described protocol is not specific to parent-child dyads, but can be applied to a variety of different dyadic constellations, such as adult strangers, romantic partners or siblings. To conclude, fNIRS hyperscanning has the potential to yield new insights into the dynamics of the ongoing social interaction, which possibly go beyond what can be studied by examining the activities of individual brains.

Introduction

In recent years, neuroscientists have started to study social interactions by recording the brain activities of two or more persons simultaneously, an approach termed hyperscanning¹. This technique opens new opportunities to elucidate the neurobiological mechanisms underlying these interactions. To fully understand social interactions, it may not be sufficient to study single brains in isolation but rather the joint activities of brains of interacting persons². Using different neuroimaging techniques, hyperscanning studies have shown that brain activities of interacting persons or groups synchronize, e.g., while they coordinate their actions³, make music⁴, communicate⁵, engage in classroom activities⁶ or cooperate⁷.

The article presents a protocol for conducting simultaneous recordings with functional near-infrared spectroscopy (fNIRS). Similar to functional magnetic resonance imaging (fMRI), fNIRS measures the hemodynamic response to brain activation. Changes in oxygenated and deoxygenated hemoglobin (oxy-Hb and deoxy-Hb) are calculated based on the amount of diffusively transmitted near-infrared light through tissue⁸. fNIRS is well suited for conducting hyperscanning experiments, especially with children, because it can be applied in less constrained and more natural settings than fMRI. Moreover, it is less prone to movement artifacts than both, fMRI and EEG⁹. In addition, fNIRS data can be acquired at high sampling frequencies (e.g., 10 Hz), thus it highly oversamples the relatively slow hemodynamic response and thereby potentially provides a more complete temporal picture of the brain hemodynamics¹⁰.

This protocol was developed within the study of Reindl et al.¹¹ and has been slightly modified (in particular with respect to the channel placement and bad channel identification) more recently. The aim of the study was to examine synchronized brain activity of parent-child dyads. Using fNIRS hyperscanning, we assessed brain-to-brain synchrony in prefrontal brain areas of children (aged five to nine years) and their parents, mostly mothers, during a cooperative and a competitive computer task. Prefrontal brain regions were targeted as they had been identified as important regions for social interactive processes in previous hyperscanning studies¹. The cooperative and competitive task were originally developed by Cui et al.¹² and recently employed by several previous studies^13,14,15. For the study of Reindl et al.¹¹, the tasks were modified to be suitable for children. Participants were instructed to either respond jointly via button presses in response to a target (cooperation) or to respond faster than the other player (competition). Each child performed each task once with the parent and once with an adult stranger of the same sex as the parent. Within each child-adult dyad, wavelet coherence was calculated for the oxy-Hb signals of corresponding channels as a measure of brain-to-brain synchrony.

This protocol describes the procedures to collect fNIRS hyperscanning data of parent and child during the cooperative and competitive game. The overall procedure, however, is not specific to this research design but is appropriate for different populations (e.g., adult strangers, romantic partners, siblings, etc.) and can be adapted for a number of different experimental tasks. This protocol also outlines one possible analytical procedure, which covers necessary and optional data analysis steps, including fNIRS data preprocessing, bad channel detection, wavelet coherence analysis and validation by random pair analysis.

Protocol

Prior to participation, all parents / children provided informed consent / assent. The study was approved by the ethics committee of the Medical Faculty of RWTH Aachen University.

1. Preparation before the Participant Arrives

1. Prepare NIRS caps.
   1. Choose the cap sizes the same size or slightly larger than the participant’s head circumference.
   2. Cut 15 holes with a diameter of approximately 15 mm each, arranged in a horizontal 3x5 grid, into the forehead area of each of 2 raw EEG caps (see Table of Materials). Make sure that the holes are spaced 30 mm from each other in any direction, that the middle column of holes is located in the center of the forehead, i.e., above the nose, and that the bottom row is located above the eyebrows.
   3. In order to make the caps more comfortable and minimize pressure marks, attach soft foam material (e.g., adhesive window sealing tape or similar flat foam rubber material) at the inner side of the holder grid between the probe sockets and at the edges. Use double-faced adhesive tape or sewing thread if necessary.
   4. Mount an empty 3x5 probe holder grid (see Table of Materials) to each of the modified EEG caps such that the holder grid itself is placed on the inside of the cap and the holder sockets stick in the holes.
      NOTE: The NIRS measurement system (see Table of Materials) has two separate probe sets, use one probe set for each participant.
   5. Gently insert the probes into the appropriate holder sockets on the grids such that only the first ridge of each probe is mounted in the socket, which results in one clicking sound.
   6. Open the probe set monitor window at the NIRS measurement system and select 2 probe sets arranged in a 3x5 grid each, one for the participating child and one for the adult. Ensure that the probe arrangements of the two caps corresponds to the arrangements in the probe set window (i.e., same location of the respective emitter and receiver probe numbers).
2. Prepare the experiment.
   1. Start the NIRS measurement system with laser diodes switched on 30 min. before measuring, such that the system reaches a stable operating temperature.
   2. Set all necessary options at the NIRS measurement system. Make sure that the device is set to event-related measurement and that the RS232 serial input, necessary for receiving triggers from the experimental paradigm, is active.
      Note: The experiment is an adapted version by a paradigm devised by Cui et al.¹², programmed in the non-commercial Psychophysics Toolbox extensions, version 3.0.11¹⁶.
   3. Prepare the experimental paradigm by starting the technical computing software (see Table of Materials) that serves as base for the Psychophysics Toolbox extensions and setting the current directory to the folder that the paradigm is saved in.
   4. Place two chin rests in front of the computer screen to prevent head movements during the experiment.

2. Participant Arrival in the Laboratory

1. Prepare the participants.
   1. Show and explain the experimental setup including the NIRS measurement system to the participants. Always make sure that the participants do not look directly into the laser beam of the NIRS measurement system as this may be harmful to the eye.
   2. Seat the participants next to each other in front of the computer screen. Adjust the height of the chin rests such that both participants sit comfortably.
   3. Instruct the participants and administer practice trials of both the cooperative and the competitive game. Give additional instructions during the practice trials if necessary.
   4. Measure and mark the Fpz point according to the 10-20 system, which is 10% of the distance between nasion and inion, on each participant’s head.
   5. Place the caps with the probes carefully on the participants’ heads, with the laser turned off. Place the front of the cap, including the probe grid, on the participant’s forehead first and then pull down the back of the cap towards the neck. Make sure that the middle probe of the bottom row is placed on Fpz and the middle probe column is aligned along the sagittal reference curve.
   6. Place the fiber strings on the holder arm attached to the NIRS measurement system so that they hang loosely without contact with the participant or chair and that they do not pull on the caps. Use an additional holder (e.g., modified microphone stand or similar) for the second participant if necessary.
   7. Push each probe further into its socket until the small white nose in the center of the top of the probe casing is visible.
      Note: The nose is pushed upwards by a coil spring mechanism as soon as the probe tip touches the participant’s scalp.
   8. Turn the laser on again and test the signal quality by clicking on the Auto Gain button in the probe set monitor window of the NIRS measurement system.
   9. If a channel does not have a sufficient signal (i.e., if it is marked in yellow), gently put the hair underneath the surrounding probe tip aside. If necessary, push the probes further into their sockets but ensure the comfort of the participant. Check whether the signal quality has improved (i.e., the channel is now marked in green) by clicking on the Auto Gain button again.
   10. If step 2.1.9. does not lead to a signal improvement, adjust the signal intensity. If there is too much signal (i.e., if the channel is marked in red), change the signal intensity to low signal intensity by repeatedly clicking on the respective probe’s symbol in the probe set monitor window of the NIRS measurement system. If there is not enough signal (i.e., if the channel is marked in yellow), change the signal intensity to high signal intensity, again by repeatedly clicking on the respective probe’s symbol.
2. Run the experiment
   1. When there are no questions after the practice trials and a good signal quality is ensured, start the experimental paradigm.
   2. Place a towel over the participants’ hands so that they cannot see the hand movements of their respective game partner.
   3. After the experiment, save the data and export the raw light intensity data as a text file by clicking on the Text File Out button. Do not apply any filters in the NIRS measurement system.
   4. Clean all necessary materials (probes, probe holders, chin rests) with ethanol. Wash the caps in a gentle cycle with mild detergent.

3. Data Analysis

1. Data Preprocessing
   Note: There are several non-commercial software packages available for fNIRS data analysis, e.g., HomER¹⁷, NIRS Brain AnalyzIR¹⁸ or SPM for fNIRS¹⁹. The latter was used for the following preprocessing steps. For more information on how to perform these steps, please see the toolbox manual.
   1. Convert the data files to the SPM for fNIRS data format.
   2. Calculate oxy-Hb and deoxy-Hb concentration changes using the modified Beer-Lambert law by pressing the Convert button in the main window. Enter the age of the subject and the distance between source and detector (e.g., 3 cm). Accept the default values for the molar absorption coefficients of oxy-Hb and deoxy-Hb at wavelength (λ) 1 and λ 2 as well as the default values for the differential pathlength factor (DPF) at λ 1 and λ 2.
   3. Preprocess the time series of hemodynamic changes to reduce motion artifacts by selecting the MARA button (for more information on the MARA algorithm see Scholkman et al.²⁰).
   4. Preprocess the time series to reduce slow drifts by selecting the DCT button.
2. Bad channel detection
   Note: Bad channel detection can be performed before and / or after fNIRS data preprocessing. In this protocol, different objective criteria for detecting bad channels and visual inspection are combined. Please note that the proposed list of objective criteria is not exhaustive. For bad channel detection, self-written scripts were used (for the technical computing software see Table of Materials).
   1. Exclude channels in which there is no signal change for several continuous samples, which is indicated by a flat line when plotting the time series.
   2. Calculate the coefficient of variation CV = SD/mean*100 for the raw attenuation data. Exclude channels in which the CV is above a predefined percentage (e.g., 10%; see for instance van der Kant et al.²¹).
   3. Plot the power spectrum of the signal. If there is no heartbeat visible in the signal spectrum around 1 Hz, as indicated by an increased power in this frequency band, exclude the channel from the analysis.
   4. Visually inspect all data before and/or after preprocessing. Decide whether to include the channel based on the objective criteria, described in 3.2.1 – 3.2.3, as well as on subjective visual detection of noisy channels.
3. Brain-to-brain connectivity
   Note: Two different estimate types of brain connectivity can be distinguished: non-directed estimates, which quantify the strength of the connectivity, and directed estimates, which seek to establish statistical evidence for causation from the data²². Here the focus was on the wavelet transform coherence (WTC), a widely applied non-directed estimate for fNIRS brain-to-brain connectivity. Several non-commercial software solutions for the computation of the WTC are available, e.g., one by Grinsted and colleagues²³ or the ASToolbox²⁴, which was used in this protocol for the following steps.
   1. In the AWCO function of the ASToolbox, specify the mother wavelet (e.g., Generalized Morse Wavelet with its parameters beta and gamma), which is used to transform each time series into the time and frequency domain by the continuous wavelet transformation.
   2. Specify the smoothing window type (e.g., Hanning window) and the smoothing window size for the time and scale domain in the AWCO function.
   3. To examine the significance of the WTC coefficients and to calculate their p-values, specify the number of surrogate time series (n ≥ 300) and the ARMA model (e.g., AR (1)) in the AWCO function.
   4. With the parameters specified in steps 3.3.1 to 3.3.3, calculate the wavelet coherence of two corresponding channels (the same channel in two participants).
   5. Choose a frequency band of interest in which the task-related brain-to-brain synchrony is expected to occur based on previous studies and visual inspection of the data (for an alternative approach see Nozawa et al. ²⁵).
   6. Calculate the mean of the WTC coefficients and / or the percentage of significant WTC coefficients in the task-related frequency band for each task block in each channel and for each dyad. Use this value as an outcome measure of brain-to-brain synchrony for further statistical analysis (for more information see Reindl et al.¹¹).
4. Comparison to Random Pairs
   Note: To validate the results, we recommend comparing the WTC of the actual dyads to the WTC of random adult-child pairings, who did not play with each other but performed the same experimental task.
   1. Calculate the WTC, as described in 3.3., for participant pairs who did not play together but performed the same experimental task (i.e., random pairs). Choose the number of random pairs (e.g., 300 for each condition) and calculate the WTC for each random pair.
   2. Compare the coherence of the random and actual pairs to avoid the detection of spurious synchronicity.

Results

Representative data of one parent-child dyad during the cooperative condition are shown in Figure 1. The cooperative task consists of three 30 s rest blocks and two task blocks, with 20 trials each, presented in alternating order. In each trial, participants have to react as simultaneously as possible to a signal to earn a point¹¹.

Discussion

In this protocol, we show how to conduct fNIRS hyperscanning experiments and one possible way to analyze brain-to-brain synchrony, measuring concentration changes of oxy-Hb and deoxy-Hb at frontal brain regions of two subjects simultaneously. FNIRS hyperscanning is relatively easy to apply: a single NIRS device is sufficient to measure brain activities of both subjects by splitting the optodes between them. Thus, no synchronization between different devices is necessary¹. Moreover, since fNIRS doe...

Materials

Name	Company	Catalog Number	Comments
NIRS measurement system with probe sets and probe holder grids	Hitachi Medical Corporation, Tokyo, Japan	ETG-4000 Optical Topography System	The current study protocol requires an optional second adult probe set for 52 channels of measurement in total as well as two 3x5 probe holder grids.
raw EEG caps	EASYCAP GmbH, Herrsching, Germany	C-SCMS-56; C-SCMS-58	Caps must be provided with holes for NIRS probes by the experimenter. Choose cap size the same size or slightly larger than participant's head circumference.
Technical computing software	The MathWorks, Inc., Natick, MA	MATLAB R2014a (or later versions)	Serves as base for Psychophysics Toolbox extensions (stimulus presentation), SPM for fNIRS toolbox  (fNIRS data analysis), and ASToolbox (WTC computation).