Name: measurement of the directional information flow in fnirs-hyperscanning data using the partial wavelet transform coherence method
Uploaded: 2021-09-03T14:30:00
Description: Watch this Scientific Journal Video about Measurement of the Directional Information Flow in fNIRS-Hyperscanning Data using the Partial Wavelet Transform Coherence Method at JoVE.com

This work was supported by the National Natural Science Foundation of China (61977008) and the Young Top Notch Talents of Ten Thousand Talent Program.

The human experiment protocol was approved by the Institutional Review Board and Ethics Committee of the State Key Laboratory of Cognitive Neuroscience and Learning at Beijing Normal University. All participants gave written informed consent before the experiment began.
1. The simulation experiment
<ol>
	<li>Generate two time series of signals that correlate with each other, with one signal having autocorrelation at a 4 s time lag. Set the correlation coefficient of r between the two signals...

In hyperscanning studies, it is usually essential to describe the directional and temporal patterns of information flow between individuals. Most previous fNIRS hyperscanning studies have used traditional WTC25 to infer these characteristics by calculating the time-lagged INS. However, as one of the intrinsic features of the fNIRS signal20,21, the autocorrelation effect might confound the time-lagged INS. To address this issue, in the prot...

Social interaction is of vital importance for human beings1,2. For understanding the dual-brain neurocognitive mechanism of social interaction, the hyperscanning approach has recently been extensively used, showing that the patterns of interpersonal neural synchronization (INS) can well characterize the social interaction process3,4,5,6,7,8,9,10,11,12,13,14. Among recent studies, an interesting finding is that the role difference of individuals in a dyad may lead to a time-lagged pattern of INS, i.e., INS occurs when the brain activity of one individual lags behind that of another individual by seconds, such as that from listeners to speakers5,9, from leaders to followers4, from teachers to students8, from mothers to children13,15, and from women to men in a romantic couple6. Most importantly, there is a good correspondence between the interval of the time-lagged INS and that of social interaction behaviors, such as between teachers questioning and students answering8 or between parenting behaviors of mothers and compliance behaviors of children15. Thus, time-lagged INS may reflect a directional information flow from one individual to another, as proposed in a recent hierarchical model for interpersonal verbal communication16.
Previously, the time-lagged INS was mainly calculated on the functional near-infrared spectroscopy (fNIRS) signal because of its relatively high spatial resolution, sound anatomical localization, and exceptionally high tolerance of motion artifacts17 when studying naturalistic social interactions. Moreover, to precisely characterize the correspondence between the neural time lag and the behavioral time lag during social interaction, it is essential to obtain the INS strength for each time lag (e.g., from no time lag to a time lag of 10 s). For this purpose, previously, the wavelet transform coherence (WTC) procedure was extensively applied after shifting the brain signal of one individual forward or backward relative to that of another individual5,6,18. When using this traditional WTC procedure for fNIRS signals, there is a potential challenge because the observed time-lagged INS may be confounded by the autocorrelation effect of the fNIRS signal for an individual19,20,21. For example, during a dyadic social interaction process, the signal of participant A at time point t may be synchronized with that of participant B at the same time point. Meanwhile, the signal of participant A at time point t may be synchronized with that of participant A at a later time point t+1 because of the autocorrelation effect. Therefore, a spurious time-lagged INS may occur between the signal of participant A at time point t and that of participant B at time point t+1.
Mihanovi&#263; and his colleagues22 first introduced a method termed partial wavelet transform coherence (pWTC), and then applied it in marine science23,24. The original purpose of this method was to control the exogenous confounding noise when estimating the coherence of two signals. Here, to address the autocorrelation issue in the fNIRS hyperscanning data, the pWTC method was extended to calculate time-lagged INS on the fNIRS signal. Precisely, a time-lagged INS (and a directional information flow) from participant A to participant B can be calculated using the equation below (Equation 1)23.
<img alt="Equation 1" src="/files/ftp_upload/62927/62927eq01.jpg" />
Here, it is assumed that there are two signals, A and B, from participants A and B, respectively. The occurrence of signal B always precedes that of signal A with a time lag of n, where WTC (At, Bt+n) is the traditional time-lagged WTC. WTC (At, At+n) is the autocorrelated WTC in participant A. WTC (At, Bt) is the time-aligned WTC at time point t between participant A and B. * is the complex conjugate operator (Figure 1A).
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/62927/62927fig01.jpg" /> 
Figure 1: Overview of pWTC. (A) The logic of the pWTC. There are two signals A and B, within a dyad. The occurrence of A always follows that of B with a lag n. A gray box is a wavelet window at a certain time point t or t+n. Based on the pWTC equation (represented in the figure), three WTCs need to be calculated: the time-lagged WTC of At+n and Bt; the autocorrelated WTC in participant A of At and At+n; and the time-aligned WTC at timepoint t, At and Bt. (B)The layout of optode probe sets. CH11 was placed at T3, and CH25 was placed at T4 following the international 10-20 system27,28. <a href="https://www.jove.com/files/ftp_upload/62927/62927fig01large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
This protocol first introduced a simulation experiment to demonstrate how well the pWTC resolves the autocorrelation challenge. Then, it explained how to conduct pWTC in a step-by-step way based on an empirical experiment of naturalistic social interactions. Here, a communication context was used to introduce the method. This is because, previously, the time-lagged INS was usually calculated in a naturalistic communication context3,4,6,8,13,15,18. Additionally, a comparison between the pWTC and the traditional WTC and validation with the Granger causality (GC) test were also conducted.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>fNIRS topography system</td>
			<td>Shimadzu Corporation</td>
			<td>Shimadzu LABNIRS systen</td>
			<td>LABNIRS system contains 40 emitters and 40 detectors for fNIRS signals measurement. In this protocol we used these emitters and detectors created two customized 26-channels probe sets and attached to two caps accroding to 10-20 system. Further, LABNIRS system also contains built-in GUI softwares for data quality check, data convert and data export.</td>
		</tr>
		<tr>
			<td>MATLAB</td>
			<td>The MathWorks, Inc.</td>
			<td>MATLAB 2019a</td>
			<td>In this protocol, several toolboxs and functions bulit in MATLAB were used: 
			SPM12 toolbox was used to normalize the valided MRI data through its GUI. 
			NIRS_SPM toolbox was used to project the MNI coordinates of the probes to the AAL template through its GUI. 
			Homer3 toolbox was used to remove motion artifacts through its function hmrMotionCorrectWavelet with default parameters. 
			Wavelet toolbox was used to compute WTC and pWTC through its function wcoherence.</td>
		</tr>
		<tr>
			<td>MRI scanner</td>
			<td>Siemens Healthineers</td>
			<td>TRIO 3-Tesla scanner</td>
			<td>In this protocol, the MRI scanner was used to obtain MNI coordinates of each channel and optpde. Scan parameters are described in main text.</td>
		</tr>
		<tr>
			<td>customized caps</td>
			<td></td>
			<td></td>
			<td>In this protocol, we first marked two nylon caps with 10-20 system. Then, we made two 26-channels customized optode probes sets. Finally, we attached probes sets to caps aligned with landmarks.</td>
		</tr>
	</tbody>
</table>

measurement of the directional information flow in fnirs-hyperscanning data using the partial wavelet transform coherence method

Social interaction is of vital importance for human beings. While the hyperscanning approach has been extensively used to study interpersonal neural synchronization (INS) during social interactions, functional near-infrared spectroscopy (fNIRS) is one of the most popular techniques for hyperscanning naturalistic social interactions because of its relatively high spatial resolution, sound anatomical localization, and exceptionally high tolerance of motion artifacts. Previous fNIRS-based hyperscanning studies usually calculate a time-lagged INS using wavelet transform coherence (WTC) to describe the direction and temporal pattern of information flow between individuals. However, the results of this method might be confounded by the autocorrelation effect of the fNIRS signal of each individual. For addressing this issue, a method termed partial wavelet transform coherence (pWTC) was introduced, which aimed to remove the autocorrelation effect and maintain the high temporal-spectrum resolution of the fNIRS signal. In this study, a simulation experiment was performed first to show the effectiveness of the pWTC in removing the impact of autocorrelation on INS. Then, step-by-step guidance was offered on the operation of the pWTC based on the fNIRS dataset from a social interaction experiment. Additionally, a comparison between the pWTC method and the traditional WTC method and that between the pWTC method and the Granger causality (GC) method was drawn. The results showed that pWTC could be used to determine the INS difference between different experimental conditions and INS's directional and temporal pattern between individuals during naturalistic social interactions. Moreover, it provides better temporal and frequency resolution than the traditional WTC and better flexibility than the GC method. Thus, pWTC is a strong candidate for inferring the direction and temporal pattern of information flow between individuals during naturalistic social interactions.

Simulation results 
The results showed that the time-lagged INSWTC with autocorrelation was significantly higher than the time-lagged INSWTC without autocorrelation (t(1998) = 4.696, p &#60; 0.001) and time-lagged INSpWTC (t(1998) = 5.098, p &#60; 0.001). Additionally, there was no significant difference between time-lagged INSWTC without autocorrelation and INSpWTC (t(1998) = 1.573, p = 0.114, <stron...

Watch this Scientific Journal Video about Measurement of the Directional Information Flow in fNIRS-Hyperscanning Data using the Partial Wavelet Transform Coherence Method at JoVE.com

Measurement of the Directional Information Flow in fNIRS-Hyperscanning Data using the Partial Wavelet Transform Coherence Method

This protocol describes partial wavelet transform coherence (pWTC) for calculating the time-lagged pattern of interpersonal neural synchronization (INS) to infer the direction and temporal pattern of information flow during social interaction. The effectiveness of pWTC in removing the confounds of signal autocorrelation on INS was proved by two experiments.

Social interaction is of vital importance for human beings. While the hyperscanning approach has been extensively used to study interpersonal neural ...

Hyperscanning is a promising method to describe the relationship between individuals and that in this study finds neural synchronization between individuals. And the most important question is to identify who is synchronizing with who. And the method we propose here is to identify the direction of the information flow between individuals.Compared to other methods, the most significant advantage of this method is to remove the various confounding factors from the neural synchronization. For example, the autocorrelation in every signal. And meanwhile, we can maintain the high spatial anatomical resolution of every signal.Start the data pre-processing by exporting the hemoglobin concentration changes data files from the equipment and transmitting the files to MATLAB readable format. Then, remove the first and last 15 seconds of the data for each session to avoid transient responses and use the MATLAB decimate built-in function to downsample the data from 55.6 hertz to 11.1 hertz. To correct motion artifacts, apply the discrete wavelet transform filter method with the appropriate filtering function.Use the PCA built-in function to remove global physiological noise. Then, remove the top 80%of the variance from the signals. After removing the initial two seconds of data from women and the last two seconds of data from men, calculate the women-led two second-lagged INS WTC value with the equation.Similarly, after removing the initial two seconds of data from men and the last two seconds of data from women, calculate the men-led two second-lagged INS WTC value. Repeat the procedure with different time lags, N, like N equals to four or six or eight seconds, across the potential 676 CH pairs and calculate the strength of women and men-led time-lagged INS WTC with the equation as described before. Calculate INS pWTC using pWTC in the same way using the equation.Generate the time-lagged time series of the fNIRS signals between men and women and calculate the values of the time-lagged WTC at different time lags. Generate autocorrelated time series of men's fNIRS signals by removing the first two seconds and the last two seconds of the data from the men. Then, calculate the two seconds autocorrelated value for men.Then, evaluate the autocorrelated WTC values at different time lags. In the same way, create a time-aligned time series of the fNIRS signals by removing the first two seconds of data from the men and the women. Then, calculate the two seconds time-aligned WTC.Then, evaluate the time-aligned WTC values at different time lags. Enter the time-aligned WTC, time-lagged WTC, and autocorrelated WTC values at different time lag into the equations of pWTC, generating INS pWTC. In the second-level fNIRS data processing, remove physiological noise and frequency bands of each signal above 0.7 hertz.Then, remove frequency bands of each signal below 0.01 hertz and within 0.15 to 0.3 hertz to filter out very low frequency fluctuations. Transform INS with Fisher Z-transformation and then average INS at the temporal dimension. For the averaged INS at each time lag, conduct a paired two sample t-test on each CH pair across the frequency range.Then, identify all significant frequency clusters. Reassign dyadic relationships by randomly assigning the participants to new, two member pairs, like the participants of a diad that had never communicated with one another. Conduct a cluster based permutation test to establish a threshold for the results.Recalculate the INS in each time lag. Perform paired t-tests in the new sample and identify significant frequency clusters. Then, select the cluster with the largest summed T value before repeating the procedures 1, 000 times to generate a null distribution of the maximum false positive T values.Compare the summed T value of each identified frequency cluster in the original sample with the null distribution to obtain significant statistical results. A simulation analysis was conducted in the study. The representative analysis illustrates that the time-lagged INS WTC with autocorrelation was significantly higher than the time-lagged INS WTC without autocorrelation and time-lagged INS pWTC.Additionally, there was no significant difference between the time-lagged INS WTC without autocorrelation and INS pWTC, indicating the efficiency of pWTC in removing the impact of the autocorrelation effect on INS. A marginal significant context effect was observed within 0.4 to 0.6 hertz, which of men lagged that of women by four seconds. In contrast, for INS pWTC, only significant context effect within 0.4 to 0.6 hertz when the sensor and motor cortex activity of men lagged that of women by four seconds was observed.The directional INS from women to men was significantly higher in the conflict contexts than in the supportive contexts. The results of INS pWTC were validated with the Granger causality method. The results showed that INS GC exhibited a similar pattern to INS pWTC.The directional INS calculated by the Granger causality analysis from women to men was significantly higher in the conflict contexts than in the supportive contexts. After calculating the neural synchronization using this method, it's also possible to conduct a lot of other methods. For example, we can compare this directional information flow between different social connection context and the social connection in different social relationship.And also, it is possible to understand the contribution of different communication behaviors to the neural synchronization. For example, whether it is verbal communication or nonverbal communication play more a important role in the neural synchronization.

Research

JoVE Journal

Neuroscience

Using Informational Connectivity to Measure the Synchronous Emergence of fMRI Multi-voxel Information Across Time

It is now appreciated that condition-relevant information can be present within distributed patterns of functional magnetic resonance imaging (fMRI) brain ...

Quadruple Immunostaining of the Olfactory Bulb for Visualization of Olfactory Sensory Axon Molecular Identity Codes

The mouse olfactory system is often used to study mechanisms of neural circuit formation&#12288;because of its simple anatomical structure. An Olfactory ...

Ocular Kinematics Measured by In Vitro Stimulation of the Cranial Nerves in the Turtle

Ocular Kinematics Measured by In Vitro Stimulation of the Cranial Nerves in the Turtle

After animals are euthanized, their tissues begin to die. Turtles offer an advantage because of a longer survival time of their tissues, especially when ...

Simultaneous Recordings of Cortical Local Field Potentials, Electrocardiogram, Electromyogram, and Breathing Rhythm from a Freely Moving Rat

Monitoring the physiological dynamics of the brain and peripheral tissues is necessary for addressing a number of questions about how the brain controls ...

Retinal Vascular Reactivity as Assessed by Optical Coherence Tomography Angiography

The vascular supply to the retina has been shown to dynamically adapt through vasoconstriction and vasodilation to accommodate the metabolic demands of ...

Reliable Acquisition of Electroencephalography Data during Simultaneous Electroencephalography and Functional MRI

Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI), EEG-fMRI, combines the complementary properties of scalp EEG ...

Inter-Brain Synchrony in Open-Ended Collaborative Learning: An fNIRS-Hyperscanning Study

fNIRS hyperscanning is widely used to detect the neurobiological underpinnings of social interaction. With this technique, researchers qualify the ...

How to Calculate and Validate Inter-brain Synchronization in a fNIRS Hyperscanning Study

The dynamics between coupled brains of individuals have been increasingly represented by inter-brain synchronization (IBS) when they coordinate with each ...

Stimulating Mouse Brain with a Low-Cost EEG and Millimeter-Sized Coil

Low-Cost Electroencephalographic Recording System Combined with a Millimeter-Sized Coil to Transcranially Stimulate the Mouse Brain In Vivo

Author Spotlight: Low-Cost Electroencephalographic Recording System Combined with a Millimeter-Sized Coil to Transcranially Stimulate the Mouse Brain In Vivo  

A low-cost electroencephalographic (EEG) recording system is proposed here to drive transcranial magnetic stimulation (TMS) of the mouse brain in vivo, ...

Mouse Retina OCT Fibergram Alignment with Confocal Images

Alignment of Visible-Light Optical Coherence Tomography Fibergrams with Confocal Images of the Same Mouse Retina

Author Spotlight: Advancements in In Vivo and Ex Vivo Retinal Imaging for Improved Glaucoma Diagnosis and Treatment 

In recent years, in vivo retinal imaging, which provides non-invasive, real-time, and longitudinal information about biological systems and processes, has ...