Name: Author Spotlight: Capturing Infant-Caregiver Interactions Through Synchronized Multimodal Data Collection
Uploaded: 2024-05-31T14:00:00
Description: Prior hyperscanning studies that record the brain activities of caregivers and children concurrently have primarily been conducted within the confines of ...

The work was funded by a Presidential Postdoctoral Fellowship Grant from Nanyang Technological University that was awarded to VR.

The protocol described has been approved by the Institutional Review Board (IRB) of the Nanyang Technological University, Singapore. Informed consent was obtained from all adult participants and from parents on behalf of their infants.
1. Considerations of equipment and space at home sessions
<ol>
	<li>Prepare for differing humidity and temperature conditions depending on country and season. For environments with high temperature and humidity levels, ensure that there is ade...

Real-Life EEG and ECG Synchronization in Infant-Caregiver Interactions

<ol>
	<li>Hari, R., Henriksson, L., Malinen, S., Parkkonen, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Centrality+of+social+interaction+in+human+brain+function.">Centrality of social interaction in human brain function.</a> Neuron. 88 (1), 181-193 (2015).</li><li>Endevelt-Shapira, Y., Djalovski, A., Dumas, G., Feldman, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Maternal+chemosignals+enhance+infant-adult+brain-to-brain+synchrony.">Maternal chemosignals enhance infant-adult brain-to-brain synchrony.</a> Sci Adv. 7 (50), (2021).</li><li>Santamaria, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Emotional+valence+modulates+the+topology+of+the+parent-infant+inter-brain+network.">Emotional valence modulates the topology of the parent-infant inter-brain network.</a> NeuroImage. 207, 116341 (2020).</li><li>Nguyen, T., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+effects+of+interaction+quality+on+neural+synchrony+during+mother-child+problem+solving.">The effects of interaction quality on neural synchrony during mother-child problem solving.</a> Cortex. 124, 235-249 (2020).</li><li>Reindl, V., Gerloff, C., Scharke, W., Konrad, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Brain-to-brain+synchrony+in+parent-child+dyads+and+the+relationship+with+emotion+regulation+revealed+by+fNIRS-based+hyperscanning.">Brain-to-brain synchrony in parent-child dyads and the relationship with emotion regulation revealed by fNIRS-based hyperscanning.</a> NeuroImage. 178, 493-502 (2018).</li><li>Reindl, V., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Conducting+hyperscanning+experiments+with+functional+near-infrared+spectroscopy.">Conducting hyperscanning experiments with functional near-infrared spectroscopy.</a> J Vis Exp. (143), e58807 (2019).</li><li>Gardner, F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Methodological+issues+in+the+direct+observation+of+parent-child+interaction:+Do+observational+findings+reflect+the+natural+behavior+of+participants.">Methodological issues in the direct observation of parent-child interaction: Do observational findings reflect the natural behavior of participants.</a> Clin Child Fam Psychol Rev. 3, 185-198 (2000).</li><li>Xu, J., Zhong, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Review+on+portable+EEG+technology+in+educational+research.">Review on portable EEG technology in educational research.</a> Comput Hum Behav. 81, 340-349 (2018).</li><li>Troller-Renfree, S. V., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Feasibility+of+assessing+brain+activity+using+mobile,+in-home+collection+of+electroencephalography:+methods+and+analysis.">Feasibility of assessing brain activity using mobile, in-home collection of electroencephalography: methods and analysis.</a> Dev Psychobiol. 63 (6), e22128 (2021).</li><li>B&#246;gels, S., Levinson, S. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+brain+behind+the+response:+Insights+into+turn-taking+in+conversation+from+neuroimaging.">The brain behind the response: Insights into turn-taking in conversation from neuroimaging.</a> Res Lang Soc. 50 (1), 71-89 (2017).</li><li>Georgieva, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Toward+the+understanding+of+topographical+and+spectral+signatures+of+infant+movement+artifacts+in+naturalistic+EEG.">Toward the understanding of topographical and spectral signatures of infant movement artifacts in naturalistic EEG.</a> Front Neurosci. 14, 452947 (2020).</li><li>Dikker, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Brain-to-brain+synchrony+tracks+real-world+dynamic+group+interactions+in+the+classroom.">Brain-to-brain synchrony tracks real-world dynamic group interactions in the classroom.</a> Curr Biol. 27 (9), 1375-1380 (2017).</li><li>Oliveira, A. S., Bryan, R. S., Hairston, W. D., Peter, K., Daniel, P. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Proposing+metrics+for+benchmarking+novel+EEG+technologies+towards+real-world+measurements.">Proposing metrics for benchmarking novel EEG technologies towards real-world measurements.</a> Front Hum Neurosci. 10, 188 (2016).</li><li>Turk, E., Endevelt-Shapira, Y., Feldman, R., vanden Heuvel, M. I., Levy, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Brains+in+sync:+Practical+guideline+for+parent-infant+EEG+during+natural+interaction.">Brains in sync: Practical guideline for parent-infant EEG during natural interaction.</a> Front Psychol. 13, 833112 (2022).</li><li>Kayhan, E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+dual+EEG+pipeline+for+developmental+hyperscanning+studies.">A dual EEG pipeline for developmental hyperscanning studies.</a> Dev Cogn Neurosci. 54, 101104 (2022).</li><li>Ayrolles, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=HyPyP:+a+Hyperscanning+Python+pipeline+for+inter-brain+connectivity+analysis.">HyPyP: a Hyperscanning Python pipeline for inter-brain connectivity analysis.</a> Soc Cogn Affect Neurosci. 16 (1-2), 72-83 (2021).</li><li>Delorme, S., Makeig, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=EEGLAB:+an+open-source+toolbox+for+analysis+of+single-trial+EEG+dynamics.">EEGLAB: an open-source toolbox for analysis of single-trial EEG dynamics.</a> J Neurosci Meth. 134, 9-21 (2004).</li><li>Nathan, K., Contreras-Vidal, J. L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Negligible+motion+artifacts+in+scalp+electroencephalography+(EEG)+during+treadmill+walking.">Negligible motion artifacts in scalp electroencephalography (EEG) during treadmill walking.</a> Front Hum Neurosci. 9, 708 (2016).</li><li>Stone, D. B., Tamburro, G., Fiedler, P., Haueisen, J., Comani, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Automatic+removal+of+physiological+artifacts+in+EEG:+The+optimized+fingerprint+method+for+sports+science+applications.">Automatic removal of physiological artifacts in EEG: The optimized fingerprint method for sports science applications.</a> Front Hum Neurosci. 12, 96 (2018).</li><li>Noreika, V., Georgieva, S., Wass, S., Leong, V. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=14+challenges+and+their+solutions+for+conducting+social+neuroscience+and+longitudinal+EEG+research+with+infants.">14 challenges and their solutions for conducting social neuroscience and longitudinal EEG research with infants.</a> Infant Behav Dev. 58, 101393 (2020).</li><li>Ng, B., Reh, R. K., Mostafavi, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+practical+guide+to+applying+machine+learning+to+infant+EEG+data.">A practical guide to applying machine learning to infant EEG data.</a> Dev Cogn Neurosci. 54, 101096 (2022).</li><li>vander Velde, B., Junge, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Limiting+data+loss+in+infant+EEG:+putting+hunches+to+the+test.">Limiting data loss in infant EEG: putting hunches to the test.</a> Dev Cogn Neurosci. 45, 100809 (2020).</li><li>Bell, M. A., Cuevas, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Using+EEG+to+study+cognitive+development:+Issues+and+practices.">Using EEG to study cognitive development: Issues and practices.</a> J Cogn Dev. 13 (3), 281-294 (2012).</li><li>Lopez, K. L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=HAPPILEE:+HAPPE+in+low+electrode+electroencephalography,+a+standardized+pre-processing+software+for+lower+density+recordings.">HAPPILEE: HAPPE in low electrode electroencephalography, a standardized pre-processing software for lower density recordings.</a> NeuroImage. 260, 119390 (2022).</li></ol>

In this protocol, we conduct measurements in the participants&#39; homes where infants and caregivers may feel more comfortable and their behaviors may be more representative of their real-life interactions as opposed to a laboratory setting, thus, increasing ecological validity7. Further, recordings in the home environment may ease the burden on&#160;the participants, e.g., with respect to travel times, and may thus make certain participant groups more accessible. However, along with these advant...

With the simultaneous recording of brain activities from two or more interacting subjects, also known as hyperscanning, it has become possible to elucidate the neural basis of social interactions in their complex, bidirectional, and fast-paced dynamics1. This technique has shifted the focus from studying individuals in isolated, tightly controlled settings to examining more naturalistic interactions, such as parent-child interactions during free play2,3, puzzle-solving4, and cooperative computer games5,6. These studies demonstrate that brain activities synchronize during social interactions, i.e., show temporal similarities, a phenomenon termed interpersonal neural synchrony (INS). However, the great majority of hyperscanning studies have been confined to laboratory settings. While this allows for better experimental control, it may come at the expense of losing some ecological validity. Behaviors observed in the laboratory may not be representative of the participants&#39; typical everyday interactive behaviors due to the unfamiliar and artificial setting and the nature of the tasks imposed7.
Recent advances in mobile neuroimaging devices, such as electroencephalography (EEG) or functional near-infrared spectroscopy (fNIRS), alleviate these issues by removing the requirement for participants to remain physically connected to the recording computer. Thus, they allow us to measure participants&#39; brain activities while they interact freely in the classroom or in their homes8,9. The advantage of EEG compared to other neuroimaging techniques, such as fNIRS, is that it has an excellent temporal resolution, which makes it particularly suitable for investigating fast-paced social dynamics10. Yet, it comes with the caveat that the EEG signal is highly vulnerable to motion and other physiological and non-physiological artifacts11.
Despite this, the first studies have successfully implemented EEG hyperscanning set-ups in realistic environments and conditions. For instance, Dikker et al.12 measured the EEG signal of a group of students while they engaged in various classroom activities, including attending lectures, watching videos, and participating in group discussions. This study, along with other studies8,9, has predominantly utilized dry EEG electrodes to ease the process of conducting measurements in non-laboratory settings. Compared to wet electrodes, which require the application of conductive gel or paste, dry electrodes offer notable advantages in terms of usability. They have been shown to exhibit comparable performance to wet electrodes in adult populations and stationary conditions; however, their performance may decrease in motion-related scenarios due to increased impedance levels13.
Here, we present a working protocol to capture synchronized recordings from a low-density seven-channel liquid gel EEG system with a single lead electrocardiography (ECG) connected to the same wireless amplifier (sampling rate: 500 Hz)&#160;of infant-caregiver dyads in a home setting. While active electrodes were used for adults, passive electrodes were used instead for infants since the latter typically comes in the form of ring electrodes, thereby easing the process of gel application. Additionally, EEG-ECG recordings were synchronized to three cameras and microphones to capture the participants&#39; behaviors from different angles. In the study, 8-12-month-old infants and their caregivers engaged in a reading and play task while their EEG, ECG, and behaviors were recorded. To minimize the impact of excessive movement on EEG signal quality, the tasks were conducted in a table-top setting (e.g., utilizing the kitchen table and an infant highchair), requiring participants to remain seated throughout the interaction task. Caregivers were provided with three age-appropriate books and table-top toys (equipped with suction cups to prevent them from falling). They were instructed to read to their child for approximately 5 min, followed by a 10 min play session with the toys.
This protocol details the methods for collecting synchronized EEG-ECG, video, and audio data during the reading and play tasks. The overall procedure, however, is not specific to this research design but is appropriate for different populations (e.g., parent-child dyads, friend dyads) and experimental tasks. The method of synchronization of different data streams will be presented. Further, a basic EEG preprocessing pipeline based on Dikker et al.12 will be outlined, and EEG data retention rates and quality control metrics will be reported. Since the specific analytical choices depend on a variety of factors (such as task design, research questions, EEG montage), hyperscanning-EEG analysis will not be detailed further, but instead, the reader will be referred to existing guidelines and toolboxes (e.g.,14 for guidelines;15,16 for hyperscanning analysis toolboxes). Finally, the protocol discusses challenges and potential solutions for EEG-ECG hyperscanning in the home and other real-world settings.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>10 cc Luer Lock Tip syringe without Needle</td>
			<td>Terumo Corporation</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>actiCAP slim 8-channel electrode set (LiveAMP8)</td>
			<td>Brain Products GmbH</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Arduino Software (IDE)</td>
			<td>Arduino</td>
			<td>Arduino IDE 1.8.19</td>
			<td>The software used to write the code for the Arduino microcontroller. Alternate programming software may be used to accompany the chosen microcontroller unit.&#160;</td>
		</tr>
		<tr>
			<td>Arduino Uno board</td>
			<td>Arduino</td>
			<td></td>
			<td>Used for building the circuit of the trigger box. Alternate microcontroller boards may be used.</td>
		</tr>
		<tr>
			<td>BNC connectors</td>
			<td></td>
			<td></td>
			<td>BNC connectors to connect the various parts of the trigger box setup.</td>
		</tr>
		<tr>
			<td>BNC Push button&#160;</td>
			<td>Brain Products GmbH</td>
			<td>BP-345-9000</td>
			<td>BNC trigger push button to send triggers.</td>
		</tr>
		<tr>
			<td>BNC to 2.5 mm jack trigger cable (80 cm)&#160;</td>
			<td>Brain Products GmbH</td>
			<td>BP-245-1200</td>
			<td>BNC cables connecting the 2 LiveAmps to the trigger box.</td>
		</tr>
		<tr>
			<td>BrainVision Analyzer Version 2.2.0.7383</td>
			<td>Brain Products GmbH</td>
			<td></td>
			<td>EEG analysis software.</td>
		</tr>
		<tr>
			<td>BrainVision Recorder License with dongle</td>
			<td>Brain Products GmbH</td>
			<td>S-BP-170-3000</td>
			<td></td>
		</tr>
		<tr>
			<td>BrainVision Recorder Version 1.23.0003</td>
			<td>Brain Products GmbH</td>
			<td></td>
			<td>EEG recording software.</td>
		</tr>
		<tr>
			<td>Custom 8Ch LiveAmp Cap passive (infant EEG caps)</td>
			<td>Brain Products GmbH</td>
			<td>LC-X6-SAHS-44, LC-X6-SAHS-46, LC-X6-SAHS-48&#160;</td>
			<td>For infant head sizes 44, 46, 48 . Alternate EEG caps may be used.</td>
		</tr>
		<tr>
			<td>Dell Latitude 3520 Laptops</td>
			<td>Dell</td>
			<td></td>
			<td>Two laptops, one for adult EEG recording and one for infant EEG recording. Alternate computers may be used.</td>
		</tr>
		<tr>
			<td>Dental Irrigation Syringes</td>
			<td></td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>LiveAmp 8-CH wireless amplifier</td>
			<td>BrainProducts GmbH</td>
			<td>BP-200-3020</td>
			<td>Two LiveAmps, one for adult EEG and one for infant EEG. Alternate amplifier may be used.</td>
		</tr>
		<tr>
			<td>Manfrotto MT190X3 Tripod with 128RC Micro Fluid Video Head</td>
			<td>Manfrotto</td>
			<td>MT190X3</td>
			<td>Alternate tripods may be used.</td>
		</tr>
		<tr>
			<td>Matlab Software</td>
			<td>The MathWorks, Inc.</td>
			<td>R2023a</td>
			<td>Alternate analysis and presentation software may be used.</td>
		</tr>
		<tr>
			<td>Power bank (10000 mAh)</td>
			<td>Philips</td>
			<td>DLP6715NB/69</td>
			<td>Alternate power banks may be used.</td>
		</tr>
		<tr>
			<td>Raw EEG caps</td>
			<td>EASYCAP GmbH</td>
			<td></td>
			<td>For Adult head sizes 52, 54, 56, 58. Alternate EEG caps may be used.</td>
		</tr>
		<tr>
			<td>Rode Wireless Go II Single Set</td>
			<td>R&#248;de Microphones</td>
			<td></td>
			<td>Alternate microphones may be used.</td>
		</tr>
		<tr>
			<td>Sony FDR-AX700 Camcorder</td>
			<td>Sony</td>
			<td>FDR-AX700</td>
			<td>Alternate camcorders or webcams may be used.</td>
		</tr>
		<tr>
			<td>SuperVisc High-Viscosity Gel&#160;</td>
			<td>EASYCAP GmbH</td>
			<td>NS-7907</td>
			<td></td>
		</tr>
	</tbody>
</table>

home-based eeg hyperscanning for infant-caregiver social interactions

Prior hyperscanning studies that record the brain activities of caregivers and children concurrently have primarily been conducted within the confines of the laboratory, thus limiting the generalizability of results to real-life settings. Here, a comprehensive protocol for capturing synchronized electroencephalography (EEG), electrocardiography (ECG), and behavioral recordings from infant-caregiver dyads during various interactive tasks at home is proposed. This protocol demonstrates how to synchronize the different data streams and report EEG data retention rates and quality checks. Additionally, critical issues and possible solutions with respect to the experimental setup, tasks, and data collection in home settings are discussed. The protocol is not limited to infant-caregiver dyads but can be applied to various dyadic constellations. Overall, we demonstrate the flexibility of EEG hyperscanning setups, which allow experiments to be conducted outside of the laboratory to capture participants' brain activities in more ecologically valid environmental settings. Yet, movement and other types of artifacts still constrain the experimental tasks that can be performed in the home setting.

Author Spotlight: Capturing Infant-Caregiver Interactions Through Synchronized Multimodal Data Collection

Home-Based EEG Hyperscanning for Infant-Caregiver Social Interactions

Our research aims to study how social interactions between infants and caregivers can impact the infant's learning and development. We have developed a comprehensive hyperscanning protocol to capture synchronized EEG, ECG, and behavioral recordings from infant-caregiver dyads during interactive play tasks in the family home. Unlike prior hyperscanning protocols, which have primarily been conducted in laboratory environments, our experiment is conducted in the home setting.One key experimental challenge is balancing the need for standardization across home environments, which can limit ecological validity, with the flexibility required to promote naturalistic interactions between participants. This protocol offers unique insights into addressing potential issues and critical considerations regarding the experimental setup, tasks, and data collection in home settings. Additionally, our protocol provides a detailed demonstration of the design and setup of the multimodal synchronization of EEG, ECG, video and audio data into a single timeline.

Participants included in this study were 8- to 12-month-old, typically developing infants and their mother and/or grandmother who spoke English or English and a second language at home. The 7-electrode EEGs and a single-lead ECG of adults and infants, as well as video and audio recordings from three cameras and microphones, were acquired simultaneously during the tasks. Neural activities were measured over F3, F4, C3, Cz, C4, P3, and P4 according to the international 10-20 system. The different data streams were temporal...

This protocol describes how synchronized electroencephalography, electrocardiography, and behavioral recordings were captured from infant-caregiver dyads in a home setting.

Prior hyperscanning studies that record the brain activities of caregivers and children concurrently have primarily been conducted within the confines of ...