performing the encephalography experiment block-by-block independently

EEG Study on Recognizing Familiar Speakers in Expressive Speech

Human vocal streams are rich in information, such as emotion1,2, health status3,4, biological sex5, age6, and, more importantly, the individual vocal identity7,8. Studies have suggested that human listeners have a robust capacity to recognize and differentiate their peers' identities through voices, overcoming within-speaker variations around speaker identity's average-based representation in the acoustic space9. Such variations are brought about by acoustic manipulation (fundamental frequency and vocal tract length, i.e., F0 and VTL) that corresponds to no clear pragmatic intentions9, emotion prosodies10, and vocal confidence that conveys speakers' feeling of knowing11. Behavioral experiments have focused on many factors that influence listeners' performance in recognizing the talkers, including language-related manipulations8,12,13, participants-related characteristics such as music experience or reading ability14,15, and stimuli-related adaptions like backward speech or nonwords16,17; more can be found in literature reviews18,19. A few recent experiments have investigated how individual variation of the speaker identity representation might undermine the recognition accuracy, considering aspects including high versus low emotional expressiveness16 and neutral versus fearful prosodies5; more possible scenarios open for further investigation, as suggested by a review20. 
For the first research gap, the study proposes that the neurological underpinnings of speaker identification have yet to fully explore how within-speaker variation challenges listeners' brain activities. For example, in an fMRI-based speaker recognition task by Z&#228;ske et al., participants' right posterior superior temporal gyrus (pSTG), right inferior/middle frontal gyrus (IFG/MFG), right medial frontal gyrus, and left caudate showed reduced activation when correctly identified as old versus new talkers, regardless of the linguistic content being the same or different21. However, an earlier electroencephalography (EEG) study by Z&#228;ske et al. did not observe this old/new effect when speaker identity variation was introduced through different texts22. Specifically, a larger, late positive component (LPC) ranging from 300 to 700 ms, detected at the Pz electrode when listeners encountered their familiar trained talker expressing the same text (i.e., hearing a replay with non-varied linguistic content), was absent when the talkers delivered new texts. 
In support of the assertion made by Z&#228;ske et al.21, this study suspects that an old/new effect can still be observed despite differences in linguistic content between training and testing sessions in event-related potential (ERP) analyses. This rationale stems from the notion that the absence of the old/new effect in Z&#228;ske et al.22, under conditions where different texts were used, may be attributed to the lack of an additional check session during the training task to ensure thorough and effective identity learning, as suggested by Lavan et al.23. Consequently, the first objective of the study is to examine and validate this hypothesis. This study aims to test this by adding a checking session to the training-testing paradigm22. 
Another key question this study aims to address is the robustness of speaker identification in the presence of speech prosody. Previous behavioral studies have suggested that listeners particularly struggle to recognize talkers across different prosodies, which indicates a modulatory role of prosodic context - listeners underperformed in the different training-testing prosody conditions. This study aims to test this by exposing listeners to recognize familiar talkers in either confident or doubtful prosodies24. This study expects that the observed ERP differences will help explain how speech prosody influences identity recognition. 
The core objective of the current study is to investigate the robustness of the old/new effect in speaker recognition, specifically examining whether there are differences in recognizing talkers in confident versus doubtful prosodies. Xu and Armony10 performed a behavioral study using a training-testing paradigm, and their findings suggest that listeners cannot overcome prosodic differences (e.g., trained to recognize a talker in neutral prosody and tested on fearful prosody) and can only achieve accuracy lower than chance level10. Acoustic analysis indicates that speakers expressing varied emotive states are associated with VTL/F0 modulation; for example, confident prosody is characterized by lengthened VTL and lower F0, whereas the opposite is true for doubtful prosody11,24. Another piece of evidence comes from the study by Lavan et al.23, which confirmed that listeners can adapt to VTL and F0 changes of the speaker and form average-based representations of the talkers. This study reconciles that, from a behavioral data perspective, listeners are likely to still recognize the talker's identity across prosodies (e.g., trained to recognize one in confident prosody but tested in doubtful prosody; reported in a separate manuscript in preparation). Yet, the neural correlates of speaker identification, specifically the generalizability of the old/new effect observed by Z&#228;ske et al.22, remain unclear. Hence, the current study is committed to validating the robustness of the old/new effect in confident versus doubtful prosodies as contexts for testing. 
The study introduces a departure from previous research paradigms in old/new effects studies. While past research focused on how old/new talker recognition influences perception, this study extends this by incorporating two confidence levels (confident versus doubtful) into the paradigm (thus, a 2+2 study). This allows us to investigate speaker recognition within the contexts of confident and doubtful speech prosodies. The paradigm enables the exploration of the robustness of old/new effects. The analyses of memory effects and regions of interest (ROI) within both confident and doubtful speech contexts serve as evidence for this investigation. 
Altogether, the study aims to update the understanding of the EEG correlates of voice recognition, with the hypotheses that the enlarged LPC of the EEG old/new effect is observable even when 1) the linguistic content is not the same, and 2) with the presence of confident versus doubtful prosody. This study investigated the hypotheses through a three-step paradigm. Firstly, during the training phase, participants established associations between three voices and their corresponding names. Subsequently, in the checking phase, they were tasked with identifying the name corresponding to a voice from a selection of three candidates. This checking, following Lavan et al.23, aims to overcome insufficient old speaker familiarization, which led to the unobserved old/new effect when the text in the training and testing phases differed6, and talkers could not recognize talkers across neutral and fearful prosodies10. Finally, in the testing phase, participants distinguished between three old and three new speaker voices in a two-alternative forced-choice task, with speech prosody presented as either confident or doubtful. EEG data were collected using a 64-channel EEG system and underwent preprocessing before analysis. Statistical analysis and event-related potential (ERP) analysis were conducted in RStudio, while MATLAB was utilized for brain topography analysis.
Regarding design details, this study proposes a speaker identity learning experiment that controls for the talker's height, which is related to VTL and influences impressions of who is talking23. This aspect also influences social impressions, such as perceived dominance25, and such higher-level impression formation might interact with decoding speaker identity26.

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Memorization-Based Training and Testing Paradigm for Robust Vocal Identity Recognition in Expressive Speech Using Event-Related Potentials Analysis

For over 50 years, scientists have been intrigued by how we recognize voices. Our recent study explores how emotional prosodies affect this process. Despite differences in prosodies, we found that the overall old-new effect is robust, indicating that our brains consistently distinguish familiar and unfamiliar voices.Current experimental challenges in speaker identification arise from the commonly used training testing paradigm in the past, which is not suitable for studying detailed variables affecting speaker identity processing. Ensuring that the training is thorough and effective is crucial to address this challenge. Our fundings reveal that while the LPC, or the old-new effect, shows differences between confident and doubtful processes, it remains robust overall.Our paradigm is well suited for investigating the processing of familiar and unfamiliar objects with internal variations. My lab focuses on voice perception, including how pragmatically marked emotions are expressed and perceived and how vocal information has an impact on listeners. We use EEGs and fMRI to measure these factors with a key focus on social factors such as accent or whether the talker is AI.This study on JoVE supports our future research in these areas.

Performing the Encephalography Experiment Block-by-Block Independently

To begin, create a new EEG recording file on the EEG data collection computer. Open brain vision recorder and click on monitor to double check the impedance followed by stat or resume recording. Open the PsychoPy program's run experiment for the behavioral experiment.Enter the participant's ID and the corresponding block number and click okay to start the experiment. Closely monitor the accuracy of data reported on the screen after the participant completes the check phase on the behavioral data computer. Pay close attention to the final reported accuracy of old versus new recognition.After the participant has completed all blocks, invite them to wash their hair. To clean the EEG cap, remove the residual conductive paste with a toothbrush and wrap them in plastic bags. Once cleaned, hang the EEG cap in a well-ventilated area to dry.Copy the EEG and behavioral data onto a portable hard drive, ensuring that the EEG data and behavioral data correspond.

performing-the-encephalography-experiment-block-by-block-independently

Research

JoVE Journal

Behavior

84 Views. To begin, create a new EEG recording file on the EEG data collection computer. Open brain vision recorder and click on monitor to double check the impedance followed by stat or resume recording. Open the PsychoPy program's run experiment for the behavioral experiment.Enter the participant's ID and the corresponding block number and click okay to start the experiment. Closely monitor the accuracy of data reported on the screen after the participant completes the check phase on the behav...