This work was funded by the National Social Science Foundation of China [16BYY076] to Peng Zhou and the Science Foundation of Beijing Language and Cultural University under the Fundamental Research Funds for the Central Universities [15YJ050003]. The authors are grateful to the children, parents, and teachers at the Enqi Autism Platform and Taolifangyuan Kindergarten in Beijing, China, for their support in running the study.

This study has been approved by the Ethics Committee of the School of Medicine at Tsinghua University. Informed consent has been obtained from all individual participants included in the study.
1. Participant Screening and Study Preparation
<ol>
	<li>Recruit Mandarin-speaking preschool children with autism. 
	NOTE: Their diagnoses should be confirmed by pediatric neurologists at hospitals using DSM-IV-TR40 or DSM-541 and, ideally, the number of participants should be no less than 15. The present study recruited 25 participants with confirmed diagnoses.</li>
	<li>Evaluate each participant independently using gold-standard diagnostic instruments like the Autism Diagnostic Observation Schedule42.</li>
	<li>Measure the verbal IQ of participants using the Wechsler Preschool and Primary Scale of Intelligence-IV (CN), a standardized IQ test designed for Mandarin-speaking children between the ages of 2-6 and 6-1143. 
	NOTE: The verbal IQ scores of the children with autism in the present study were all above 80. They were all high-functioning children with autism.</li>
	<li>Calculate each participant&#39;s mean length of utterance (MLU) by dividing the total number of words by the number of utterances in each speech sample. Record 100 utterances for each participant from either their interactions with parents or with teachers. Then, calculate the MLU by dividing the total number of words in each participant&#39;s utterances by 100. 
	NOTE: MLU indicates the participant&#39;s sentence complexity levels.</li>
	<li>Recruit TD children. Ideally match the TD children to the children with autism for age (TD group 1), MLU (TD group 2), and verbal IQ (TD group 3). 
	NOTE: The present study recruited 50 TD children (25 boys and 25 girls) from local kindergartens. 25 matched to the children with autism for age and 25 matched to the children with autism for both MLU and verbal IQ.</li>
</ol>
2. Warm-up Session
<ol>
	<li>Invite the participants for a warm-up session before the actual test. Introduce the participant to the research environment and interact with him or her to establish a good rapport. 
	NOTE: This can be done on the same day as the testing session or organized on a different day. In the warm-up session, two experimenters are typically involved and interact with the participant using toys and props.</li>
</ol>
3. Conditions and Experimental Design
<ol>
	<li>Construct the test stimuli. Create 12 target items, each comprised of a visual stimulus, and two spoken sentences containing the morphological markers BA and BEI, respectively. Construct the spoken sentences using the same structure: morphological marker + noun phrase (NP) + adverb + verb phrase (VP) (see Examples 1a and 1b below). 
	NOTE: The marker BA indicates that the following NP is the recipient of the holding event (see 2a), and BEI indicates that the following NP is the initiator of the event (see 2b). The subject NP of a sentence in Mandarin can often be omitted when the referent of the NP is contextually available. 
	 
	Example: 
	(1)&#160;a. BA shizi qingqingdi bao-le&#160; &#160; &#160; qilai. 
	&#160; &#160; &#160; &#160; &#160;BA&#160; lion&#160; &#160;gently&#160; &#160; &#160; &#160;hold&#160; &#160; &#160; &#160; &#160;up 
	&#160; &#160; &#160; &#160; &#160;Meaning: Someone gently holds the lion. 
	&#160; &#160; &#160;b. BEI shizi qingqingdi bao-le&#160; &#160; &#160; qilai. 
	&#160; &#160; &#160; &#160; &#160;BEI lion&#160; &#160;gently&#160; &#160; &#160; &#160;hold&#160; &#160; &#160; &#160; &#160; up 
	&#160; &#160; &#160; &#160; &#160;Meaning: Someone is gently held by the lion. 
	(2)&#160;a. BA + [NP]Recipient 
	&#160; &#160; &#160;b. BEI + [NP]Initiator
	<ol>
		<li>Use Pixelmator (or another image editor) to create visual images. Open Pixelmator. Click on the Pixelmator icon. Create a visual image from a template. Click Show Details in the template chooser. Double-click the template to open it. Adjust the width, height, resolution, and color depth from the pop-up menus. Enter the relevant parameters. Click OK.</li>
		<li>Use Praat (or another audio editor) to construct spoken sentences. Set up the microphone. Open Praat. Click on the Praat icon. Select Record Mono sound from the New menu. Set the recording conditions by clicking the sample rate option of 44100. Click the Record button.</li>
		<li>Record the spoken sentences by asking a native Beijing Mandarin-speaker to produce the sentences in a child-directed manner. Save the recordings by clicking Save. 
		NOTE: Typically, 12 to 16 target items are constructed for sentence comprehension studies with children. Test stimuli can be created using other image and audio editors for a visual world study.</li>
	</ol>
	</li>
	<li>Construct visual images, each containing two pictures. The two pictures depict the same event involving the same characters. Reverse the event roles (initiator or recipient) of the two characters in the two pictures. Make one picture compatible with the construction containing BA (BA-target event) and one with the construction containing BEI (BEI-target event). An example is provided in Figure 1. 
	NOTE: This figure has been reprinted with permission from Zhou and Ma (2018)19.</li>
	<li>Counterbalance and randomization: divide the target trials into two experimental lists, with a participant seeing each visual stimulus but listening to only one of the recorded sentences for the stimulus. Counterbalance the spoken sentences containing BA and BEI across the two experimental lists, with 6 constructions containing BA and 6 containing BEI. Add 12 filler items to each experimental list and arrange the target and filler trials in a random order. Randomly assign the participants to the two lists.</li>
</ol>
4. Experimental Procedure
<ol>
	<li>Eye-tracking procedure.
	<ol>
		<li>Invite the participants to sit comfortably in front of the display monitor of the remote eye tracker. Set the distance between the participants&#39; eyes and the monitor around 60 cm. Perform the standard calibration and validation procedures by asking the participants to fixate on a grid of five fixation targets in random succession.</li>
		<li>Present the participants with a spoken sentence while they are seeing a visual image, as done in the standard visual world paradigm10,44. Use the monocular eye-tracking option by tracking the eye that is on the same side as the illuminator of the eye tracker. Record the participant&#39;s eye movements using the eye tracker. 
		NOTE: The eye tracker used in the present study allows remote eye-tracking with a sampling rate of 500 Hz.</li>
	</ol>
	</li>
	<li>Testing and measuring.
	<ol>
		<li>Test the participants individually. Simply tell the participants to listen to the spoken sentences while they are viewing the pictures. Ask one experimenter to monitor the participant on the computer and one to stand behind the participant and gently rest her hands on the participant&#39;s shoulders to minimize the participant&#39;s sudden movements.</li>
		<li>Measure the participant&#39;s eye movements that arise as automatic responses to the linguistic input using the eye tracker. 
		NOTE: The task does not ask participants to make any conscious judgments about the spoken sentences to minimize their computational burden. The eye tracker automatically records the eye movements.</li>
		<li>Monitoring during the test: use the live viewer mode on the computer screen, exhibited by the eye tracker during the test, to observe the participant&#39;s looking behavior. Ask the experimenter who monitors data collection via the live viewer mode to signal to the experimenter who stands behind the participant to reorient the participant if his or her eye gaze wanders off the computer screen.</li>
	</ol>
	</li>
</ol>
5. Data Treatment and Analysis
<ol>
	<li>Code the participants&#39; fixations in two interest areas. Use Data Viewer to draw the two interest areas: BA-target event area and BEI-target event area (see Figure 1). Open Data Viewer. Select one of the interest area shape icons on the tool bar. Use the mouse to drag a box around the region you want to define as an interest area. Save the interest area in the Interest Area Set folder. Apply the interest area to other visual images. 
	NOTE: The depicted event in the upper panel of Figure 1 matches the BA-construction, hence the BA-target event, and the event depicted in the lower panel matches Figure 1b, hence the BEI-target event. The software used for data coding is Data Viewer, which comes with the eye tracker used in the study. Other data analysis software is also available.</li>
	<li>Analyze the eye gaze patterns using Data Viewer.
	<ol>
		<li>Open Data Viewer. Choose the sample report function from the menu to set the time windows for analysis (e.g., every 200 ms for the time window in the present study). Use the same function to time lock the fixation proportions in the interest areas to the onset of the marker for each trial. Export the raw data into an excel file using the export function from the menu.</li>
		<li>Use the excel functions to average the fixation proportions following the onset of the marker for each area. Use the excel functions to compute the fixation proportions in each time window of 200 ms over a period of 5200 ms (the mean length of the target sentences + 200 ms) from the onset of the marker for the two areas. Apply linear mixed-effects models to the eye movement data, detailed in Representative Results below. 
		NOTE: The use of 200 ms as a time window is based on the standard procedure for analyzing child eye gaze data in the literature12,13,18,19,45,46,47, and it is generally assumed that it takes about 200 ms to observe the effects of linguistic markers on eye movements48.</li>
	</ol>
	</li>
</ol>

The authors have nothing to disclose.

In the current study, we present an eye-tracking paradigm that can directly and effectively assess the sentence comprehension abilities of children with autism. We found that 5-year-old children with autism, like their age-matched TD peers, exhibited eye gaze patterns that reflect effective and rapid use of linguistic cues during real-time sentence comprehension.
The findings provide evidence that eye-tracking (in particular, the visual world paradigm) is a sensitive measure of real-time sente...

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information1,2,3,4,5,6,7,8,9,10,11. Prior research has found that young typically developing (TD) children incrementally compute the meaning of a sentence using both linguistic and non-linguistic cues12,13,14,15,16,17,18,19. However, there is currently a paucity of research exploring how preschool children with autism understand a sentence using different types of cues. The mechanisms underlying their sentence comprehension remains largely unclear.
It is generally acknowledged that there is enormous variability in the language abilities of children with autism, especially in their expressive language; for example, some children with autism have relatively good structural language, some exhibit deficits in both lexical and grammatical domains, some demonstrate impaired grammar, and some never acquire functional spoken language20,21,22,23,24,25. In addition, prior research seems to suggest that their receptive language is relatively more impaired than their expressive language26,27,28,29. Most research that has assessed sentence comprehension abilities of children with autism have used offline tasks (e.g., standardized tests, caregiver reports), and the findings suggest that their sentence comprehension abilities might be particularly impaired30,31,32,33,34,35,36,37. However, it has been pointed out that poor comprehension abilities are more likely related to these children&#39;s overall lack of social responsiveness than to language processing deficits38,39. Note that these offline tasks used in previous research often require high response demands or interactions with the experimenters, which might pose particular difficulties for children with autism, because they often exhibit various challenging behaviors or symptoms. As a result, this may interact with the high task and communication demands and mask their comprehension abilities [for an overview of methods for assessing receptive language in children with autism, see Kasari et al. (2013)27 and Plesa-Skwerer et al. (2016)29]. Thus, experimental paradigms that can better control these confounding factors are required to further understand the nature of sentence-processing mechanisms in autism.
In the current study, we present an eye-tracking paradigm that can directly and effectively assess sentence comprehension abilities of children with autism. Compared to offline tasks, eye-tracking is a more sensitive testing paradigm to demonstrate children&#39;s comprehension abilities. It is sensitive to the time course of the comprehension process and requires no explicit motor or language responses from the participant, making it a promising method to study younger children and minimally verbal children with autism. In addition, we record eye movements as automatic responses to linguistic input instead of measuring eye movements that accompany conscious responses to linguistic input.

<table><tbody><tr><td>EyeLink 1000 plus eye tracker&amp;nbsp;</td><td>SR Research Ltd.</td><td></td><td>&amp;nbsp;The EyeLink 1000 plus allows remote eye tracking, without a head support. The eye tracker provides information about the participant&amp;rsquo;s point of gaze at a sampling rate of 500 Hz, and it has accuracy of 0.5 degrees of visual angle.&amp;nbsp;</td></tr></tbody></table>

using the visual world paradigm to study sentence comprehension in mandarin-speaking children with autism

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information. However, there is currently a paucity of research exploring how preschool children with autism understand sentences using different types of cues. The mechanisms underlying sentence comprehension remains largely unclear. The present study presents a protocol to examine the sentence comprehension abilities of preschool children with autism. More specifically, a visual world paradigm of eye-tracking is used to explore the moment-to-moment sentence comprehension in the children. The paradigm has multiple advantages. First, it is sensitive to the time course of sentence comprehension and thus can provide rich information about how sentence comprehension unfolds over time. Second, it requires minimal task and communication demands, so it is ideal for testing children with autism. To further minimize the computational burden of children, the present study measures eye movements that arise as automatic responses to linguistic input rather than measuring eye movements that accompany conscious responses to spoken instructions.

The present study uses minimal pairs as in Examples 1a and 1b to investigate if and how fast children with autism can use event information encoded in two morphological markers during real-time sentence comprehension. It was predicted that if they are able to rapidly and effectively use event information in the two markers during real-time sentence comprehension, then they should look more at the BA-target event when hearing BA than when hearing BEI. Also, they should fixate more on the B...

Watch this Scientific Journal Video about Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism at JoVE.com

Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism

We present a protocol to examine the use of morphological cues during real-time sentence comprehension by children with autism.

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information. However, there is ...

using-visual-world-paradigm-to-study-sentence-comprehension-mandarin

Research

JoVE Journal

Behavior

7.6K Views.  Tsinghua University. We present a protocol to examine the use of morphological cues during real-time sentence comprehension by children with autism.

Video: Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism

Assessment and Evaluation of the High Risk Neonate: The NICU Network Neurobehavioral Scale

There has been a long-standing interest in the assessment of the neurobehavioral integrity of the newborn infant. The NICU Network Neurobehavioral Scale (NNNS) was developed as an assessment for the at-risk infant. These are infants who are at increased risk for poor developmental outcome because of insults during prenatal development, such as substance exposure or prematurity or factors such as poverty, poor nutrition or lack of prenatal care that can have adverse effects on the intrauterine environment and affect the developing fetus. The NNNS assesses the full range of infant neurobehavioral performance including neurological integrity, behavioral functioning, and signs of stress/abstinence. The NNNS is a noninvasive neonatal assessment tool with demonstrated validity as a predictor, not only of medical outcomes such as cerebral palsy diagnosis, neurological abnormalities, and diseases with risks to the brain, but also of developmental outcomes such as mental and motor functioning, behavior problems, school readiness, and IQ. The NNNS can identify infants at high risk for abnormal developmental outcome and is an important clinical tool that enables medical researchers and health practitioners to identify these infants and develop intervention programs to optimize the development of these infants as early as possible. The video shows the NNNS procedures, shows examples of normal and abnormal performance and the various clinical populations in which the exam can be used.

The NICU Network Neurobehavioral Scale (NNNS) was developed as an assessment for the at-risk infant. The purpose of this article is to describe the NNNS, provide video examples of the NNNS procedures and discuss the ways in which the exam has been used.

There has been a long-standing interest in the assessment of the neurobehavioral integrity of the newborn infant.  The NICU Network Neurobehavioral Scale ...

Using Eye Movements to Evaluate the Cognitive Processes Involved in Text Comprehension

The present article describes how to use eye tracking methodologies to study the cognitive processes involved in text comprehension. Measuring eye movements during reading is one of the most precise methods for measuring moment-by-moment (online) processing demands during text comprehension. Cognitive processing demands are reflected by several aspects of eye movement behavior, such as fixation duration, number of fixations, and number of regressions (returning to prior parts of a text). Important properties of eye tracking equipment that researchers need to consider are described, including how frequently the eye position is measured (sampling rate), accuracy of determining eye position, how much head movement is allowed, and ease of use. Also described are properties of stimuli that influence eye movements that need to be controlled in studies of text comprehension, such as the position, frequency, and length of target words. Procedural recommendations related to preparing the participant, setting up and calibrating the equipment, and running a study are given. Representative results are presented to illustrate how data can be evaluated. Although the methodology is described in terms of reading comprehension, much of the information presented can be applied to any study in which participants read verbal stimuli.

The present article describes how to use eye tracking methodologies to study the cognitive processes involved in text comprehension. Descriptions of eye tracking equipment, how to develop experimental stimuli, and procedural recommendations are included. The information presented can be applied to most any study using verbal stimuli.

The present article describes how to use eye  tracking methodologies to study the cognitive processes involved in text  comprehension. Measuring eye ...

Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder

In addition to impairments in social communication and the presence of restricted interests and repetitive behaviors, deficits in sensory processing are now recognized as a core symptom in autism spectrum disorder (ASD). Our ability to perceive and interact with the external world is rooted in sensory processing. For example, listening to a conversation entails processing the auditory cues coming from the speaker (speech content, prosody, syntax) as well as the associated visual information (facial expressions, gestures). Collectively, the &#8220;integration&#8221; of these multisensory (i.e., combined audiovisual) pieces of information results in better comprehension. Such multisensory integration has been shown to be strongly dependent upon the temporal relationship of the paired stimuli. Thus, stimuli that occur in close temporal proximity are highly likely to result in behavioral and perceptual benefits &#8211; gains believed to be reflective of the perceptual system&#39;s judgment of the likelihood that these two stimuli came from the same source. Changes in this temporal integration are expected to strongly alter perceptual processes, and are likely to diminish the ability to accurately perceive and interact with our world. Here, a battery of tasks designed to characterize various aspects of sensory and multisensory temporal processing in children with ASD is described. In addition to its utility in autism, this battery has great potential for characterizing changes in sensory function in other clinical populations, as well as being used to examine changes in these processes across the lifespan.

We describe how to implement a battery of behavioral tasks to examine the processing and integration of sensory stimuli in children with ASD. The goal is to characterize individual differences in temporal processing of simple auditory and visual stimuli and relate these to higher order perceptual skills like speech perception.

In addition to impairments in social communication and the presence of restricted interests and repetitive behaviors, deficits in sensory processing are ...

Eye Tracking Young Children with Autism

The rise of accessible commercial eye-tracking systems has fueled a rapid increase in their use in psychological and psychiatric research. By providing a direct, detailed and objective measure of gaze behavior, eye-tracking has become a valuable tool for examining abnormal perceptual strategies in clinical populations and has been used to identify disorder-specific characteristics1, promote early identification2, and inform treatment3. In particular, investigators of autism spectrum disorders (ASD) have benefited from integrating eye-tracking into their research paradigms4-7. Eye-tracking has largely been used in these studies to reveal mechanisms underlying impaired task performance8 and abnormal brain functioning9, particularly during the processing of social information1,10-11. While older children and adults with ASD comprise the preponderance of research in this area, eye-tracking may be especially useful for studying young children with the disorder as it offers a non-invasive tool for assessing and quantifying early-emerging developmental abnormalities2,12-13. Implementing eye-tracking with young children with ASD, however, is associated with a number of unique challenges, including issues with compliant behavior resulting from specific task demands and disorder-related psychosocial considerations. In this protocol, we detail methodological considerations for optimizing research design, data acquisition and psychometric analysis while eye-tracking young children with ASD. The provided recommendations are also designed to be more broadly applicable for eye-tracking children with other developmental disabilities. By offering guidelines for best practices in these areas based upon lessons derived from our own work, we hope to help other investigators make sound research design and analysis choices while avoiding common pitfalls that can compromise data acquisition while eye-tracking young children with ASD or other developmental difficulties.

Eye tracking has long been used to study gaze patterns in typically-developing individuals, but recent technological advancements have made its use with clinical populations, including autism, more feasible. While eye-tracking young children with autism can offer insight into early symptom manifestations, it involves methodological challenges. Suggestions for best practices are provided.

The rise of accessible commercial eye-tracking systems has fueled a rapid increase in their use in psychological and psychiatric research. By providing a ...

Medicine

Portable Intermodal Preferential Looking (IPL): Investigating Language Comprehension in Typically Developing Toddlers and Young Children with Autism

One of the defining characteristics of autism spectrum disorder (ASD) is difficulty with language and communication.1 Children with ASD's onset of speaking is usually delayed, and many children with ASD consistently produce language less frequently and of lower lexical and grammatical complexity than their typically developing (TD) peers.6,8,12,23 However, children with ASD also exhibit a significant social deficit, and researchers and clinicians continue to debate the extent to which the deficits in social interaction account for or contribute to the deficits in language production.5,14,19,25
Standardized assessments of language in children with ASD usually do include a comprehension component; however, many such comprehension tasks assess just one aspect of language (e.g., vocabulary),5 or include a significant motor component (e.g., pointing, act-out), and/or require children to deliberately choose between a number of alternatives. These last two behaviors are known to also be challenging to children with ASD.7,12,13,16
We present a method which can assess the language comprehension of young typically developing children (9-36 months) and children with autism.2,4,9,11,22 This method, Portable Intermodal Preferential Looking (P-IPL), projects side-by-side video images from a laptop onto a portable screen. The video images are paired first with a 'baseline' (nondirecting) audio, and then presented again paired with a 'test' linguistic audio that matches only one of the video images. Children's eye movements while watching the video are filmed and later coded. Children who understand the linguistic audio will look more quickly to, and longer at, the video that matches the linguistic audio.2,4,11,18,22,26
This paradigm includes a number of components that have recently been miniaturized (projector, camcorder, digitizer) to enable portability and easy setup in children's homes. This is a crucial point for assessing young children with ASD, who are frequently uncomfortable in new (e.g., laboratory) settings. Videos can be created to assess a wide range of specific components of linguistic knowledge, such as Subject-Verb-Object word order, wh-questions, and tense/aspect suffixes on verbs; videos can also assess principles of word learning such as a noun bias, a shape bias, and syntactic bootstrapping.10,14,17,21,24 Videos include characters and speech that are visually and acoustically salient and well tolerated by children with ASD.

Portable Intermodal Preferential Looking IPL: Investigating Language Comprehension in Typically Developing Toddlers and Young Children with Autism

A reliable home-based way to assess the language comprehension of very young typically developing children, as well as those with autism, is described. The method analyzes children's eye gaze while viewing side-by-side images but hearing an audio that matches only one image. Stimuli are designed with young participants in mind.

One  of the defining characteristics of autism spectrum disorder (ASD) is difficulty  with language and communication.1  Children with ASD's onset of ...

A Semantic Priming Event-related Potential (ERP) Task to Study Lexico-semantic and Visuo-semantic Processing in Autism Spectrum Disorder

Individuals with autism spectrum disorder (ASD) have characteristic deficits in understanding the meaning of language, or semantic processing. However, some evidence indicates that semantic processing of non-linguistic stimuli is intact, suggesting that semantic deficits may be language-specific. To appropriately characterize semantic processing deficits in individuals with ASD, comparison of within-modality linguistic (e.g., written words) and non-linguistic (e.g., pictures) stimuli is required. This paper describes such a methodology that makes use of a semantic priming paradigm during concurrent recording of electroencephalographic (EEG) data. EEG provides a dynamic measure of brain activity that is well-suited to characterize subtle differences in semantic processing that may not be observable at the behavioral level. The semantic priming paradigm presents a prime picture or word (e.g., dog) followed by a target picture or word that is either related (e.g., cat) or unrelated (e.g., pencil) to the prime. This paradigm can thus be used to evaluate semantic processing across different modalities, and to compare lexico-semantic and visuo-semantic processing abilities in individuals with ASD and how they might differ from TD individuals. The specific steps involved in creating the stimuli, performing the EEG testing, and analyzing the EEG data are discussed. Representative results illustrate how the N400 component of the event-related potential (ERP) is reduced following semantically-related prime-target pairs compared to unrelated pairs. Comparisons of the N400 between conditions, modalities, and groups can provide estimates of the success of semantic processing, and can thereby be used to characterize semantic deficits in individuals with ASD or other clinical populations.

A Semantic Priming Event-related Potential ERP Task to Study Lexico-semantic and Visuo-semantic Processing in Autism Spectrum Disorder

This paper describes a semantic priming ERP task using within-modality pairs of pictures and words to investigate semantic processing in individuals with autism spectrum disorder (ASD).

Individuals with autism spectrum disorder (ASD) have characteristic deficits in understanding the meaning of language, or semantic processing. However, ...

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

The present work is a description and an assessment of a methodology designed to quantify different aspects of the interaction between language processing and the perception of the visual world. The recording of eye-gaze patterns has provided good evidence for the contribution of both the visual context and linguistic/world knowledge to language comprehension. Initial research assessed object-context effects to test&#160;theories of modularity in language processing. In the introduction, we describe how subsequent investigations have taken the role of the wider visual context in language processing as a research topic in its own right, asking questions such as how our visual perception of events and of speakers contributes to&#160;comprehension informed by&#160;comprehenders&#39; experience. Among the examined aspects of the visual context are actions, events, a speaker&#39;s gaze, and emotional facial expressions, as well as spatial object configurations. Following an overview of the eye-tracking method and its different applications, we list the key steps of the methodology in the protocol, illustrating how to successfully use it to study visually-situated language comprehension. A final section presents three sets of representative results and illustrates the benefits and limitations of eye tracking for investigating the interplay between the perception of the visual world and language comprehension.

The present article reviews an eye-tracking methodology for studies on language comprehension. To obtain reliable data, key steps of the protocol must be followed. Among these are the correct set-up of the eye tracker (e.g., ensuring good quality of the eye and head images) and accurate calibration.

The present work is a description and an assessment of a methodology designed to quantify different aspects of the interaction between language processing ...

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

In a typical eye tracking study using the visual world paradigm, participants' eye movements to objects or pictures in the visual workspace are recorded via an eye tracker as the participant produces or comprehends a spoken language describing the concurrent visual world. This paradigm has high versatility, as it can be used in a wide range of populations, including those who cannot read and/or who cannot overtly give their behavioral responses, such as preliterate children, elderly adults, and patients. More importantly, the paradigm is extremely sensitive to fine grained manipulations of the speech signal, and it can be used to study the online processing of most topics in language comprehension at multiple levels, such as the fine grained acoustic phonetic features, the properties of words, and the linguistic structures. The protocol described in this article illustrates how a typical visual world eye tracking study is conducted, with an example showing how the online processing of some semantically complex statements can be explored with the visual world paradigm.

The visual world paradigm monitors participants' eye movements in the visual workspace as they are listening to or speaking a spoken language. This paradigm can be used to investigate the online processing of a wide range of psycholinguistic questions, including semantically complex statements, such as disjunctive statements.

In a typical eye tracking study using the visual world paradigm, participants' eye movements to objects or pictures in the visual workspace are recorded ...

Comparing Eye-tracking Data of Children with High-functioning ASD, Comorbid ADHD, and of a Control Watching Social Videos

Children with autism spectrum disorders (ASD) are known to have sensory-perceptual processing deficits that weaken their abilities to attend and perceive social stimuli in daily living contexts. Since daily social episodes consist of subtle dynamic changes in social information, any failure to attend to or process subtle human nonverbal cues, such as facial expression, postures, and gestures, might lead to inappropriate social interaction. Traditional behavioral rating scales or assessment tools based on static social scenes have limitations in capturing the moment-to-moment changes in social scenarios. An eye-tracking assessment, which can be administered in a video-based mode, is therefore preferred, to augment clinical observation. In this study, using the single-case comparison design, the eye-tracking data of three participants, a child with autism spectrum disorder (ASD), another with comorbid attention deficit-hyperactive disorder (ADHD), and a neurotypical control, are captured while they view a video of social scenarios. The eye-tracking experiment has helped answer the research question: How does social attention differ between the three participants? By predefining areas of interest (AOIs), their visual attention on relevant or irrelevant social stimuli, how fast each participant attends to the first social stimuli appearing in the videos, for how long each participant continues to attend to those stimuli within the AOIs, and the gaze shifts between multiple social stimuli appearing concurrently in the same social scene are captured, compared, and analyzed in a video-based eye-tracking experiment.

This is a qualitative comparative case study analysis of eye-tracking data on the first moments of social video scenes as viewed by three participants: one with autism spectrum disorder, one with comorbid attention deficit-hyperactive disorder, and one neurotypical control.

Children with autism spectrum disorders (ASD) are known to have sensory-perceptual processing deficits that weaken their abilities to attend and perceive ...

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Working memory (WM) plays a central role in the comprehension of complex sentences. Its function in the processing of spoken complex sentences is especially evident because spoken complex sentence processing is memory-intensive. The dual-modal interference paradigm has been used to examine how the WM system is involved in complex syntactic processing. This article presents two exemplar experiments involving auditory processing with either intra- or extrasentential interferences. In the first experiment, auditory stimuli [spoken-Chinese relative clause (RC) sentences with two syntactic types: subject-gapped (SRC) vs. object-gapped (ORC)] are interfered with via a visually presented lexical decision task within a sentence and manipulated using three different interference timepoints. In the second experiment, the same auditory stimuli, presented via an auditory window moving technique, are interfered with via a visually presented digital recall task beyond the sentence and manipulated using three digital memory loads. By assessing how the primary task of comprehending the RC sentences is affected by the secondary task, we can tackle the controversial issue concerning Chinese RC processing asymmetry. Our results reveal different patterns of RC processing compared to those reported in previous studies. Experiment 1 manifests no clear RC processing advantages in either SRC or ORC; however, a preference for ORC is observed at the ends of sentences, and a preference for SRC is found at the main verb site. Likewise, Experiment 2 presents a dynamic pattern. Under a no-digit load, SRCs show processing advantages in the RC marker region. However, under higher-digit-load interference, ORCs show processing advantages in the same region. These results lead to the conjecture that no obvious or intrinsic processing asymmetry exists in the processing of Chinese RCs. Using the approach of assessing specific interference during syntactic processing, these experiments demonstrate future research applications that explore the processing metrics of spoken sentences involving working memory.

Here, we present a protocol employing dual-modal interference tasks to examine online processing of spoken-Chinese relative clause sentences. Two exemplar experiments involving auditory processing with intra- and extrasentential interferences are described. The paradigm provides a methodology for addressing the nature of working memory and its effects on sentence processing.