Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism

Summary

We present a protocol to examine the use of morphological cues during real-time sentence comprehension by children with autism.

Abstract

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information. However, there is currently a paucity of research exploring how preschool children with autism understand sentences using different types of cues. The mechanisms underlying sentence comprehension remains largely unclear. The present study presents a protocol to examine the sentence comprehension abilities of preschool children with autism. More specifically, a visual world paradigm of eye-tracking is used to explore the moment-to-moment sentence comprehension in the children. The paradigm has multiple advantages. First, it is sensitive to the time course of sentence comprehension and thus can provide rich information about how sentence comprehension unfolds over time. Second, it requires minimal task and communication demands, so it is ideal for testing children with autism. To further minimize the computational burden of children, the present study measures eye movements that arise as automatic responses to linguistic input rather than measuring eye movements that accompany conscious responses to spoken instructions.

Introduction

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information^{1,2,3,4,5,6,7,8,9,10,11}. Prior research has found that young typically developing (TD) children incrementally compute the meaning of a sentence using both linguistic and non-linguistic cues^{12,13,14,15,16,17,18,19}. However, there is currently a paucity of research exploring how preschool children with autism understand a sentence using different types of cues. The mechanisms underlying their sentence comprehension remains largely unclear.

It is generally acknowledged that there is enormous variability in the language abilities of children with autism, especially in their expressive language; for example, some children with autism have relatively good structural language, some exhibit deficits in both lexical and grammatical domains, some demonstrate impaired grammar, and some never acquire functional spoken language^{20,21,22,23,24,25}. In addition, prior research seems to suggest that their receptive language is relatively more impaired than their expressive language^26,27,28,29. Most research that has assessed sentence comprehension abilities of children with autism have used offline tasks (e.g., standardized tests, caregiver reports), and the findings suggest that their sentence comprehension abilities might be particularly impaired^{30,31,32,33,34,35,36,37}. However, it has been pointed out that poor comprehension abilities are more likely related to these children's overall lack of social responsiveness than to language processing deficits^38,39. Note that these offline tasks used in previous research often require high response demands or interactions with the experimenters, which might pose particular difficulties for children with autism, because they often exhibit various challenging behaviors or symptoms. As a result, this may interact with the high task and communication demands and mask their comprehension abilities [for an overview of methods for assessing receptive language in children with autism, see Kasari et al. (2013)²⁷ and Plesa-Skwerer et al. (2016)²⁹]. Thus, experimental paradigms that can better control these confounding factors are required to further understand the nature of sentence-processing mechanisms in autism.

In the current study, we present an eye-tracking paradigm that can directly and effectively assess sentence comprehension abilities of children with autism. Compared to offline tasks, eye-tracking is a more sensitive testing paradigm to demonstrate children's comprehension abilities. It is sensitive to the time course of the comprehension process and requires no explicit motor or language responses from the participant, making it a promising method to study younger children and minimally verbal children with autism. In addition, we record eye movements as automatic responses to linguistic input instead of measuring eye movements that accompany conscious responses to linguistic input.

Protocol

This study has been approved by the Ethics Committee of the School of Medicine at Tsinghua University. Informed consent has been obtained from all individual participants included in the study.

1. Participant Screening and Study Preparation

1. Recruit Mandarin-speaking preschool children with autism.
   NOTE: Their diagnoses should be confirmed by pediatric neurologists at hospitals using DSM-IV-TR⁴⁰ or DSM-5⁴¹ and, ideally, the number of participants should be no less than 15. The present study recruited 25 participants with confirmed diagnoses.
2. Evaluate each participant independently using gold-standard diagnostic instruments like the Autism Diagnostic Observation Schedule⁴².
3. Measure the verbal IQ of participants using the Wechsler Preschool and Primary Scale of Intelligence-IV (CN), a standardized IQ test designed for Mandarin-speaking children between the ages of 2-6 and 6-11⁴³.
   NOTE: The verbal IQ scores of the children with autism in the present study were all above 80. They were all high-functioning children with autism.
4. Calculate each participant's mean length of utterance (MLU) by dividing the total number of words by the number of utterances in each speech sample. Record 100 utterances for each participant from either their interactions with parents or with teachers. Then, calculate the MLU by dividing the total number of words in each participant's utterances by 100.
   NOTE: MLU indicates the participant's sentence complexity levels.
5. Recruit TD children. Ideally match the TD children to the children with autism for age (TD group 1), MLU (TD group 2), and verbal IQ (TD group 3).
   NOTE: The present study recruited 50 TD children (25 boys and 25 girls) from local kindergartens. 25 matched to the children with autism for age and 25 matched to the children with autism for both MLU and verbal IQ.

2. Warm-up Session

1. Invite the participants for a warm-up session before the actual test. Introduce the participant to the research environment and interact with him or her to establish a good rapport.
   NOTE: This can be done on the same day as the testing session or organized on a different day. In the warm-up session, two experimenters are typically involved and interact with the participant using toys and props.

3. Conditions and Experimental Design

1. Construct the test stimuli. Create 12 target items, each comprised of a visual stimulus, and two spoken sentences containing the morphological markers BA and BEI, respectively. Construct the spoken sentences using the same structure: morphological marker + noun phrase (NP) + adverb + verb phrase (VP) (see Examples 1a and 1b below).
   NOTE: The marker BA indicates that the following NP is the recipient of the holding event (see 2a), and BEI indicates that the following NP is the initiator of the event (see 2b). The subject NP of a sentence in Mandarin can often be omitted when the referent of the NP is contextually available.
   
   Example:
   (1) a. BA shizi qingqingdi bao-le      qilai.
            BA  lion   gently       hold         up
            Meaning: Someone gently holds the lion.
        b. BEI shizi qingqingdi bao-le      qilai.
            BEI lion   gently       hold          up
            Meaning: Someone is gently held by the lion.
   (2) a. BA + [NP]_Recipient
        b. BEI + [NP]_Initiator
   1. Use Pixelmator (or another image editor) to create visual images. Open Pixelmator. Click on the Pixelmator icon. Create a visual image from a template. Click Show Details in the template chooser. Double-click the template to open it. Adjust the width, height, resolution, and color depth from the pop-up menus. Enter the relevant parameters. Click OK.
   2. Use Praat (or another audio editor) to construct spoken sentences. Set up the microphone. Open Praat. Click on the Praat icon. Select Record Mono sound from the New menu. Set the recording conditions by clicking the sample rate option of 44100. Click the Record button.
   3. Record the spoken sentences by asking a native Beijing Mandarin-speaker to produce the sentences in a child-directed manner. Save the recordings by clicking Save.
      NOTE: Typically, 12 to 16 target items are constructed for sentence comprehension studies with children. Test stimuli can be created using other image and audio editors for a visual world study.
2. Construct visual images, each containing two pictures. The two pictures depict the same event involving the same characters. Reverse the event roles (initiator or recipient) of the two characters in the two pictures. Make one picture compatible with the construction containing BA (BA-target event) and one with the construction containing BEI (BEI-target event). An example is provided in Figure 1.
   NOTE: This figure has been reprinted with permission from Zhou and Ma (2018)¹⁹.
3. Counterbalance and randomization: divide the target trials into two experimental lists, with a participant seeing each visual stimulus but listening to only one of the recorded sentences for the stimulus. Counterbalance the spoken sentences containing BA and BEI across the two experimental lists, with 6 constructions containing BA and 6 containing BEI. Add 12 filler items to each experimental list and arrange the target and filler trials in a random order. Randomly assign the participants to the two lists.

4. Experimental Procedure

1. Eye-tracking procedure.
   1. Invite the participants to sit comfortably in front of the display monitor of the remote eye tracker. Set the distance between the participants' eyes and the monitor around 60 cm. Perform the standard calibration and validation procedures by asking the participants to fixate on a grid of five fixation targets in random succession.
   2. Present the participants with a spoken sentence while they are seeing a visual image, as done in the standard visual world paradigm^10,44. Use the monocular eye-tracking option by tracking the eye that is on the same side as the illuminator of the eye tracker. Record the participant's eye movements using the eye tracker.
      NOTE: The eye tracker used in the present study allows remote eye-tracking with a sampling rate of 500 Hz.
2. Testing and measuring.
   1. Test the participants individually. Simply tell the participants to listen to the spoken sentences while they are viewing the pictures. Ask one experimenter to monitor the participant on the computer and one to stand behind the participant and gently rest her hands on the participant's shoulders to minimize the participant's sudden movements.
   2. Measure the participant's eye movements that arise as automatic responses to the linguistic input using the eye tracker.
      NOTE: The task does not ask participants to make any conscious judgments about the spoken sentences to minimize their computational burden. The eye tracker automatically records the eye movements.
   3. Monitoring during the test: use the live viewer mode on the computer screen, exhibited by the eye tracker during the test, to observe the participant's looking behavior. Ask the experimenter who monitors data collection via the live viewer mode to signal to the experimenter who stands behind the participant to reorient the participant if his or her eye gaze wanders off the computer screen.

5. Data Treatment and Analysis

1. Code the participants' fixations in two interest areas. Use Data Viewer to draw the two interest areas: BA-target event area and BEI-target event area (see Figure 1). Open Data Viewer. Select one of the interest area shape icons on the tool bar. Use the mouse to drag a box around the region you want to define as an interest area. Save the interest area in the Interest Area Set folder. Apply the interest area to other visual images.
   NOTE: The depicted event in the upper panel of Figure 1 matches the BA-construction, hence the BA-target event, and the event depicted in the lower panel matches Figure 1b, hence the BEI-target event. The software used for data coding is Data Viewer, which comes with the eye tracker used in the study. Other data analysis software is also available.
2. Analyze the eye gaze patterns using Data Viewer.
   1. Open Data Viewer. Choose the sample report function from the menu to set the time windows for analysis (e.g., every 200 ms for the time window in the present study). Use the same function to time lock the fixation proportions in the interest areas to the onset of the marker for each trial. Export the raw data into an excel file using the export function from the menu.
   2. Use the excel functions to average the fixation proportions following the onset of the marker for each area. Use the excel functions to compute the fixation proportions in each time window of 200 ms over a period of 5200 ms (the mean length of the target sentences + 200 ms) from the onset of the marker for the two areas. Apply linear mixed-effects models to the eye movement data, detailed in Representative Results below.
      NOTE: The use of 200 ms as a time window is based on the standard procedure for analyzing child eye gaze data in the literature^{12,13,18,19,45,46,47}, and it is generally assumed that it takes about 200 ms to observe the effects of linguistic markers on eye movements⁴⁸.

Results

The present study uses minimal pairs as in Examples 1a and 1b to investigate if and how fast children with autism can use event information encoded in two morphological markers during real-time sentence comprehension. It was predicted that if they are able to rapidly and effectively use event information in the two markers during real-time sentence comprehension, then they should look more at the BA-target event when hearing BA than when hearing BEI. Also, they should fixate more on the B...

Discussion

In the current study, we present an eye-tracking paradigm that can directly and effectively assess the sentence comprehension abilities of children with autism. We found that 5-year-old children with autism, like their age-matched TD peers, exhibited eye gaze patterns that reflect effective and rapid use of linguistic cues during real-time sentence comprehension.

The findings provide evidence that eye-tracking (in particular, the visual world paradigm) is a sensitive measure of real-time sente...

Materials

Name	Company	Catalog Number	Comments
EyeLink 1000 plus eye tracker	SR Research Ltd.		The EyeLink 1000 plus allows remote eye tracking, without a head support. The eye tracker provides information about the participant’s point of gaze at a sampling rate of 500 Hz, and it has accuracy of 0.5 degrees of visual angle.