The authors acknowledge that the wider study from which this paper is generated is financially supported by the General Research Fund under the University Grants Council of Hong Kong Special Administration Region, China (grant number: GRF 844813); and by the Research Support Scheme 2017/18 of the Department of Special Education and Counselling at the Education University of Hong Kong.

Parental and participant consent was obtained during the recruitment process in a primary school and a children service center for ASD in Hong Kong and the study was approved by the university ethical review committee of the Education University of Hong Kong.
1. Use of a Video-based Assessment
<ol>
	<li>Produce several social videos, about one minute long, that consist of daily life scenarios involving several people in a social context (Figure 1).
	<ol>
		<li>P...

<ol>
	<li>American Psychiatric Association. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Diagnostic and Statistical Manual of Mental Disorders - 5th Edition (DSM-V). , (2013).</li><li>Williams, D. L., Goldstein, G., Minshew, N. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Neuropsychologic+functioning+in+children+with+autism:+further+evidence+of+disordered+complex+information+processing.">Neuropsychologic functioning in children with autism: further evidence of disordered complex information processing.</a> Child Neuropsychology. 12, 279-298 (2006).</li><li>Bons, D., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=emotional,+and+cognitive+empathy+in+children+and+adolescents+with+autism+spectrum+disorder+and+conduct+disorder.">emotional, and cognitive empathy in children and adolescents with autism spectrum disorder and conduct disorder.</a> Journal of Abnormal Child Psychology. 41, 425-443 (2013).</li><li>Van der Meer, J. M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Are+autism+spectrum+disorder+and+attention-deficit/hyperactivity+disorder+different+manifestations+of+one+overarching+disorder?+Cognitive+and+symptom+evidence+from+a+clinical+and+population-based+sample.">Are autism spectrum disorder and attention-deficit/hyperactivity disorder different manifestations of one overarching disorder? Cognitive and symptom evidence from a clinical and population-based sample.</a> Journal of the American Academy of Child and Adolescent Psychiatry. 51 (11), 1160-1172 (2012).</li><li>Brosnan, M. J., Scott, F. J., Fox, S., Pye, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Gestalt+processing+in+autism:+Failure+to+process+perceptual+relationships+and+the+implications+for+contextual+understanding.">Gestalt processing in autism: Failure to process perceptual relationships and the implications for contextual understanding.</a> Journal of Child Psychology and Psychiatry. 45 (3), 459-469 (2004).</li><li>Behrmann, M., Thomas, C., Humphreys, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Seeing+it+differently:+Visual+processing+in+autism.">Seeing it differently: Visual processing in autism.</a> Trends in Cognitive Sciences. 10 (6), 258-264 (2006).</li><li>Happ&#233;, F. G. E., Frith, U. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+weak+coherence+account:+Detail-focused+cognitive+style+in+autism+spectrum+disorders.">The weak coherence account: Detail-focused cognitive style in autism spectrum disorders.</a> Journal of Autism and Developmental Disorders. 36, 5-25 (2006).</li><li>Tsang, V. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Eye-tracking+study+on+facial+emotion+recognition+tasks+in+individuals+with+high-functioning+autism+spectrum+disorders.">Eye-tracking study on facial emotion recognition tasks in individuals with high-functioning autism spectrum disorders.</a> Autism. 22 (2), 161-170 (2018).</li><li>Renzi, C., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Featural+and+Configural+processing+of+faces+are+dissociated+in+the+dorsolateral+prefrontal+cortex:+A+TMS+study.">Featural and Configural processing of faces are dissociated in the dorsolateral prefrontal cortex: A TMS study.</a> NeuroImage. 74, 45-51 (2013).</li><li>Samson, F., Mottron, L., Soulieres, I., Zeffiro, T. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Enhanced+visual+functioning+in+autism:+An+ALE+meta-analysis.">Enhanced visual functioning in autism: An ALE meta-analysis.</a> Human Brain Mapping. 33 (7), 1553-1581 (2012).</li><li>Sasson, N. J., Turner-Brown, L. M., Holtzclaw, T. N., Lam, K. S. L., Bodfish, J. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Children+with+autism+demonstrate+circumscribed+attention+during+passive+viewing+of+complex+social+and+nonsocial+picture+arrays.">Children with autism demonstrate circumscribed attention during passive viewing of complex social and nonsocial picture arrays.</a> Autism Research. 1, 31-42 (2008).</li><li>Klin, A., Jones, W., Schultz, R., Volkmar, F., Cohen, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual+fixation+patterns+during+viewing+of+naturalistic+social+situations+as+predictors+of+social+competence+in+individuals+with+autism.">Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism.</a> Archives of General Psychiatry. 59, 809-816 (2002).</li><li>Rutherford, M. D., Towns, A. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Scan+path+differences+and+similarities+during+emotion+perception+in+those+with+and+without+autism+spectrum+disorders.">Scan path differences and similarities during emotion perception in those with and without autism spectrum disorders.</a> Journal of Autism and Developmental Disorders. 38 (7), 1371-1381 (2008).</li><li>Wallace, S., Coleman, M., Bailey, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+investigation+of+basic+facial+expression+recognition+in+autism+spectrum+disorders.">An investigation of basic facial expression recognition in autism spectrum disorders.</a> Cognition and Emotion. 22 (7), 1353-1380 (2008).</li><li>Byrge, L., Dubois, J., Tyszka, J. M., Adolphs, R., Kennedy, D. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Idiosyncratic+Brain+Activation+Patterns+Are+Associated+with+Poor+Social+Comprehension+in+Autism.">Idiosyncratic Brain Activation Patterns Are Associated with Poor Social Comprehension in Autism.</a> The Journal of Neuroscience. 35, 5837-5850 (2015).</li><li>Raven, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Manual for Raven&#39;s Progressive Matrices and Vocabulary Scales. Research Supplement No.1: The 1979 British Standardisation of the Standard Progressive Matrices and Mill Hill Vocabulary Scales. , (1981).</li><li>Sasson, N. J., Elison, J. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Eye+tracking+young+children+with+autism.">Eye tracking young children with autism.</a> Journal of Visualized Experiments. (61), e3675 (2012).</li></ol>

The authors have nothing to disclose.

The first-moment fixation duration was shorter for the ASD-ADHD and ASD children than for the neurotypical child. The total fixation duration was shorter for the ASD-ADHD child than for the neurotypical child, demonstrating a general reduction in visual attention to social stimuli. This showed that the ASD-ADHD child showed a delay in attending to the entry of social stimuli in a social scene. This delay might cause the child to skip registering important momentary social information, which may lead to the misinterpretat...

Persons with ASD are known to be characterized by behavioral deficits in social communication, based on conventional behavioral evidence from structured observational assessments and parent interviews. In addition, sensory processing abnormalities have been recently incorporated into the DSM-5 diagnostic criteria of ASD1. Social information processing involves the lower level sensory-perceptual processing and higher level social cognitive processing of social information. Sensory-perceptual processing refers to the ability to attend to social stimuli and encode them in a short-term memory bank for instant retrieval and response-planning, while social cognitive processing refers to the interpretation of social information by social reasoning and problem-solving2,3. As such, social information-processing deficits often lead to other psychobehavioral characteristics, such as social anxiety and inattentiveness. This can be illustrated by the high comorbid prevalence rate of ASD with attention deficit-hyperactive disorder (ADHD). The range of comorbidity for ADHD in ASD has been estimated at 30% to 80%, whereas the presence of comorbid ASD in ADHD has been estimated at 20% to 50%4.
Two major hypotheses have been put forward to account for the deficits in social information processing&#8212;namely, enhanced perceptual functioning (EPF) and weak central coherence (WCC). EPF refers to the overattentiveness to or preoccupation with specific parts by individuals with ASD, whereas WCC refers to their weakness to derive the essence of wholes by pulling together the interelement relationships of the parts5. Both theoretical frameworks attest to their failure to globally configure or process the multiple stimuli concurrently presented in a confined social context6,7. In an earlier face emotion recognition study using static face expression photos8, it was found that the ASD group tended to show localized processing of facial features (such as the shape of the mouth) using EPF, but seem to be weaker in configural processing, which demands pulling together the more abstract perceptual concepts as postulated by WCC, such as the spatial relationships between multiple facial components (e.g., the distance between the eyebrows and the intensity of the eye gaze)9,10.
Since daily social episodes consist of dynamic moment-to-moment subtle changes in social information, any failure to attend or engage in the sensory-perceptual processing of subtle human nonverbal cues, such as facial expression, postures, and gestures, and to make sense of the relationships of the different social stimuli might lead to inappropriate social cognitive processing. Eye-tracking experiments have been increasingly used to supplement clinical observation in social information processing studies. Eye-tracking data, in the form of scanpath patterns, visual fixation counts, and visual duration, have been major biomarkers to investigate social information processing in ASD11,12,13,14,15.
In this study, we illustrate the use of the eye-tracking technique to investigate whether the two participants with ASD and with ASD-ADHD process the first moments of social video scenes differently than the neurotypical child. The eye tracker equipment captures four major indices during viewing: the number of visual fixations, the first fixation duration, the total fixation duration, and the scanpath patterns in the form of spatial arrangement and sequence of fixation points. In this way, how fast each participant attends to the audio-visual stimuli predefined by AOIs as they first appear into the social scenes, for how long they continue to look at those AOIs, and their gaze shifts between multiple AOIs appearing concurrently in the same social scene can be captured. Any delay to fixate AOIs during the first moments (i.e., 500 ms) and the trajectory of the scanpaths provide important evidence for data analysis. Representative findings from the qualitative analysis of this single-case comparative study using this paradigm are reported.

<table border="0" cellpadding="0" cellspacing="0" style="width:741px;" width="741">
	<tbody>
		<tr height="21">
			<td height="21" style="height:21px;width:227px;">Tobii Pro TX300</td>
			<td style="width:79px;">Tobii</td>
			<td style="width:123px;">N/A</td>
			<td style="width:313px;">Screen based eye-tracker (300Hz refreshing rate)</td>
		</tr>
		<tr height="21">
			<td height="21" style="height:21px;width:227px;">Tobii Pro Studio</td>
			<td style="width:79px;">Tobii</td>
			<td style="width:123px;">N/A</td>
			<td>Software for analyzing eyetracking data</td>
		</tr>
	</tbody>
</table>

comparing eye-tracking data of children with high-functioning asd, comorbid adhd, and of a control watching social videos

Children with autism spectrum disorders (ASD) are known to have sensory-perceptual processing deficits that weaken their abilities to attend and perceive social stimuli in daily living contexts. Since daily social episodes consist of subtle dynamic changes in social information, any failure to attend to or process subtle human nonverbal cues, such as facial expression, postures, and gestures, might lead to inappropriate social interaction. Traditional behavioral rating scales or assessment tools based on static social scenes have limitations in capturing the moment-to-moment changes in social scenarios. An eye-tracking assessment, which can be administered in a video-based mode, is therefore preferred, to augment clinical observation. In this study, using the single-case comparison design, the eye-tracking data of three participants, a child with autism spectrum disorder (ASD), another with comorbid attention deficit-hyperactive disorder (ADHD), and a neurotypical control, are captured while they view a video of social scenarios. The eye-tracking experiment has helped answer the research question: How does social attention differ between the three participants? By predefining areas of interest (AOIs), their visual attention on relevant or irrelevant social stimuli, how fast each participant attends to the first social stimuli appearing in the videos, for how long each participant continues to attend to those stimuli within the AOIs, and the gaze shifts between multiple social stimuli appearing concurrently in the same social scene are captured, compared, and analyzed in a video-based eye-tracking experiment.

The eye-tracking data of the three Cantonese-speaking children (with ASD, with ASD-ADHD, and a control) aged between the ages of 7 and 9 viewing three social videos using the aforementioned paradigm is presented here (Table 1).
The first fixation duration (per 500 ms target AOI) was longer for the neurotypical child (150 ms) than for the ASD and ASD-ADHD children (both 110 ms). The total fixation duration (per 5...

Watch this Scientific Journal Video about Comparing Eye-tracking Data of Children with High-functioning ASD, Comorbid ADHD, and of a Control Watching Social Videos at JoVE.com

Comparing Eye-tracking Data of Children with High-functioning ASD, Comorbid ADHD, and of a Control Watching Social Videos

This is a qualitative comparative case study analysis of eye-tracking data on the first moments of social video scenes as viewed by three participants: one with autism spectrum disorder, one with comorbid attention deficit-hyperactive disorder, and one neurotypical control.

Children with autism spectrum disorders (ASD) are known to have sensory-perceptual processing deficits that weaken their abilities to attend and perceive ...

The eye-tracking method can help researchers in the field of special educational needs to explore the differences in visual attention performances for children with and without autism spectrum disorders, ASD, when they view videos of social scenarios. The main advantage of the eye-tracking technique is that the eye-gaze paths and vision fixation durations on various relevant social stimuli can be captured, quantified, and compared between children with and without autism. Visual demonstration of this method is critical.As the steps on choosing the target area of interest in the video for analysis, may be difficult to illustrate purely in written format. Begin, by escorting the child into the eye-tracking room and having them sit in front of the 23-inch color LCD monitor with the eye-tracker 60 centimeters away. Sit next to the child, and instruct them to look at the screen.Have another research investigator operate the eye-tracker from the other side of the partition. Have the child watch the calibration dots that set the viewing boundaries across the screen as it captures the eye movements using infrared corneal reflectance technology. After calibration, instruct the participants to view the social videos one after another, while eye movement data is captured during viewing.Provide them with three social scenario videos to watch in total. Once the experiment is complete, analyze the data by first choosing context-relevant targets, such as the face or hands in their initial 500 milliseconds of appearance in each scene of the video as areas of interest, or AOIs. Label the AOIs in the information box on the left panel.Upon the completion of the addition and selection of the AOIs in the current frame, move the cursors in the timeline bar at the bottom panel to the next frame. Then, manually adjust the location and boundary of the AOIs in each frame of the video in the presentation video software of the eye-tracker, as the target areas change each timeframe of the video due to the movement of the people or objects as the story of the social video develops. If some existing AOIs are present for 500 milliseconds in the current scene, or if they are not relevant in the new frame in the video, right-click these AOIs to deactivate them in the new frame.Then, to run a statistical analysis of the eye-tracking indices, first choose the media file for analysis. Click Analyze the Selected Media. Choose the descriptive statistics, and select the dependent measures in Metrics.Then, choose Recordings in Rows, and select AOI Media Summary in Columns. Pick Update to analyze the eye-tracking patterns. Lastly, create the scan path of a scene from the data by choosing Visualization and Gaze Plot in the software.Select the media and recordings in the left panel for visualization. In the bottom timeline, move the lower cursor to the beginning of the target scene, and move the upper cursor to the end of the target scene. Ensure Accumulate is chosen for the data field to show the accumulative scan path.Then, finally click Export and Visualization Image to save the scan path as a separate image file. Here, the colored ovals are the areas of interest. For example, the face, eyes, mouth, hands, mobile phone, and the bag of the lady, which show the first moments in one of the scenes in Video One.For a social scene, the blue dots trace the scan paths for the neuro-typical control child, and green dots for the ASD child, and the red dots for the ASD-ADHD child. The dots in the figure indicate the locations of visual fixations. The first fixation duration was longer for the neuro-typical child than for the ASD and ASD-ADHD children.The total fixation duration was shorter for the ASD-ADHD child, than for both the neuro-typical child and the ASD child. Further, the total number of fixation counts was the largest for the ASD child, second for the neuro-typical child, and the shortest for the ASD-ADHD child. While eye-tracking evidence confirms that children with ASD have difficulty in perceiving multiple social stimuli simultaneously and contextually, whereas children with comorbid ASD-ADHD tend to miss important momentary information, which leads them to misjudge social situations.

comparing-eye-tracking-data-children-with-high-functioning-asd

Research

JoVE Journal

Behavior

8.7K Views.  Education University of Hong Kong. The eye-tracking method can help researchers in the field of special educational needs to explore the differences in visual attention performances for children with and without autism spectrum disorders, ASD, when they view videos of social scenarios. The main advantage of the eye-tracking technique is that the eye-gaze paths and vision fixation durations on various relevant social stimuli can be captured, quantified, and compared between children with and without autism. Visual demonstration...