Name: gaze in action: head-mounted eye tracking of children's dynamic visual attention during naturalistic behavior
Uploaded: 2018-11-14T13:15:00
Description: Watch this Scientific Journal Video about Gaze in Action: Head-mounted Eye Tracking of Children's Dynamic Visual Attention During Naturalistic Behavior at JoVE.com

This research was funded by the National Institutes of Health grants R01HD074601 (C.Y.), T32HD007475-22 (J.I.B., D.H.A.), and F32HD093280 (L.K.S.); National Science Foundation grant BCS1523982 (L.B.S., C.Y.); and by Indiana University through the Emerging Area Research Initiative - Learning: Brains, Machines, and Children (L.B.S.). The authors thank the child and parent volunteers who participated in this research and who agreed to be used in the figures and filming of this protocol. We also appreciate the members of the Computational Cognition and Learning Laboratory, especially Sven Bambach, Anting Chen, Steven Elmlinger, Seth Foster, Grace Lisandrelli, and Charlene Tay, for their assistance in developing and honing this protocol.

This tutorial is based on a procedure for collecting head-mounted eye-tracking data with toddlers approved by the Institutional Review Board at Indiana University. Informed parental consent was obtained prior to toddlers&#39; participation in the experiment.
1. Preparation for the Study
<ol>
	<li>Eye-Tracking Equipment. Select one of the several head-mounted eye-tracking systems that are commercially available, either one marketed as specifically for children or modify the s...

<ol>
	<li>Tatler, B. W., Hayhoe, M. M., Land, M. F., Ballard, D. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Eye+guidance+in+natural+vision:+Reinterpreting+salience.">Eye guidance in natural vision: Reinterpreting salience.</a> Journal of Vision. 11 (5), 1-23 (2011).</li><li>Hayhoe, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Vision+using+routines:+A+functional+account+of+vision.">Vision using routines: A functional account of vision.</a> Visual Cognition. 7 (1-3), 43-64 (2000).</li><li>Land, M., Mennie, N., Rusted, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+Roles+of+Vision+and+Eye+Movements+in+the+Control+of+Activities+of+Daily+Living.">The Roles of Vision and Eye Movements in the Control of Activities of Daily Living.</a> Perception. 28 (11), 1311-1328 (1999).</li><li>Franchak, J. M., Kretch, K. S., Adolph, K. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=See+and+be+seen:+Infant-caregiver+social+looking+during+locomotor+free+play.">See and be seen: Infant-caregiver social looking during locomotor free play.</a> Developmental Science. 21 (4), 12626 (2018).</li><li>Franchak, J. M., Kretch, K. S., Soska, K. C., Adolph, K. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Head-mounted+eye+tracking:+a+new+method+to+describe+infant+looking.">Head-mounted eye tracking: a new method to describe infant looking.</a> Child Development. 82 (6), 1738-1750 (2011).</li><li>Kretch, K. S., Adolph, K. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+organization+of+exploratory+behaviors+in+infant+locomotor+planning.">The organization of exploratory behaviors in infant locomotor planning.</a> Developmental Science. 20 (4), 12421 (2017).</li><li>Yu, C., Smith, L. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Hand-Eye+Coordination+Predicts+Joint+Attention.">Hand-Eye Coordination Predicts Joint Attention.</a> Child Development. 88 (6), 2060-2078 (2017).</li><li>Yu, C., Smith, L. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Joint+Attention+without+Gaze+Following:+Human+Infants+and+Their+Parents+Coordinate+Visual+Attention+to+Objects+through+Eye-Hand+Coordination.">Joint Attention without Gaze Following: Human Infants and Their Parents Coordinate Visual Attention to Objects through Eye-Hand Coordination.</a> PLoS One. 8 (11), 79659 (2013).</li><li>Yu, C., Smith, L. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multiple+Sensory-Motor+Pathways+Lead+to+Coordinated+Visual+Attention.">Multiple Sensory-Motor Pathways Lead to Coordinated Visual Attention.</a> Cognitive Science. 41, 5-31 (2016).</li><li>Yu, C., Smith, L. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+Social+Origins+of+Sustained+Attention+in+One-Year-Old+Human+Infants.">The Social Origins of Sustained Attention in One-Year-Old Human Infants.</a> Current Biology. 26 (9), 1-6 (2016).</li><li>Kretch, K. S., Franchak, J. M., Adolph, K. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Crawling+and+walking+infants+see+the+world+differently.">Crawling and walking infants see the world differently.</a> Child Development. 85 (4), 1503-1518 (2014).</li><li>Yu, C., Suanda, S. H., Smith, L. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Infant+sustained+attention+but+not+joint+attention+to+objects+at+9+months+predicts+vocabulary+at+12+and+15+months.">Infant sustained attention but not joint attention to objects at 9 months predicts vocabulary at 12 and 15 months.</a> Developmental Science. , (2018).</li><li>Hennessey, C., Lawrence, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Noncontact+binocular+eye-gaze+tracking+for+point-of-gaze+estimation+in+three+dimensions.">Noncontact binocular eye-gaze tracking for point-of-gaze estimation in three dimensions.</a> IEEE Transactions on Biomedical Engineering. 56 (3), 790-799 (2009).</li><li>Elmadjian, C., Shukla, P., Tula, A. D., Morimoto, C. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=3D+gaze+estimation+in+the+scene+volume+with+a+head-mounted+eye+tracker.">3D gaze estimation in the scene volume with a head-mounted eye tracker.</a> Proceedings of the Workshop on Communication by Gaze Interaction. , 3 (2018).</li><li>Castellanos, I., Pisoni, D. B., Yu, C., Chen, C., Houston, D. M., Knoors, H., Marschark, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Embodied+cognition+in+prelingually+deaf+children+with+cochlear+implants:+Preliminary+findings.">Embodied cognition in prelingually deaf children with cochlear implants: Preliminary findings.</a> Educating Deaf Learners: New Perspectives. , (2018).</li><li>Kennedy, D. P., Lisandrelli, G., Shaffer, R., Pedapati, E., Erickson, C. A., Yu, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Face+Looking,+Eye+Contact,+and+Joint+Attention+during+Naturalistic+Toy+Play:+A+Dual+Head-Mounted+Eye+Tracking+Study+in+Young+Children+with+ASD.">Face Looking, Eye Contact, and Joint Attention during Naturalistic Toy Play: A Dual Head-Mounted Eye Tracking Study in Young Children with ASD.</a> Poster at the International Society for Autism Research Annual Meeting. , (2018).</li><li>Yurkovic, J. R., Lisandrelli, G., Shaffer, R., Pedapati, E., Erickson, C. A., Yu, C., Kennedy, D. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Using+Dual+Head-Mounted+Eye+Tracking+to+Index+Social+Responsiveness+in+Naturalistic+Parent-Child+Interaction.">Using Dual Head-Mounted Eye Tracking to Index Social Responsiveness in Naturalistic Parent-Child Interaction.</a> Talk at the International Congress for Infant Studies Biennial Congress. , (2018).</li><li>Holmqvist, K., Nystr&#246;m, M., Andersson, R., Dewhurst, R., Jarodzka, H., Van de Weijer, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Eye tracking: A comprehensive guide to methods and measures. , (2011).</li><li>Saez de Urabain, I. R., Johnson, M. H., Smith, T. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=GraFIX:+a+semiautomatic+approach+for+parsing+low-+and+high-quality+eye-tracking+data.">GraFIX: a semiautomatic approach for parsing low- and high-quality eye-tracking data.</a> Behavior Research Methods. 47 (1), 53-72 (2015).</li><li>Franchak, J. M., Hopkins, B., Geangu, E., Linkenauger, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Using+head-mounted+eye+tracking+to+study+development.">Using head-mounted eye tracking to study development.</a> The Cambridge Encyclopedia of Child Development 2nd ed. , 113-116 (2017).</li><li>Yonas, A., Arterberry, M. E., Granrud, C. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Four-month-old+infants'+sensitivity+to+binocular+and+kinetic+information+for+three-dimensional-object+shape.">Four-month-old infants' sensitivity to binocular and kinetic information for three-dimensional-object shape.</a> Child Development. 58 (4), 910-917 (1987).</li><li>Smith, T. J., Saez de Urabain, I. R., Hopkins, B., Geangu, E., Linkenauger, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Eye+tracking.">Eye tracking.</a> The Cambridge Encyclopedia of Child Development. , 97-101 (2017).</li></ol>

This protocol provides guiding principles and practical recommendations for implementing head-mounted eye tracking with infants and young children. This protocol was based on the study of natural toddler behaviors in the context of parent-toddler free play with toys in a laboratory setting. In-house eye-tracking equipment and software were used for calibration and data coding. Nevertheless, this protocol is intended to be generally applicable to researchers using a variety of head-mounted eye-tracking systems to study a ...

The last several decades have seen growing interest in studying the development of infant and toddler visual attention. This interest has stemmed in large part from the use of looking time measurements as a primary means to assess other cognitive functions in infancy and has evolved into the study of infant visual attention in its own right. Contemporary investigations of infant and toddler visual attention primarily measure eye movements during screen-based eye-tracking tasks. Infants sit in a chair or parent&#39;s lap in front of a screen while their eye movements are monitored during the presentation of static images or events. Such tasks, however, fail to capture the dynamic nature of natural visual attention and the means by which children&#39;s natural visual environments are generated - active exploration.
Infants and toddlers are active creatures, moving their hands, heads, eyes, and bodies to explore the objects, people, and spaces around them. Each new development in body morphology, motor skill, and behavior - crawling, walking, picking up objects, engaging with social partners - is accompanied by concomitant changes in the early visual environment. Because what infants do determines what they see, and what they see serves for what they do in visually guided action, studying the natural development of visual attention is best carried out in the context of natural behavior1.
Head-mounted eye trackers (ETs) have been invented and used for adults for decades2,3. Only recently have technological advances made head-mounted eye-tracking technology suitable for infants and toddlers. Participants are outfitted with two lightweight cameras on the head, a scene camera facing outward that captures the first person perspective of the participant and an eye camera facing inward that captures the eye image. A calibration procedure provides training data to an algorithm that maps as accurately as possible the changing positions of the pupil and corneal reflection (CR) in the eye image to the corresponding pixels in the scene image that were being visually attended. The goal of this method is to capture both the natural visual environments of infants and infants&#39; active visual exploration of those environments as infants move freely. Such data can help to answer questions not only about visual attention, but also about a broad range of perceptual, cognitive, and social developments4,5,6,7,8. The use of these techniques has transformed understandings of joint attention7,8,9, sustained attention10, changing visual experiences with age and motor development4,6,11, and the role of visual experiences in word learning12. The present paper provides guiding principles and practical recommendations for carrying out head-mounted eye-tracking experiments with infants and toddlers and illustrates the types of data that can be generated from head-mounted eye tracking in one natural context for toddlers: free-flowing toy play with a parent.

<table border="0" cellpadding="0" cellspacing="0" style="width:639px;" width="639">
	<tbody>
		<tr height="42">
			<td height="42" style="height:42px;width:216px;">Head-mounted eye tracker</td>
			<td style="width:71px;">Pupil Labs</td>
			<td style="width:352px;">World Camera and Eye Camera</td>
			<td></td>
		</tr>
	</tbody>
</table>

gaze in action: head-mounted eye tracking of children's dynamic visual attention during naturalistic behavior

Young children's visual environments are dynamic, changing moment-by-moment as children physically and visually explore spaces and objects and interact with people around them. Head-mounted eye tracking offers a unique opportunity to capture children's dynamic egocentric views and how they allocate visual attention within those views. This protocol provides guiding principles and practical recommendations for researchers using head-mounted eye trackers in both laboratory and more naturalistic settings. Head-mounted eye tracking complements other experimental methods by enhancing opportunities for data collection in more ecologically valid contexts through increased portability and freedom of head and body movements compared to screen-based eye tracking. This protocol can also be integrated with other technologies, such as motion tracking and heart-rate monitoring, to provide a high-density multimodal dataset for examining natural behavior, learning, and development than previously possible. This paper illustrates the types of data generated from head-mounted eye tracking in a study designed to investigate visual attention in one natural context for toddlers: free-flowing toy play with a parent. Successful use of this protocol will allow researchers to collect data that can be used to answer questions not only about visual attention, but also about a broad range of other perceptual, cognitive, and social skills and their development.

The method discussed here was applied to a free-flowing toy play context between toddlers and their parents. The study was designed to investigate natural visual attention in a cluttered environment. Dyads were instructed to play freely with a set of 24 toys for six minutes. Toddlers&#39; visual attention was measured by coding the onset and offset of looks to specific regions of interest (ROIs) -- each of the 24 toys and the parent&#39;s face -- and by analyzing the duration and proporti...

Watch this Scientific Journal Video about Gaze in Action: Head-mounted Eye Tracking of Children's Dynamic Visual Attention During Naturalistic Behavior at JoVE.com

Gaze in Action: Head-mounted Eye Tracking of Children&#039;s Dynamic Visual Attention During Naturalistic Behavior

Young children do not passively observe the world, but rather actively explore and engage with their environment. This protocol provides guiding principles and practical recommendations for using head-mounted eye trackers to record infants' and toddlers' dynamic visual environments and visual attention in the context of natural behavior.

Gaze in Action: Head-mounted Eye Tracking of Children's Dynamic Visual Attention During Naturalistic Behavior

Young children's visual environments are dynamic, changing moment-by-moment as children physically and visually explore spaces and objects and interact ...

This method can help researchers understand what the world looks like from a young child's perspective, as well as how they allocate their real attention within that view. Compared with screen-based eye tracking, which is widely used in behavioral science, head-mounted eye tracking allows us to monitor where children look during everyday activities like toy play and picture book reading. Demonstrating the procedure will be graduate students Catalina Suarez-Rivera and Yayun Zhang, and lab manager Daniel Pearcy.Before beginning an experiment, modify a system to work with a custom-made infant cap. Select a scene camera that is adjustable in terms of positioning, and has a wide enough angle to capture a field of view appropriate for addressing the research questions of interest. Select an eye camera that is adjustable in terms of positioning, and has an infrared LED positioned in such a way that the cornea of the child's eye will reflect this light.The eye tracking system should be as unobtrusive and lightweight as possible, to provide the greatest chance that young children will tolerate wearing the equipment. Then attach the scene and the eye cameras to a hook and loop strap that is affixed to the opposite side of a piece of hook and loop strap sewn onto a child-sized cap, to embed the system into the cap, and position the cameras so that they will be out of the center of the child's view. For eye-tracking data collection, have two researchers present:one to interact with, and to occupy the child, and one to place and position the eye tracking system.Fully engage the child in an activity that occupies the child's hands, so that the child does not reach up to move or grab the eye-tracking system, and place the eye-tracking system onto the child's head. Position the scene camera low on the forehead to best approximate the child's field of view, and center the scene camera view on what the child will be looking at during the study. To obtain high-quality gaze data, position the eye camera to detect both the pupil and corneal reflection with no cheek or eyelash occlusion throughout the eye's full range of motion.The trickiest part of this protocol is placing the equipment on the child's head and adjusting the cameras without upsetting the child. Speed, confidence, and practice are essential. Once the scene and eye images are as high-quality as can be obtained, draw the child's attention to different locations in their field of view to collect calibration data.Take care that the child's body positioning during the calibration matches the position that will be used during the study. When all the calibration points have been obtained, begin collecting the eye-tracking data. Taking note of any points at which the eye-tracking system gets bumped or misaligned, to allow recalibration as necessary, and to allow separate coding of the data before and after the misalignment.To calibrate the eye-tracking data at the end of the study, open an appropriate calibration software program, and adjust the thresholds of the various detection parameters within the calibration software to obtain a good eye image. During the first round of calibration, identify calibration points at moments when the child is clearly looking to a distinct point in the scene image, keeping in mind that these can be points intentionally created by the researcher during the data collection, or points from within the study, in which the point of gaze is easily identifiable, as long as the pupil is accurately detected for those frames. Create a series of calibration points to establish the mapping between the scene and the eye.If the eye-tracking system changed position at any time during the study, create separate calibrations for the portions before and after the change in position. To code the regions of interest, compile a list of all the regions of interest that should be coded based on the research questions, and use the child's eye image, scene image, and point of gaze track to determine which region of interest is being visually attended. Scroll through frames one by one to watch for moments of the pupil within the eye image as the primary cue that the region of interest may have changed.When a visible movement of the eye occurs, check whether the child was shifting the point of gaze to a new region of interest, or to no defined region of interest. Although the regions of interest are coded separately for each frame, use frames before and after the frame being analyzed to gain contextual information that may aid in determining the correct region of interest. Here, sample region of interest streams for two 18-month old children are shown.Each colored block represents continuous frames during which the child looked at a particular region of interest. Children showed individual differences in their selectivity for different subsets of toys, as evidenced by the differences in proportion of the interactions that each child spent looking at each of the toy regions of interest. Although the total proportion of time both children spent looking at all of the toys was somewhat similar, the proportions of time spent on individual toys varied greatly, both within and between subjects.Moreover, how these proportions of looking time were achieved also differed, with Child Two's mean look duration almost double that of Child One. Another property demonstrated by these data is that both children rarely looked to the faces of their parents during the session, and that when they did, each gaze duration was typically for less than one second. Researchers can place head-mounted eye trackers on children and their social partner simultaneously, as well as integrate this procedure with techniques such as motion-tracking and heart rate monitoring, to provide high-density, multi-modal datasets for answering a variety of questions.The use of these techniques has transformed our understanding of many topics in the developmental literature, including joint and sustained attention, changing visual experiences with age and motor development, and the role of visual experiences in word learning. This protocol has been successfully employed with clinical populations, including children with cochlear implants and children diagnosed with autism spectrum disorders.

gaze-action-head-mounted-eye-tracking-children-s-dynamic-visual

Research

JoVE Journal

Behavior

Corticospinal Excitability Modulation During Action Observation

This study used the transcranial magnetic stimulation/motor evoked potential (TMS/MEP) technique to pinpoint when the automatic tendency to mirror someone else's action becomes anticipatory simulation of a complementary act. TMS was delivered to the left primary motor cortex corresponding to the hand to induce the highest level of MEP activity from the abductor digiti minimi (ADM; the muscle serving little finger abduction) as well as the first dorsal interosseus (FDI; the muscle serving index finger flexion/extension) muscles. A neuronavigation system was used to maintain the position of the TMS coil, and electromyographic (EMG) activity was recorded from the right ADM and FDI muscles. Producing original data with regard to motor resonance, the combined TMS/MEP technique has taken research on the perception-action coupling mechanism a step further. Specifically, it has answered the questions of how and when observing another person's actions produces motor facilitation in an onlooker's corresponding muscles and in what way corticospinal excitability is modulated in social contexts.

Single-pulse transcranial magnetic stimulation over the primary motor cortex, neuronavigation, and registration of electromyographic activity of hand muscles were used in this study to explore corticospinal excitability while participants were observing action sequences.

This study used the transcranial magnetic stimulation/motor evoked potential (TMS/MEP) technique to pinpoint when the automatic tendency to mirror someone ...

fMRI Validation of fNIRS Measurements During a Naturalistic Task

We present a method to compare brain activity recorded with near-infrared spectroscopy (fNIRS) in a dance video game task to that recorded in a reduced version of the task using fMRI (functional magnetic resonance imaging). Recently, it has been shown that fNIRS can accurately record functional brain activities equivalent to those concurrently recorded with functional magnetic resonance imaging for classic psychophysical tasks and simple finger tapping paradigms. However, an often quoted benefit of fNIRS is that the technique allows for studying neural mechanisms of complex, naturalistic behaviors that are not possible using the constrained environment of fMRI. Our goal was to extend the findings of previous studies that have shown high correlation between concurrently recorded fNIRS and fMRI signals to compare neural recordings obtained in fMRI procedures to those separately obtained in naturalistic fNIRS experiments. Specifically, we developed a modified version of the dance video game Dance Dance Revolution (DDR) to be compatible with both fMRI and fNIRS imaging procedures. In this methodology we explain the modifications to the software and hardware for compatibility with each technique as well as the scanning and calibration procedures used to obtain representative results. The results of the study show a task-related increase in oxyhemoglobin in both modalities and demonstrate that it is possible to replicate the findings of fMRI using fNIRS in a naturalistic task. This technique represents a methodology to compare fMRI imaging paradigms which utilize a reduced-world environment to fNIRS in closer approximation to naturalistic, full-body activities and behaviors. Further development of this technique may apply to neurodegenerative diseases, such as Parkinson&#8217;s disease, late states of dementia, or those with magnetic susceptibility which are contraindicated for fMRI scanning.

We present a method to compare functional brain activity recorded during a naturalistic task using fNIRS with activity recorded during fMRI.

We present a method to compare brain activity recorded with near-infrared spectroscopy (fNIRS) in a dance video game task to that recorded in a reduced ...

Methods to Test Visual Attention Online

Online data collection methods have particular appeal to behavioral scientists because they offer the promise of much larger and much more representative data samples than can typically be collected on college campuses. However, before such methods can be widely adopted, a number of technological challenges must be overcome &#8211; in particular in experiments where tight control over stimulus properties is necessary. Here we present methods for collecting performance data on two tests of visual attention. Both tests require control over the visual angle of the stimuli (which in turn requires knowledge of the viewing distance, monitor size, screen resolution, etc.) and the timing of the stimuli (as the tests involve either briefly flashed stimuli or stimuli that move at specific rates). Data collected on these tests from over 1,700 online participants were consistent with data collected in laboratory-based versions of the exact same tests. These results suggest that with proper care, timing/stimulus size dependent tasks can be deployed in web-based settings.

To replicate laboratory settings, online data collection methods for visual tasks require tight control over stimulus presentation. We outline methods for the use of a web application to collect performance data on two tests of visual attention.

Online data collection methods have particular appeal to behavioral scientists because they offer the promise of much larger and much more representative ...

A Method to Quantify Visual Information Processing in Children Using Eye Tracking

Visual problems that occur early in life can have major impact on a child's development. Without verbal communication and only based on observational methods, it is difficult to make a quantitative assessment of a child's visual problems. This limits accurate diagnostics in children under the age of 4 years and in children with intellectual disabilities. Here we describe a quantitative method that overcomes these problems. The method uses a remote eye tracker and a four choice preferential looking paradigm to measure eye movement responses to different visual stimuli. The child sits without head support in front of a monitor with integrated infrared cameras. In one of four monitor quadrants a visual stimulus is presented. Each stimulus has a specific visual modality with respect to the background, e.g., form, motion, contrast or color. From the reflexive eye movement responses to these specific visual modalities, output parameters such as reaction times, fixation accuracy and fixation duration are calculated to quantify a child's viewing behavior. With this approach, the quality of visual information processing can be assessed without the use of communication. By comparing results with reference values obtained in typically developing children from 0-12 years, the method provides a characterization of visual information processing in visually impaired children. The quantitative information provided by this method can be advantageous for the field of clinical visual assessment and rehabilitation in multiple ways. The parameter values provide a good basis to: (i) characterize early visual capacities and consequently to enable early interventions; (ii) compare risk groups and follow visual development over time; and (iii), construct an individual visual profile for each child.

A method is described to quantify the quality of visual information processing based on reflexive eye movements in response to specific visual modalities. Reaction times and fixation output parameters are used to characterize visual performance in children with and without visual impairments from 6 months of age.

Visual problems that occur early in life can have major impact on a child's development. Without verbal communication and only based on observational ...

Central and Divided Visual Field Presentation of Emotional Images to Measure Hemispheric Differences in Motivated Attention

Two dominant theories on lateralized processing of emotional information exist in the literature. One theory posits that unpleasant emotions are processed by right frontal regions, while pleasant emotions are processed by left frontal regions. The other theory posits that the right hemisphere is more specialized for the processing of emotional information overall, particularly in posterior regions.
Assessing the different roles of the cerebral hemispheres in processing emotional information can be difficult without the use of neuroimaging methodologies, which are not accessible or affordable to all scientists. Divided visual field presentation of stimuli can allow for the investigation of lateralized processing of information without the use of neuroimaging technology. 
This study compared central versus divided visual field presentations of emotional images to assess differences in motivated attention between the two hemispheres. The late positive potential (LPP) was recorded using electroencephalography (EEG) and event-related potentials (ERPs) methodologies to assess motivated attention. Future work will pair this paradigm with a more active behavioral task to explore the behavioral impacts on the attentional differences found.

This study compared central versus divided visual field presentations of emotional images to assess differences in motivated attention between the two hemispheres. The late positive potential (LPP) was recorded using electroencephalography (EEG) and event-related potentials (ERPs) methodologies to assess motivated attention.

Two dominant theories on lateralized processing of emotional information exist in the literature. One theory posits that unpleasant emotions are processed ...

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

The present work is a description and an assessment of a methodology designed to quantify different aspects of the interaction between language processing and the perception of the visual world. The recording of eye-gaze patterns has provided good evidence for the contribution of both the visual context and linguistic/world knowledge to language comprehension. Initial research assessed object-context effects to test&#160;theories of modularity in language processing. In the introduction, we describe how subsequent investigations have taken the role of the wider visual context in language processing as a research topic in its own right, asking questions such as how our visual perception of events and of speakers contributes to&#160;comprehension informed by&#160;comprehenders&#39; experience. Among the examined aspects of the visual context are actions, events, a speaker&#39;s gaze, and emotional facial expressions, as well as spatial object configurations. Following an overview of the eye-tracking method and its different applications, we list the key steps of the methodology in the protocol, illustrating how to successfully use it to study visually-situated language comprehension. A final section presents three sets of representative results and illustrates the benefits and limitations of eye tracking for investigating the interplay between the perception of the visual world and language comprehension.

The present article reviews an eye-tracking methodology for studies on language comprehension. To obtain reliable data, key steps of the protocol must be followed. Among these are the correct set-up of the eye tracker (e.g., ensuring good quality of the eye and head images) and accurate calibration.

The present work is a description and an assessment of a methodology designed to quantify different aspects of the interaction between language processing ...

Eye-tracking to Distinguish Comprehension-based and Oculomotor-based Regressive Eye Movements During Reading

Regressive eye movements are eye movements that move backwards through the text and comprise approximately 10-25% of eye movements during reading. As such, understanding the causes and mechanisms of regressions plays an important role in understanding eye movement behavior. Inhibition of return (IOR) is an oculomotor effect that results in increased latency to return attention to a previously attended target versus a target that was not previously attended. Thus, IOR may affect regressions. This paper describes how to design materials to distinguish between regressions caused by comprehension-related and oculomotor processes; the latter is subject to IOR. The method allows researchers to identify IOR and control the causes of regressions. While the method requires tightly controlled materials and large numbers of participants and materials, it allows researchers to distinguish and control the types of regressions that occur in their reading studies.

The method was designed to investigate the role of inhibition of return (IOR) in regressive eye movements during reading. The focus is on differentiating between regressions triggered as a result of comprehension difficulty versus those triggered from oculomotor error, including the role of IOR in the two types of regressions.

Regressive eye movements are eye movements that move backwards through the text and comprise approximately 10-25% of eye movements during reading. As ...

Simultaneous Eye Tracking and Single-Neuron Recordings in Human Epilepsy Patients

Intracranial recordings from patients with intractable epilepsy provide a unique opportunity to study the activity of individual human neurons during active behavior. An important tool for quantifying behavior is eye tracking, which is an indispensable tool for studying visual attention. However, eye tracking is challenging to use concurrently with invasive electrophysiology and this approach has consequently been little used. Here, we present a proven experimental protocol to conduct single-neuron recordings with simultaneous eye tracking in humans. We describe how the systems are connected and the optimal settings to record neurons and eye movements. To illustrate the utility of this method, we summarize results that were made possible by this setup. This data shows how using eye tracking in a memory-guided visual search task allowed us to describe a new class of neurons called target neurons, whose response was reflective of top-down attention to the current search target. Lastly, we discuss the significance and solutions to potential problems of this setup. Together, our protocol and results suggest that single-neuron recordings with simultaneous eye tracking in humans are an effective method to study human brain function. It provides a key missing link between animal neurophysiology and human cognitive neuroscience.

We describe a method to conduct single-neuron recordings with simultaneous eye tracking in humans. We demonstrate the utility of this method and illustrate how we used this approach to obtain neurons in the human medial temporal lobe that encode targets of a visual search.

Intracranial recordings from patients with intractable epilepsy provide a unique opportunity to study the activity of individual human neurons during ...

Investigating the Deployment of Visual Attention Before Accurate and Averaging Saccades via Eye Tracking and Assessment of Visual Sensitivity

This experimental protocol was designed to investigate whether visual attention is obligatorily deployed at the endpoint of saccades. To this end, we recorded the eye position of human participants engaged in a saccade task via eye tracking and assessed visual orientation discrimination performance at various locations during saccade preparation. Importantly, instead of using a single saccade target paradigm for which the saccade endpoint typically coincides roughly with the target, this protocol comprised the presentation of two nearby saccade targets, leading to a distinct spatial dissociation between target locations and saccade endpoint on a substantial number of trials. The paradigm allowed us to compare presaccadic visual discrimination performance at the endpoint of accurate saccades (landing at one of the saccade targets) and of averaging saccades (landing at an intermediate location in between the two targets). We observed a selective enhancement of visual sensitivity at the endpoint of accurate saccades but not at the endpoint of averaging saccades. Rather, before the execution of averaging saccades, visual sensitivity was equally enhanced at both targets, suggesting that saccade averaging follows from unresolved attentional selection among the saccade targets. These results argue against a mandatory coupling between visual attention and saccade programming based on a direct measure of presaccadic visual sensitivity rather than saccadic reaction times, which have been used in other protocols to draw similar conclusions. While our protocol provides a useful framework to investigate the relationship between visual attention and saccadic eye movements at the behavioral level, it can also be combined with electrophysiological measures to extend insights at the neuronal level.

This experimental protocol combined eye tracking and the assessment of presaccadic visual sensitivity in a dual task paradigm, consisting of a free choice saccade task and a visual discrimination task, to investigate the deployment of visual spatial attention before both, accurate and averaging saccades.

This experimental protocol was designed to investigate whether visual attention is obligatorily deployed at the endpoint of saccades. To this end, we ...

Exploring Infant Sensitivity to Visual Language using Eye Tracking and the Preferential Looking Paradigm

We discuss the use of the preferential looking paradigm in eye tracking studies in order to study how infants develop, understand, and attend to the world around them. Eye tracking is a safe and non-invasive way to collect gaze data from infants, and the preferential looking paradigm is simple to design and only requires the infant to be attending to the screen. By simultaneously showing two visual stimuli that differ in one dimension, we can assess whether infants show different looking behavior for either stimulus, thus demonstrating sensitivity to that difference. The challenges in such experimental approaches are that experiments must be kept brief (no more than 10 min) and be carefully controlled such that the two stimuli differ in only one way. The interpretation of null results must also be carefully considered. In this paper, we illustrate a successful example of an infant eye tracking study with a preferential looking paradigm to discover that 6-month-olds are sensitive to linguistic cues in a signed language despite having no prior exposure to signed language, suggesting that infants possess intrinsic or innate sensitivities to these cues.

Eye tracking studies using a preferential looking paradigm can be used to study infants&#39;&#160;emerging understanding of, and attention to, their external visual world.

We discuss the use of the preferential looking paradigm in eye tracking studies in order to study how infants develop, understand, and attend to the world ...