Name: defining the role of language in infants' object categorization with eye-tracking paradigms
Uploaded: 2019-02-08T15:00:00
Description: Watch this Scientific Journal Video about Defining the Role Of Language in Infants' Object Categorization with Eye-tracking Paradigms at JoVE.com

The research reported here was supported by the National Institute of Child Health and Human Development of the National Institutes of Health under award number R01HD083310 and a National Science Foundation Graduate Research Fellowship under grant no. DGE&#8208;1324585. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the National Science Foundation.

All methods described here have been approved by the Northwestern University Institutional Review Board.
1. Stimuli Creation
NOTE: The visual stimuli (see Figure 1) used in the representative design reported below were originally developed in Havy and Waxman (2016)18 and are available for download at https://osf.io/n6uy8/.
<ol>
	<li>To create a new continuous category, first design a pair of novel digital image...

<ol>
	<li>Eimas, P. D., Quinn, P. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Studies+on+the+Formation+of+Perceptually+Based+Basic-Level+Categories+in+Young+Infants.">Studies on the Formation of Perceptually Based Basic-Level Categories in Young Infants.</a> Child Development. 65 (3), 903-917 (1994).</li><li>Madole, K. L., Oakes, L. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Making+sense+of+infant+categorization:+Stable+processes+and+changing+representations.">Making sense of infant categorization: Stable processes and changing representations.</a> Developmental Review. 19 (2), 263-296 (1999).</li><li>Gelman, S. A., Markman, E. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Categories+and+induction+in+young+children.">Categories and induction in young children.</a> Cognition. 23 (3), 183-209 (1986).</li><li>Ferry, A. L., Hespos, S. J., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Categorization+in+3-+and+4-month-old+infants:+An+advantage+of+words+over+tones.">Categorization in 3- and 4-month-old infants: An advantage of words over tones.</a> Child development. 81 (2), 472-479 (2010).</li><li>Fulkerson, A. L., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Words+(but+not+Tones)+Facilitate+Object+Categorization:+Evidence+From+6-+and+12-Month-Olds.">Words (but not Tones) Facilitate Object Categorization: Evidence From 6- and 12-Month-Olds.</a> Cognition. 105 (1), 218-228 (2007).</li><li>Balaban, M. T., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Do+words+facilitate+object+categorization+in+9-month-old+infants?.">Do words facilitate object categorization in 9-month-old infants?.</a> Journal of Experimental Child Psychology. 64 (1), 3-26 (1997).</li><li>Waxman, S. R., Braun, I. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Consistent+(but+not+variable)+names+as+invitations+to+form+object+categories:+New+evidence+from+12-month-old+infants.">Consistent (but not variable) names as invitations to form object categories: New evidence from 12-month-old infants.</a> Cognition. 95 (3), B59-B68 (2005).</li><li>Balaban, M. T., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+examination+of+the+factors+underlying+the+facilitative+effect+of+word+phrases+on+object+categorization+in+9-month-old+infants.">An examination of the factors underlying the facilitative effect of word phrases on object categorization in 9-month-old infants.</a> Proceedings of the 20th Boston University Conference on Language Development. 1, 483-493 (1996).</li><li>Waxman, S. R., Markow, D. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Words+as+invitations+to+form+categories:+evidence+from+12-+to+13-month-old+infants.">Words as invitations to form categories: evidence from 12- to 13-month-old infants.</a> Cognitive Psychology. 29 (3), 257-302 (1995).</li><li>Zhu, X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Semi-supervised learning literature survey. , (2005).</li><li>Chapelle, O., Scholkopf, B., Zien, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Semi-supervised learning: Adaptive computation and machine learning. , (2006).</li><li>Zhu, X., Goldberg, A. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Introduction+to+semi-supervised+learning.">Introduction to semi-supervised learning.</a> Synthesis lectures on artificial intelligence and machine learning. 3 (1), 1-130 (2009).</li><li>Ferry, A. L., Hespos, S. J., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Nonhuman+primate+vocalizations+support+categorization+in+very+young+human+infants.">Nonhuman primate vocalizations support categorization in very young human infants.</a> Proceedings of the National Academy of Sciences of the United States of America. 110 (38), 15231-15235 (2013).</li><li>Hunter, M. A., Ames, E. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+multifactor+model+of+infant+preferences+for+novel+and+familiar+stimuli.">A multifactor model of infant preferences for novel and familiar stimuli.</a> Advances in infancy research. , (1988).</li><li>Rose, S. A., Feldman, J. F., Jankowski, J. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Infant+visual+recognition+memory.">Infant visual recognition memory.</a> Developmental Review. 24 (1), 74-100 (2004).</li><li>Wetherford, M. J., Cohen, L. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Developmental+changes+in+infant+visual+preferences+for+novelty+and+familiarity.">Developmental changes in infant visual preferences for novelty and familiarity.</a> Child Development. , 416-424 (1973).</li><li>Perone, S., Spencer, J. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Autonomous+visual+exploration+creates+developmental+change+in+familiarity+and+novelty+seeking+behaviors.">Autonomous visual exploration creates developmental change in familiarity and novelty seeking behaviors.</a> Frontiers in psychology. 4, 648 (2013).</li><li>Havy, M., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Naming+influences+9-month-olds&#8217;+identification+of+discrete+categories+along+a+perceptual+continuum.">Naming influences 9-month-olds&#8217; identification of discrete categories along a perceptual continuum.</a> Cognition. 156, 41-51 (2016).</li><li>Althaus, N., Plunkett, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Timing+matters:+The+impact+of+label+synchrony+on+infant+categorisation.">Timing matters: The impact of label synchrony on infant categorisation.</a> Cognition. 139, 1-9 (2015).</li><li>Maris, E., Oostenveld, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Nonparametric+statistical+testing+of+EEG-+and+MEG-data.">Nonparametric statistical testing of EEG- and MEG-data.</a> Journal of Neuroscience Methods. 164 (1), 177-190 (2007).</li><li>Raudenbush, S. W., Bryk, A. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Hierarchical+Linear+Models:+Applications+and+Data+Analysis+Methods.">Hierarchical Linear Models: Applications and Data Analysis Methods.</a> SAGE. , (2002).</li><li>LaTourrette, A., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+little+labeling+goes+a+long+way:+Semi-supervised+learning+in+infancy.">A little labeling goes a long way: Semi-supervised learning in infancy.</a> Developmental Science. , e12736 (2018).</li><li>Dink, J., Ferguson, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> eyetrackingR: An R library for eyetracking data analysis. , (2015).</li><li>Kalish, C. W., Zhu, X., Rogers, T. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Drift+in+children's+categories:+When+experienced+distributions+conflict+with+prior+learning.">Drift in children's categories: When experienced distributions conflict with prior learning.</a> Developmental Science. 18 (6), 940-956 (2015).</li><li>Gibson, B. R., Rogers, T. T., Zhu, X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Human+semi-supervised+learning.">Human semi-supervised learning.</a> Topics in Cognitive Science. 5 (1), 132-172 (2013).</li><li>Keates, J., Graham, S. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Category+Markers+or+Attributes+Why+Do+Labels+Guide+Infants'+Inductive+Inferences?.">Category Markers or Attributes Why Do Labels Guide Infants' Inductive Inferences?.</a> Psychological Science. 19 (12), 1287-1293 (2008).</li><li>Booth, A. E., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+horse+of+a+different+color:+Specifying+with+precision+infants&#8217;+mappings+of+novel+nouns+and+adjectives.">A horse of a different color: Specifying with precision infants&#8217; mappings of novel nouns and adjectives.</a> Child development. 80 (1), 15-22 (2009).</li><li>Perszyk, D. R., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Listening+to+the+calls+of+the+wild:+The+role+of+experience+in+linking+language+and+cognition+in+young+infants.">Listening to the calls of the wild: The role of experience in linking language and cognition in young infants.</a> Cognition. 153, 175-181 (2016).</li><li>Althaus, N., Mareschal, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Labels+direct+infants&#8217;+attention+to+commonalities+during+novel+category+learning.">Labels direct infants&#8217; attention to commonalities during novel category learning.</a> PLoS ONE. 9 (7), e99670 (2014).</li><li>Fulkerson, A. L., Haaf, R. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+influence+of+labels,+non-labeling+sounds,+and+source+of+auditory+input+on+9-+and+15-month-olds&#8217;+object+categorization.">The influence of labels, non-labeling sounds, and source of auditory input on 9- and 15-month-olds&#8217; object categorization.</a> Infancy. 4 (3), 349-369 (2003).</li></ol>

Here, we present a procedure for evaluating the role of labeling in categorization. By presenting 2-year-olds with a realistic mix of labeled and unlabeled exemplars, we demonstrate that very young children are capable of learning in semi-supervised environments, extending work with adults and older children24,25. Thus, this method offers a resolution to the paradox posed above: if even a few labeled exemplars can spark category learning, then labels can be both ...

Categorization is a fundamental building block of human cognition: infants&#8217; categorization abilities emerge early in infancy and become increasingly sophisticated with age.1,2,3 Research has also revealed a powerful role for&#160;language in infant categorization: from 3 months of age, infants learn categories more successfully when category exemplars are paired with language.4,5,6 Moreover, by the end of the first year, infants are attuned to the role of count noun labels in categorization. Pairing category exemplars with a consistent labeling phrase (&#8220;This is a vep!&#8221;) facilitates infants&#8217; category learning relative to providing either a distinct label for each exemplar (&#8220;This is a vep,&#8221; &#8220;This is a dax,&#8221; etc.) or a non-labeling phrase (&#8220;Look at this.&#8221;).7,8,9
In infants&#8217; everyday experiences, however, the vast majority of objects they encounter will likely remain unlabeled. No caregiver could label every object an infant&#160;sees much less provide the labels which apply to every object (e.g., &#8220;malamute,&#8221; &#8220;dog,&#8221; &#8220;pet,&#8221; &#8220;animal&#8221;). This presents a paradox: how can we reconcile the power of labels in infant categorization with their relative scarcity in infants&#8217; daily lives?
To answer this question, we developed a protocol to assess how infants learn categories in a variety of different learning environments, including when they receive a mixture of labeled and unlabeled exemplars. Specifically, we propose that receiving even a few labeled exemplars at the beginning of learning can facilitate categorization&#8212;by enhancing infants&#8217; ability to learn from subsequent, unlabeled exemplars as well. This strategy of using a small number of labeled exemplars as a foundation for learning from a larger number of unlabeled exemplars has been widely implemented in the field of machine learning, spawning a family of semi-supervised learning (SSL) algorithms10,11,12. Of course, the learning strategies implemented are not identical across different kinds of learners: in machine learning, algorithms typically are exposed to many more exemplars, make explicit guesses about each exemplar, and learn multiple categories simultaneously. Nevertheless, both machine and infant learners may benefit from successfully integrating both labeled and unlabeled exemplars&#160;to learn new categories in sparse labeling environments.
Our design focuses on whether 2-year-old children, in the process of acquiring words for numerous new categories, are capable of this kind of semi-supervised learning. We employ a standard infant categorization measure: a familiarization-test task. In this paradigm, 2-year-olds were exposed to a series of exemplars from a novel category during a familiarization phase. Each exemplar was paired with a different auditory stimulus, depending on the condition (i.e., either a labeling or a non-labeling phrase). Then, at the test, all 2-year-olds saw two new objects presented in silence: one object from the now-familiar category and one from a novel category.
If the 2-year-olds successfully form the category during the familiarization phase, then they should distinguish between the two exemplars&#160;presented at the test. Importantly, because a systematic preference for either the novel or familiar test image reflects an ability to distinguish between them, both familiarity and novelty preferences are interpreted as evidence of successful categorization. Note that on a given task, the nature of this preference is a function of infants&#8217; processing efficiency for the stimulus materials, with familiarity preferences associated with less efficient stimulus processing 4,13,14,15,16,17. Presenting the test phase in silence makes it possible to directly assess infants&#8217; success in object categorization and how this success varies according to the information that accompanied the exemplars during familiarization. Thus, this paradigm provides a compelling test of how different types of linguistic environments affect category learning. If labeling enhances category learning in both semi-supervised and fully supervised environments, then 2-year-olds in these conditions should show stronger test preferences than infants in other environments.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Final Cut Pro X</td>
			<td>Apple</td>
			<td>N/A</td>
			<td>Video editing, composition software</td>
		</tr>
		<tr>
			<td>MorphX</td>
			<td>Norrkross</td>
			<td>N/A</td>
			<td>Image-morphing software</td>
		</tr>
		<tr>
			<td>PhotoShop</td>
			<td>Adobe</td>
			<td>N/A</td>
			<td>Image-editing software</td>
		</tr>
		<tr>
			<td>R</td>
			<td>R Core Team</td>
			<td>N/A</td>
			<td>Statistical analysis software</td>
		</tr>
		<tr>
			<td>T60XL Eyetracker</td>
			<td>Tobii Pro</td>
			<td>Discontinued</td>
			<td>Large, arm-mounted eyetracker suitable for work with infants and children</td>
		</tr>
		<tr>
			<td>Tobii Pro Studio</td>
			<td>Tobii Pro</td>
			<td>N/A</td>
			<td>Software directing eyetracker display, data collection</td>
		</tr>
	</tbody>
</table>

defining the role of language in infants' object categorization with eye-tracking paradigms

Assessing infant category learning is a challenging but vital aspect of studying infant cognition. By employing a familiarization-test paradigm, we straightforwardly measure infants&#8217; success in learning a novel category while relying only on their looking behavior. Moreover, the paradigm can directly measure the impact of different auditory signals on the infant categorization across a range of ages. For instance, we assessed how 2-year-olds learn categories in a variety of labeling environments: in our task, 2-year-olds successfully learned categories when all exemplars were labeled&#160;or the first two exemplars were labeled, but they failed to categorize when no exemplars were labeled or only the final two exemplars were labeled. To determine infants&#8217; success in such tasks, researchers can examine both the overall preference displayed by infants in each condition and infants&#8217; pattern of looking over the course of the test phase, using an eye-tracker to provide fine-grained time-course data. Thus, we present a powerful paradigm for identifying the role of language, or any auditory signal, in infants&#8217; object category learning.

Using the protocol above, we ran two experiments22. Analyses were conducted with the eyetrackingR package23, and the data and code are available at https://github.com/sandylat/ssl-in-infancy. In the first experiment, we contrasted a fully supervised condition (n = 24, Mage = 26.8 mo), featuring only labeled exemplars, with an unsupervised condition (n = 24, Mage = 26.9 mo), featuring only unlabeled ex...

Watch this Scientific Journal Video about Defining the Role Of Language in Infants' Object Categorization with Eye-tracking Paradigms at JoVE.com

Defining the Role Of Language in Infants' Object Categorization with Eye-tracking Paradigms

Here we present a protocol for familiarization-test paradigms which provide a direct test of infant categorization and help to define the role of language in early category learning.

Assessing infant category learning is a challenging but vital aspect of studying infant cognition. By employing a familiarization-test paradigm, we ...

This familiarization test method reveals how infants form visual object categories and what kinds of conditions most powerfully support their learning. Using only infants eye movements we can measure how successfully they learn a category. Then we can directly compare that learning across a huge range of different learning conditions.Begin by creating the visual stimuli for the experiment. To create a new continuous category, first design a pair of novel digital images. Then, morph the pair of images together to form a continuum of exemplars between the two original images.Create at least two categories in this way, so that one can serve as the category to be learned, while the other provides the novel category exemplar for the test trial. Next, select the familiarization exemplars at evenly spaced intervals from across each learned categories continuum. Select an appropriate number of exemplars and commensurate with the difficulty of the category and age of the participants.To create the exemplars for the test phase, select the midpoints of the familiar categories continuum and the novel categories continuum. Then match the color of the novel exemplar to that of the familiar exemplar using an image manipulation program. For both the labeling and non-labeling phrases use previously recorded auditory stimuli produced by a female native English speaker in a sound proof booth.To collect adequate data for a familiarization test measure, most widely available eye trackers will suffice. The objects occupy large portions of the screen and the data analysis investigates performance over a long window. Using the eye tracker software, create four different conditions, fully supervised, unsupervised, semi-supervised and reversed semi-supervised.Ensure these conditions are separate, so that each infant will see only one condition. Next, generate at least two pseudorandom orders of the learning exemplars with the constraint that no more than two exemplars from the same side of the continuum can be shown consecutively. Now use video editing software to create familiarization videos that pair the previously created auditory stimuli with the visual stimuli as appropriate for each condition.For instance, in the fully supervised condition pair each familiarization exemplar with a labeling phrase. In the unsupervised condition, pair each familiarization exemplar with a non-labeling phrase. In the semi-supervised condition, pair only the first two exemplars in each order with labeling phrases, but the rest with non-labeling phrases.For the reversed semi-supervised condition, pair the final two exemplars with labeling phrases but the first four with non-labeling phrases. Upload these videos into the eye tracker software being sure to order the familiarization videos as determined by the pseudorandomized order. Also upload a short attention grabbing animation displayed in the center of the screen after familiarization, to ensure that most infants look at the center of the screen when the test phase begins.Finally, for each learning category, design two test trials each featuring two exemplars displayed side by side. For both, ensure that one exemplar represents the midpoint of the now familiar category, while the other represents the midpoint of the novel category. Ensure that test trials last 5 to 20 seconds to accumulate sufficient looking, and counterbalance the trials so that the left right positioning of the novel exemplar in the test trial is reversed across the videos.Upload the task trials to the eye tracker software, positioning them after the post familiarization attention getter. Counterbalance these trials presentation, so each infant has an equal chance of seeing a left novel or right novel test trial. Before the infant arrives, setup the eye tracker.Randomly assign the infant to a condition and an order, then open the eye tracker software, and select the assigned condition order pair, and enter the participant number for the recording. After obtaining consent for the study, bring the infant and the caregiver to the eye tracking room. Ensure this room is moderately lit without any distracting decorations on the walls.Seat the caregiver at an appropriate distance for the model of eye tracker being used. If the infant does not wish to sit in the caregivers lap, they may sit on their own or they may sit in a car seat. If the infant is sitting on the caregivers lap, instruct the caregiver not to bias infants behavior in any way, but to try to keep the infant centered on the caregivers lap.Then, provide the caregiver with a pair of blacked out sunglasses to wear, so they cannot see the stimuli. Now ask the infant to look at the eye tracker screen. Consider displaying an engaging image or video to attract their attention.The screen should be positioned, so that the infant eyes are within the calibration window. Next, perform the eye trackers calibration procedure using a five point calibration, if possible. Infants often respond better when the calibration image is animation with auditory accompaniment.If the infant passes calibration, then begin the experiment if not, recalibrate until they are successful. Any infants who cannot be calibrated should be excluded from the study. This figure shows mean preference scores across conditions.Infants in the fully supervised and semi-supervised conditions, displayed novelty preferences significantly above chance. While infants in the unsupervised and reversed semi-supervised conditions, performed at chance levels. Here we see the infants looking patterns during testing.In the fully supervised and unsupervised conditions and in the semi-supervised and reversed semi-supervised conditions, infants pattern of looking to the exemplars diverged between 3450 and 3850 milliseconds. The colored shaded regions indicates standard error of the mean. This procedure can be used with infants of various ages and materials that vary in complexity.As a result, each new design must reflect infants learning capacity at that intended age. We can go on to ask, whether and how infants capacity as measured here, correlates with or even predicts infants acquisition of later language milestones. Such as vocabulary size or word order.This technique has allowed developmental scientist to study cognition and perception, in infants as young as two months. It continues to illuminate the ways that language facilitates learning throughout infancy.

defining-role-language-infants-object-categorization-with-eye

Research

JoVE Journal

Behavior

A Method to Quantify Visual Information Processing in Children Using Eye Tracking

Visual problems that occur early in life can have major impact on a child's development. Without verbal communication and only based on observational methods, it is difficult to make a quantitative assessment of a child's visual problems. This limits accurate diagnostics in children under the age of 4 years and in children with intellectual disabilities. Here we describe a quantitative method that overcomes these problems. The method uses a remote eye tracker and a four choice preferential looking paradigm to measure eye movement responses to different visual stimuli. The child sits without head support in front of a monitor with integrated infrared cameras. In one of four monitor quadrants a visual stimulus is presented. Each stimulus has a specific visual modality with respect to the background, e.g., form, motion, contrast or color. From the reflexive eye movement responses to these specific visual modalities, output parameters such as reaction times, fixation accuracy and fixation duration are calculated to quantify a child's viewing behavior. With this approach, the quality of visual information processing can be assessed without the use of communication. By comparing results with reference values obtained in typically developing children from 0-12 years, the method provides a characterization of visual information processing in visually impaired children. The quantitative information provided by this method can be advantageous for the field of clinical visual assessment and rehabilitation in multiple ways. The parameter values provide a good basis to: (i) characterize early visual capacities and consequently to enable early interventions; (ii) compare risk groups and follow visual development over time; and (iii), construct an individual visual profile for each child.

A method is described to quantify the quality of visual information processing based on reflexive eye movements in response to specific visual modalities. Reaction times and fixation output parameters are used to characterize visual performance in children with and without visual impairments from 6 months of age.

Visual problems that occur early in life can have major impact on a child's development. Without verbal communication and only based on observational ...

Eye-Tracking Control to Assess Cognitive Functions in Patients with Amyotrophic Lateral Sclerosis

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disorder with pathological involvement of upper and lower motoneurons, subsequently leading to progressive loss of motor and speech abilities. In addition, cognitive functions are impaired in a subset of patients. To evaluate these potential deficits in severely physically impaired ALS patients, eye-tracking is a promising means to conduct cognitive tests. The present article focuses on how eye movements, an indirect means of communication for physically disabled patients, can be utilized to allow for detailed neuropsychological assessment. The requirements, in terms of oculomotor parameters that have to be met for sufficient eye-tracking in ALS patients are presented. The properties of stimuli, including type of neuropsychological tests and style of presentation, best suited to successfully assess cognitive functioning, are also described. Furthermore, recommendations regarding procedural requirements are provided. Overall, this methodology provides a reliable, easy to administer and fast approach for assessing cognitive deficits in patients who are unable to speak or write such as patients with severe ALS. The only confounding factor might be deficits in voluntary eye movement control in a subset of ALS patients.

Cognitive deficits are common in about one third of patients with amyotrophic lateral sclerosis, a neurological condition leading to progressive impairments in speech and movement abilities. To conduct cognitive tests in patients unable to speak or write a reliable and easy to administer eye-tracking paradigm was developed.

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disorder with pathological involvement of upper and lower motoneurons, subsequently leading to ...

Experience is Instrumental in Tuning a Link Between Language and Cognition: Evidence from 6-  to 7- Month-Old Infants' Object Categorization

At birth, infants not only prefer listening to human vocalizations, but also have begun to link these vocalizations to cognition: For infants as young as three months of age, listening to human language supports object categorization, a core cognitive capacity. This precocious link is initially broad: At 3 and 4 months, vocalizations of both humans and nonhuman primates support categorization. But by 6 months, infants have narrowed the link: Only human vocalizations support object categorization. Here we ask what guides infants as they tune their initially broad link to a more precise one, engaged only by the vocalizations of our species. Across three studies, we use a novel exposure paradigm to examine the effects of experience. We document that merely exposing infants to nonhuman primate vocalizations enables infants to preserve the early-established link between this signal and categorization. In contrast, exposing infants to backward speech &#8212; a signal that fails to support categorization at any age &#8212; offers no such advantage. Our findings reveal the power of early experience as infants specify which signals, from an initially broad set, they will continue to link to cognition.

At 3-4 months, listening to human and nonhuman primate vocalizations boosts infants&#39; cognition; by 6 months, only human vocalizations exert this cognitive advantage. We describe an exposure manipulation that reveals the powerful shaping role of experience as infants specify which sounds to link to cognition and which to tune out.

At birth, infants not only prefer listening to human vocalizations, but also have begun to link these vocalizations to cognition: For infants as young as ...

The (Spatial) Memory Game: Testing the Relationship Between Spatial Language, Object Knowledge, and Spatial Cognition

The memory game paradigm is a behavioral procedure to explore the relationship between language, spatial memory, and object knowledge. Using two different versions of the paradigm, spatial language use and memory for object location are tested under different, experimentally manipulated conditions. This allows us to tease apart proposed models explaining the influence of object knowledge on spatial language (e.g., spatial demonstratives), and spatial memory, as well as understanding the parameters that affect demonstrative choice and spatial memory more broadly. Key to the development of the method was the need to collect data on language use (e.g., spatial demonstratives: &#34;this/that&#34;) and spatial memory data under strictly controlled conditions, while retaining a degree of ecological validity. The language version (section 3.1) of the memory game tests how conditions affect language use. Participants refer verbally to objects placed at different locations (e.g., using spatial demonstratives: &#34;this/that red circle&#34;). Different parameters can be experimentally manipulated: the distance from the participant, the position of a conspecific, and for example whether the participant owns, knows, or sees the object while referring to it. The same parameters can be manipulated in the memory version of the memory game (section 3.2). This version tests the effects of the different conditions on object-location memory. Following object placement, participants get 10 seconds to memorize the object&#39;s location. After the object and location cues are removed, participants verbally direct the experimenter to move a stick to indicate where the object was. The difference between the memorized and the actual location shows the direction and strength of the memory error, allowing comparisons between the influences of the respective parameters.

The Spatial Memory Game: Testing the Relationship Between Spatial Language, Object Knowledge, and Spatial Cognition

We present a protocol to explore the relationship between spatial language production, spatial memory, and object knowledge. The procedure allows experimental manipulation of, and control over, conditions of object knowledge, language at instruction, and physical location, thus teasing apart cognitive and linguistic models describing interactions between these variables.

The memory game paradigm is a behavioral procedure to explore the relationship between language, spatial memory, and object knowledge. Using two different ...

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

The present work is a description and an assessment of a methodology designed to quantify different aspects of the interaction between language processing and the perception of the visual world. The recording of eye-gaze patterns has provided good evidence for the contribution of both the visual context and linguistic/world knowledge to language comprehension. Initial research assessed object-context effects to test&#160;theories of modularity in language processing. In the introduction, we describe how subsequent investigations have taken the role of the wider visual context in language processing as a research topic in its own right, asking questions such as how our visual perception of events and of speakers contributes to&#160;comprehension informed by&#160;comprehenders&#39; experience. Among the examined aspects of the visual context are actions, events, a speaker&#39;s gaze, and emotional facial expressions, as well as spatial object configurations. Following an overview of the eye-tracking method and its different applications, we list the key steps of the methodology in the protocol, illustrating how to successfully use it to study visually-situated language comprehension. A final section presents three sets of representative results and illustrates the benefits and limitations of eye tracking for investigating the interplay between the perception of the visual world and language comprehension.

The present article reviews an eye-tracking methodology for studies on language comprehension. To obtain reliable data, key steps of the protocol must be followed. Among these are the correct set-up of the eye tracker (e.g., ensuring good quality of the eye and head images) and accurate calibration.

The present work is a description and an assessment of a methodology designed to quantify different aspects of the interaction between language processing ...

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

In a typical eye tracking study using the visual world paradigm, participants' eye movements to objects or pictures in the visual workspace are recorded via an eye tracker as the participant produces or comprehends a spoken language describing the concurrent visual world. This paradigm has high versatility, as it can be used in a wide range of populations, including those who cannot read and/or who cannot overtly give their behavioral responses, such as preliterate children, elderly adults, and patients. More importantly, the paradigm is extremely sensitive to fine grained manipulations of the speech signal, and it can be used to study the online processing of most topics in language comprehension at multiple levels, such as the fine grained acoustic phonetic features, the properties of words, and the linguistic structures. The protocol described in this article illustrates how a typical visual world eye tracking study is conducted, with an example showing how the online processing of some semantically complex statements can be explored with the visual world paradigm.

The visual world paradigm monitors participants' eye movements in the visual workspace as they are listening to or speaking a spoken language. This paradigm can be used to investigate the online processing of a wide range of psycholinguistic questions, including semantically complex statements, such as disjunctive statements.

In a typical eye tracking study using the visual world paradigm, participants' eye movements to objects or pictures in the visual workspace are recorded ...

Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information. However, there is currently a paucity of research exploring how preschool children with autism understand sentences using different types of cues. The mechanisms underlying sentence comprehension remains largely unclear. The present study presents a protocol to examine the sentence comprehension abilities of preschool children with autism. More specifically, a visual world paradigm of eye-tracking is used to explore the moment-to-moment sentence comprehension in the children. The paradigm has multiple advantages. First, it is sensitive to the time course of sentence comprehension and thus can provide rich information about how sentence comprehension unfolds over time. Second, it requires minimal task and communication demands, so it is ideal for testing children with autism. To further minimize the computational burden of children, the present study measures eye movements that arise as automatic responses to linguistic input rather than measuring eye movements that accompany conscious responses to spoken instructions.

We present a protocol to examine the use of morphological cues during real-time sentence comprehension by children with autism.

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information. However, there is ...

Gaze in Action: Head-mounted Eye Tracking of Children's Dynamic Visual Attention During Naturalistic Behavior

Young children's visual environments are dynamic, changing moment-by-moment as children physically and visually explore spaces and objects and interact with people around them. Head-mounted eye tracking offers a unique opportunity to capture children's dynamic egocentric views and how they allocate visual attention within those views. This protocol provides guiding principles and practical recommendations for researchers using head-mounted eye trackers in both laboratory and more naturalistic settings. Head-mounted eye tracking complements other experimental methods by enhancing opportunities for data collection in more ecologically valid contexts through increased portability and freedom of head and body movements compared to screen-based eye tracking. This protocol can also be integrated with other technologies, such as motion tracking and heart-rate monitoring, to provide a high-density multimodal dataset for examining natural behavior, learning, and development than previously possible. This paper illustrates the types of data generated from head-mounted eye tracking in a study designed to investigate visual attention in one natural context for toddlers: free-flowing toy play with a parent. Successful use of this protocol will allow researchers to collect data that can be used to answer questions not only about visual attention, but also about a broad range of other perceptual, cognitive, and social skills and their development.

Young children do not passively observe the world, but rather actively explore and engage with their environment. This protocol provides guiding principles and practical recommendations for using head-mounted eye trackers to record infants' and toddlers' dynamic visual environments and visual attention in the context of natural behavior.

Young children's visual environments are dynamic, changing moment-by-moment as children physically and visually explore spaces and objects and interact ...

Simultaneous Eye Tracking and Single-Neuron Recordings in Human Epilepsy Patients

Intracranial recordings from patients with intractable epilepsy provide a unique opportunity to study the activity of individual human neurons during active behavior. An important tool for quantifying behavior is eye tracking, which is an indispensable tool for studying visual attention. However, eye tracking is challenging to use concurrently with invasive electrophysiology and this approach has consequently been little used. Here, we present a proven experimental protocol to conduct single-neuron recordings with simultaneous eye tracking in humans. We describe how the systems are connected and the optimal settings to record neurons and eye movements. To illustrate the utility of this method, we summarize results that were made possible by this setup. This data shows how using eye tracking in a memory-guided visual search task allowed us to describe a new class of neurons called target neurons, whose response was reflective of top-down attention to the current search target. Lastly, we discuss the significance and solutions to potential problems of this setup. Together, our protocol and results suggest that single-neuron recordings with simultaneous eye tracking in humans are an effective method to study human brain function. It provides a key missing link between animal neurophysiology and human cognitive neuroscience.

We describe a method to conduct single-neuron recordings with simultaneous eye tracking in humans. We demonstrate the utility of this method and illustrate how we used this approach to obtain neurons in the human medial temporal lobe that encode targets of a visual search.

Intracranial recordings from patients with intractable epilepsy provide a unique opportunity to study the activity of individual human neurons during ...

Exploring Infant Sensitivity to Visual Language using Eye Tracking and the Preferential Looking Paradigm

We discuss the use of the preferential looking paradigm in eye tracking studies in order to study how infants develop, understand, and attend to the world around them. Eye tracking is a safe and non-invasive way to collect gaze data from infants, and the preferential looking paradigm is simple to design and only requires the infant to be attending to the screen. By simultaneously showing two visual stimuli that differ in one dimension, we can assess whether infants show different looking behavior for either stimulus, thus demonstrating sensitivity to that difference. The challenges in such experimental approaches are that experiments must be kept brief (no more than 10 min) and be carefully controlled such that the two stimuli differ in only one way. The interpretation of null results must also be carefully considered. In this paper, we illustrate a successful example of an infant eye tracking study with a preferential looking paradigm to discover that 6-month-olds are sensitive to linguistic cues in a signed language despite having no prior exposure to signed language, suggesting that infants possess intrinsic or innate sensitivities to these cues.

Eye tracking studies using a preferential looking paradigm can be used to study infants&#39;&#160;emerging understanding of, and attention to, their external visual world.

We discuss the use of the preferential looking paradigm in eye tracking studies in order to study how infants develop, understand, and attend to the world ...