Name: defining the role of language in infants' object categorization with eye-tracking paradigms
Uploaded: 2019-02-08T15:00:00
Description: Watch this Scientific Journal Video about Defining the Role Of Language in Infants' Object Categorization with Eye-tracking Paradigms at JoVE.com

The research reported here was supported by the National Institute of Child Health and Human Development of the National Institutes of Health under award number R01HD083310 and a National Science Foundation Graduate Research Fellowship under grant no. DGE&#8208;1324585. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the National Science Foundation.

All methods described here have been approved by the Northwestern University Institutional Review Board.
1. Stimuli Creation
NOTE: The visual stimuli (see Figure 1) used in the representative design reported below were originally developed in Havy and Waxman (2016)18 and are available for download at https://osf.io/n6uy8/.
<ol>
	<li>To create a new continuous category, first design a pair of novel digital image...

<ol>
	<li>Eimas, P. D., Quinn, P. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Studies+on+the+Formation+of+Perceptually+Based+Basic-Level+Categories+in+Young+Infants.">Studies on the Formation of Perceptually Based Basic-Level Categories in Young Infants.</a> Child Development. 65 (3), 903-917 (1994).</li><li>Madole, K. L., Oakes, L. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Making+sense+of+infant+categorization:+Stable+processes+and+changing+representations.">Making sense of infant categorization: Stable processes and changing representations.</a> Developmental Review. 19 (2), 263-296 (1999).</li><li>Gelman, S. A., Markman, E. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Categories+and+induction+in+young+children.">Categories and induction in young children.</a> Cognition. 23 (3), 183-209 (1986).</li><li>Ferry, A. L., Hespos, S. J., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Categorization+in+3-+and+4-month-old+infants:+An+advantage+of+words+over+tones.">Categorization in 3- and 4-month-old infants: An advantage of words over tones.</a> Child development. 81 (2), 472-479 (2010).</li><li>Fulkerson, A. L., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Words+(but+not+Tones)+Facilitate+Object+Categorization:+Evidence+From+6-+and+12-Month-Olds.">Words (but not Tones) Facilitate Object Categorization: Evidence From 6- and 12-Month-Olds.</a> Cognition. 105 (1), 218-228 (2007).</li><li>Balaban, M. T., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Do+words+facilitate+object+categorization+in+9-month-old+infants?.">Do words facilitate object categorization in 9-month-old infants?.</a> Journal of Experimental Child Psychology. 64 (1), 3-26 (1997).</li><li>Waxman, S. R., Braun, I. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Consistent+(but+not+variable)+names+as+invitations+to+form+object+categories:+New+evidence+from+12-month-old+infants.">Consistent (but not variable) names as invitations to form object categories: New evidence from 12-month-old infants.</a> Cognition. 95 (3), B59-B68 (2005).</li><li>Balaban, M. T., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+examination+of+the+factors+underlying+the+facilitative+effect+of+word+phrases+on+object+categorization+in+9-month-old+infants.">An examination of the factors underlying the facilitative effect of word phrases on object categorization in 9-month-old infants.</a> Proceedings of the 20th Boston University Conference on Language Development. 1, 483-493 (1996).</li><li>Waxman, S. R., Markow, D. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Words+as+invitations+to+form+categories:+evidence+from+12-+to+13-month-old+infants.">Words as invitations to form categories: evidence from 12- to 13-month-old infants.</a> Cognitive Psychology. 29 (3), 257-302 (1995).</li><li>Zhu, X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Semi-supervised learning literature survey. , (2005).</li><li>Chapelle, O., Scholkopf, B., Zien, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Semi-supervised learning: Adaptive computation and machine learning. , (2006).</li><li>Zhu, X., Goldberg, A. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Introduction+to+semi-supervised+learning.">Introduction to semi-supervised learning.</a> Synthesis lectures on artificial intelligence and machine learning. 3 (1), 1-130 (2009).</li><li>Ferry, A. L., Hespos, S. J., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Nonhuman+primate+vocalizations+support+categorization+in+very+young+human+infants.">Nonhuman primate vocalizations support categorization in very young human infants.</a> Proceedings of the National Academy of Sciences of the United States of America. 110 (38), 15231-15235 (2013).</li><li>Hunter, M. A., Ames, E. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+multifactor+model+of+infant+preferences+for+novel+and+familiar+stimuli.">A multifactor model of infant preferences for novel and familiar stimuli.</a> Advances in infancy research. , (1988).</li><li>Rose, S. A., Feldman, J. F., Jankowski, J. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Infant+visual+recognition+memory.">Infant visual recognition memory.</a> Developmental Review. 24 (1), 74-100 (2004).</li><li>Wetherford, M. J., Cohen, L. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Developmental+changes+in+infant+visual+preferences+for+novelty+and+familiarity.">Developmental changes in infant visual preferences for novelty and familiarity.</a> Child Development. , 416-424 (1973).</li><li>Perone, S., Spencer, J. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Autonomous+visual+exploration+creates+developmental+change+in+familiarity+and+novelty+seeking+behaviors.">Autonomous visual exploration creates developmental change in familiarity and novelty seeking behaviors.</a> Frontiers in psychology. 4, 648 (2013).</li><li>Havy, M., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Naming+influences+9-month-olds&#8217;+identification+of+discrete+categories+along+a+perceptual+continuum.">Naming influences 9-month-olds&#8217; identification of discrete categories along a perceptual continuum.</a> Cognition. 156, 41-51 (2016).</li><li>Althaus, N., Plunkett, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Timing+matters:+The+impact+of+label+synchrony+on+infant+categorisation.">Timing matters: The impact of label synchrony on infant categorisation.</a> Cognition. 139, 1-9 (2015).</li><li>Maris, E., Oostenveld, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Nonparametric+statistical+testing+of+EEG-+and+MEG-data.">Nonparametric statistical testing of EEG- and MEG-data.</a> Journal of Neuroscience Methods. 164 (1), 177-190 (2007).</li><li>Raudenbush, S. W., Bryk, A. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Hierarchical+Linear+Models:+Applications+and+Data+Analysis+Methods.">Hierarchical Linear Models: Applications and Data Analysis Methods.</a> SAGE. , (2002).</li><li>LaTourrette, A., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+little+labeling+goes+a+long+way:+Semi-supervised+learning+in+infancy.">A little labeling goes a long way: Semi-supervised learning in infancy.</a> Developmental Science. , e12736 (2018).</li><li>Dink, J., Ferguson, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> eyetrackingR: An R library for eyetracking data analysis. , (2015).</li><li>Kalish, C. W., Zhu, X., Rogers, T. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Drift+in+children's+categories:+When+experienced+distributions+conflict+with+prior+learning.">Drift in children's categories: When experienced distributions conflict with prior learning.</a> Developmental Science. 18 (6), 940-956 (2015).</li><li>Gibson, B. R., Rogers, T. T., Zhu, X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Human+semi-supervised+learning.">Human semi-supervised learning.</a> Topics in Cognitive Science. 5 (1), 132-172 (2013).</li><li>Keates, J., Graham, S. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Category+Markers+or+Attributes+Why+Do+Labels+Guide+Infants'+Inductive+Inferences?.">Category Markers or Attributes Why Do Labels Guide Infants' Inductive Inferences?.</a> Psychological Science. 19 (12), 1287-1293 (2008).</li><li>Booth, A. E., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+horse+of+a+different+color:+Specifying+with+precision+infants&#8217;+mappings+of+novel+nouns+and+adjectives.">A horse of a different color: Specifying with precision infants&#8217; mappings of novel nouns and adjectives.</a> Child development. 80 (1), 15-22 (2009).</li><li>Perszyk, D. R., Waxman, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Listening+to+the+calls+of+the+wild:+The+role+of+experience+in+linking+language+and+cognition+in+young+infants.">Listening to the calls of the wild: The role of experience in linking language and cognition in young infants.</a> Cognition. 153, 175-181 (2016).</li><li>Althaus, N., Mareschal, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Labels+direct+infants&#8217;+attention+to+commonalities+during+novel+category+learning.">Labels direct infants&#8217; attention to commonalities during novel category learning.</a> PLoS ONE. 9 (7), e99670 (2014).</li><li>Fulkerson, A. L., Haaf, R. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+influence+of+labels,+non-labeling+sounds,+and+source+of+auditory+input+on+9-+and+15-month-olds&#8217;+object+categorization.">The influence of labels, non-labeling sounds, and source of auditory input on 9- and 15-month-olds&#8217; object categorization.</a> Infancy. 4 (3), 349-369 (2003).</li></ol>

Here, we present a procedure for evaluating the role of labeling in categorization. By presenting 2-year-olds with a realistic mix of labeled and unlabeled exemplars, we demonstrate that very young children are capable of learning in semi-supervised environments, extending work with adults and older children24,25. Thus, this method offers a resolution to the paradox posed above: if even a few labeled exemplars can spark category learning, then labels can be both ...

Categorization is a fundamental building block of human cognition: infants&#8217; categorization abilities emerge early in infancy and become increasingly sophisticated with age.1,2,3 Research has also revealed a powerful role for&#160;language in infant categorization: from 3 months of age, infants learn categories more successfully when category exemplars are paired with language.4,5,6 Moreover, by the end of the first year, infants are attuned to the role of count noun labels in categorization. Pairing category exemplars with a consistent labeling phrase (&#8220;This is a vep!&#8221;) facilitates infants&#8217; category learning relative to providing either a distinct label for each exemplar (&#8220;This is a vep,&#8221; &#8220;This is a dax,&#8221; etc.) or a non-labeling phrase (&#8220;Look at this.&#8221;).7,8,9
In infants&#8217; everyday experiences, however, the vast majority of objects they encounter will likely remain unlabeled. No caregiver could label every object an infant&#160;sees much less provide the labels which apply to every object (e.g., &#8220;malamute,&#8221; &#8220;dog,&#8221; &#8220;pet,&#8221; &#8220;animal&#8221;). This presents a paradox: how can we reconcile the power of labels in infant categorization with their relative scarcity in infants&#8217; daily lives?
To answer this question, we developed a protocol to assess how infants learn categories in a variety of different learning environments, including when they receive a mixture of labeled and unlabeled exemplars. Specifically, we propose that receiving even a few labeled exemplars at the beginning of learning can facilitate categorization&#8212;by enhancing infants&#8217; ability to learn from subsequent, unlabeled exemplars as well. This strategy of using a small number of labeled exemplars as a foundation for learning from a larger number of unlabeled exemplars has been widely implemented in the field of machine learning, spawning a family of semi-supervised learning (SSL) algorithms10,11,12. Of course, the learning strategies implemented are not identical across different kinds of learners: in machine learning, algorithms typically are exposed to many more exemplars, make explicit guesses about each exemplar, and learn multiple categories simultaneously. Nevertheless, both machine and infant learners may benefit from successfully integrating both labeled and unlabeled exemplars&#160;to learn new categories in sparse labeling environments.
Our design focuses on whether 2-year-old children, in the process of acquiring words for numerous new categories, are capable of this kind of semi-supervised learning. We employ a standard infant categorization measure: a familiarization-test task. In this paradigm, 2-year-olds were exposed to a series of exemplars from a novel category during a familiarization phase. Each exemplar was paired with a different auditory stimulus, depending on the condition (i.e., either a labeling or a non-labeling phrase). Then, at the test, all 2-year-olds saw two new objects presented in silence: one object from the now-familiar category and one from a novel category.
If the 2-year-olds successfully form the category during the familiarization phase, then they should distinguish between the two exemplars&#160;presented at the test. Importantly, because a systematic preference for either the novel or familiar test image reflects an ability to distinguish between them, both familiarity and novelty preferences are interpreted as evidence of successful categorization. Note that on a given task, the nature of this preference is a function of infants&#8217; processing efficiency for the stimulus materials, with familiarity preferences associated with less efficient stimulus processing 4,13,14,15,16,17. Presenting the test phase in silence makes it possible to directly assess infants&#8217; success in object categorization and how this success varies according to the information that accompanied the exemplars during familiarization. Thus, this paradigm provides a compelling test of how different types of linguistic environments affect category learning. If labeling enhances category learning in both semi-supervised and fully supervised environments, then 2-year-olds in these conditions should show stronger test preferences than infants in other environments.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Final Cut Pro X</td>
			<td>Apple</td>
			<td>N/A</td>
			<td>Video editing, composition software</td>
		</tr>
		<tr>
			<td>MorphX</td>
			<td>Norrkross</td>
			<td>N/A</td>
			<td>Image-morphing software</td>
		</tr>
		<tr>
			<td>PhotoShop</td>
			<td>Adobe</td>
			<td>N/A</td>
			<td>Image-editing software</td>
		</tr>
		<tr>
			<td>R</td>
			<td>R Core Team</td>
			<td>N/A</td>
			<td>Statistical analysis software</td>
		</tr>
		<tr>
			<td>T60XL Eyetracker</td>
			<td>Tobii Pro</td>
			<td>Discontinued</td>
			<td>Large, arm-mounted eyetracker suitable for work with infants and children</td>
		</tr>
		<tr>
			<td>Tobii Pro Studio</td>
			<td>Tobii Pro</td>
			<td>N/A</td>
			<td>Software directing eyetracker display, data collection</td>
		</tr>
	</tbody>
</table>

defining the role of language in infants' object categorization with eye-tracking paradigms

Assessing infant category learning is a challenging but vital aspect of studying infant cognition. By employing a familiarization-test paradigm, we straightforwardly measure infants&#8217; success in learning a novel category while relying only on their looking behavior. Moreover, the paradigm can directly measure the impact of different auditory signals on the infant categorization across a range of ages. For instance, we assessed how 2-year-olds learn categories in a variety of labeling environments: in our task, 2-year-olds successfully learned categories when all exemplars were labeled&#160;or the first two exemplars were labeled, but they failed to categorize when no exemplars were labeled or only the final two exemplars were labeled. To determine infants&#8217; success in such tasks, researchers can examine both the overall preference displayed by infants in each condition and infants&#8217; pattern of looking over the course of the test phase, using an eye-tracker to provide fine-grained time-course data. Thus, we present a powerful paradigm for identifying the role of language, or any auditory signal, in infants&#8217; object category learning.

Using the protocol above, we ran two experiments22. Analyses were conducted with the eyetrackingR package23, and the data and code are available at https://github.com/sandylat/ssl-in-infancy. In the first experiment, we contrasted a fully supervised condition (n = 24, Mage = 26.8 mo), featuring only labeled exemplars, with an unsupervised condition (n = 24, Mage = 26.9 mo), featuring only unlabeled ex...

Watch this Scientific Journal Video about Defining the Role Of Language in Infants' Object Categorization with Eye-tracking Paradigms at JoVE.com

Defining the Role Of Language in Infants' Object Categorization with Eye-tracking Paradigms

Here we present a protocol for familiarization-test paradigms which provide a direct test of infant categorization and help to define the role of language in early category learning.

Assessing infant category learning is a challenging but vital aspect of studying infant cognition. By employing a familiarization-test paradigm, we ...

Research

JoVE Journal

Behavior

A Method to Quantify Visual Information Processing in Children Using Eye Tracking

Visual problems that occur early in life can have major impact on a child's development. Without verbal communication and only based on observational ...

Eye-Tracking Control to Assess Cognitive Functions in Patients with Amyotrophic Lateral Sclerosis

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disorder with pathological involvement of upper and lower motoneurons, subsequently leading to ...

Experience is Instrumental in Tuning a Link Between Language and Cognition: Evidence from 6-  to 7- Month-Old Infants' Object Categorization

At birth, infants not only prefer listening to human vocalizations, but also have begun to link these vocalizations to cognition: For infants as young as ...

The (Spatial) Memory Game: Testing the Relationship Between Spatial Language, Object Knowledge, and Spatial Cognition

The Spatial Memory Game: Testing the Relationship Between Spatial Language, Object Knowledge, and Spatial Cognition

The memory game paradigm is a behavioral procedure to explore the relationship between language, spatial memory, and object knowledge. Using two different ...

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

The present work is a description and an assessment of a methodology designed to quantify different aspects of the interaction between language processing ...

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

In a typical eye tracking study using the visual world paradigm, participants' eye movements to objects or pictures in the visual workspace are recorded ...

Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information. However, there is ...

Gaze in Action: Head-mounted Eye Tracking of Children's Dynamic Visual Attention During Naturalistic Behavior

Young children's visual environments are dynamic, changing moment-by-moment as children physically and visually explore spaces and objects and interact ...

Comparing Eye-tracking Data of Children with High-functioning ASD, Comorbid ADHD, and of a Control Watching Social Videos

Children with autism spectrum disorders (ASD) are known to have sensory-perceptual processing deficits that weaken their abilities to attend and perceive ...

Simultaneous Eye Tracking and Single-Neuron Recordings in Human Epilepsy Patients

Intracranial recordings from patients with intractable epilepsy provide a unique opportunity to study the activity of individual human neurons during ...