This study was supported by the National Social Science Fund of China for the project (19ZDA332) entitled &#8220;Deep Convergence of Media and Innovation Model of Social Governance in the New Era;&#8221; a PhD Start-up Fund of the Natural Science Foundation of Guangdong Province, China (2017A030310536); and the Fundamental Research Funds for Central Universities (19JNQM04). The authors would like to thank Dean/Professor Zhi Tingrong for support and investment in the eye tracking laboratory.

This protocol adheres to the Jinan University research guidelines. As only the medical school of the university has an IRB board, no other discipline is required to have IRB approvals. However, the researchers confirm that all ethical rules and regulations were followed. The project did not pose any physical or psychological harm to participants.
1. Participant screening for the experiment
<ol>
	<li>Recruit native Chinese speakers from a southern Chinese university with normal or corrected-to-normal vision and no hearing impairments nor psychiatric history.</li>
	<li>Measure loneliness using the third edition of the UCLA scale60,61. Categorize those who score 44 or above in the scale as the lonely group and place the rest in the normal group. In terms of gender and age distribution, the group corresponds to the demographic characteristics of barrage viewers47.</li>
</ol>
2. Stimuli construction
<ol>
	<li>Use rational appeal ads and emotional appeal ads as video stimuli. Positive emotional ads provide emotional support for lonely audiences, where rational ads do not.
	<ol>
		<li>Choose ads that are standalone video pieces that do not require contextual information to comprehend. The typical ad length is also ideal for quick variations, experimental manipulations, and for eye-tracking data collection62. The length of each video is about one minute.</li>
	</ol>
	</li>
	<li>To ensure that emotional and rational appeals are manipulated successfully, have people watch and rate a pool of preselected ads based on these appeals. Here, thirteen coders majoring in advertising were used.
	<ol>
		<li>To maximize manipulation, choose videos with the highest scores in either categories to be the experimental stimuli. The final selected ads represented eight kinds of products. The same product has an emotional ad and a rational ad.</li>
	</ol>
	</li>
	<li>Make the barrage video. There are two ways to make a barrage video for research.
	<ol>
		<li>Upload the video to a barrage website, such as the B station at Bilibili (https://www.bilibili.com/). Have participants log into their accounts to watch, and comment. The uploader controls the position of the barrage, usually at the upper third of the screen.</li>
		<li>Alternatively, use video editing software to convert barrage into subtitles, so comments can be manually added to the video barrage area. The ready-made video can then be called in the data-collection process. This experiment used the second method.</li>
	</ol>
	</li>
	<li>Produce four presentation orders for the experiment to randomize presentational effects. In this design, each participant only saw one version of a particular video, either emotional or rational appeal of an ad, and each saw eight ads altogether.</li>
</ol>
3. Eye tracking protocol
<ol>
	<li>Eye tracking procedure
	<ol>
		<li>Use a commercial eye tracker in the study. Set the default setting for the tracker gaze sample rate at 60 Hz per second.</li>
		<li>Place a 24-inch computer screen 50 cm from the participant&#8217;s chair. Attach the eye tracker to the computer.</li>
		<li>Invite the participant in the lab. Ask the participant to read and sign an informed consent form. Ask the participant to sit comfortably in front of the test computer.</li>
		<li>Have the experimenter check, and adjust the chair height if necessary, to make sure that the TV screen is at the participant&#8217;s eye level.</li>
		<li>Ask participants to sit still to complete a calibration task to ensure that data collected during the experiment are accurate. Inform participant that a 5-point calibration is necessary to achieve the highest accuracy in data collection, and to track participants&#8217; gaze within 2&#176; of accuracy.</li>
		<li>Ask participant to follow a moving red dot on the computer screen with both eyes and to fixate on it when it stopped. If a participant looks away during calibration, then repeat the process.</li>
		<li>Check the tracker software to see if a participant misses a calibration point. If so, repeat the calibration.</li>
		<li>Ask the participant to click the left mouse button to start an exercise test to familiarize them with the experimental procedure.</li>
		<li>Start the main experiment and tell the participant that he/she will see a red &#34;+&#34; sign in the middle of the screen, which lasts 500 ms, alerting to the start of the experiment.</li>
		<li>Ask participant to watch the first video while eye tracking is on.</li>
		<li>After the first video, ask participant to complete a questionnaire page, which automatically pops up. Have participants complete a battery of evaluative measures on satisfaction with the viewing by clicking the left mouse button and choosing ratings.</li>
		<li>Ask participant to take a break if desired or continue to another video (see Figure 3, credited to computers in Human Behaviors (CHB) for a flowchart of the experiment).</li>
		<li>Repeat the procedure seven more times for each participant to complete the entire protocol and the eight ads for each.</li>
		<li>Thank, debrief, and pay participant CN&#165;10.</li>
	</ol>
	</li>
</ol>
<img alt="Figure 3" class="xfigimg" src="/files/ftp_upload/61089/61089fig03.jpg" /> 
Figure 3: Experimental flowchart <a href="https://www.jove.com/files/ftp_upload/61089/61089fig03large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<ol start="2">
	<li>Eye-tracking data and self-reported data analysis 
	NOTE: The eye tracker recorded the whole length of the experiment, including the segments of ad watching and questionnaire responding. In this recording, eye movement data were superimposed over video. Two measures were used in this study, including fixations and durations. A fixation is where the eyes were relatively still, with the central foveal vision being held in place so the human visual system could process the information in that point. A fixation in an eye-tracker was typically defined as a succession of raw gaze points where the velocity was below a pre-defined threshold in the tracker&#8217;s gaze filter. More than 60 ms of gaze would be considered as a fixation. Duration, on the other hand, was the length of time between the onset of first gaze point and the last gaze point that make up a fixation.
	<ol>
		<li>Slice the entire recording into eight segments corresponding to each ad watching segment. Each clip still contains the original ad and eye movement data.</li>
		<li>On the sliced video, use the tracker software to draw an area of interest (AOI) to distinguish between eye movement data in the barrage area and non-barrage area. The top third was the AOI for barrage, and the lower two-thirds were the AOI for non-barrage.</li>
		<li>First count the number of fixations for each video segment, and separate them into fixations on the barrage AOI and non-barrage AOI.</li>
		<li>Calculate durations.</li>
		<li>Compare duration and the number of fixations at the barrage AOI relative to the whole scene. These two measures allow researchers to infer where participants are focusing and what elements they are paying attention to while watching the videos.</li>
		<li>Analyze self-reported data to examine participants&#39; satisfaction towards video</li>
	</ol>
	</li>
</ol>

The authors declare that they have nothing to disclose.

In this study, eye tracking technology and self-reports are combined to test the validity of the proposed AMSM model. Previous studies mainly used self-reports to explore the relationship between loneliness and media use at the mercy of participants&#8217; ability to articulate. These offline methods fail to understand the psychological processes while watching barrage videos. In this study, eye tracking technology was used to record duration and fixation so researchers could reliably infer how much cognitive resources t...

Eye-tracking technique 
The eye is often referred to as a window of the mind1,2. Eighty percent of human information intake is obtained visually3. Since the 19th century, people began to study human psychological activities by directly observing eye movements of participants. Miles invented the peephole to make observations when participants read4. In the study, the experimenter sat opposite a participant and observed the participant&#8217;s eye movements through a small hole in the middle of the reading material. Since then, technology has vastly improved. Currently, state-of-the-art eye-movement tracking devices mainly focus on electric current-recording, magnetic-induction and optical-recording, which includes corneal reflection and iris-scleral reflection methods5,6,7. The noninvasive features widely prevalent today make eye movement recording more natural, enhancing ecological validity. Today, eye movement techniques generally refer to the use of computer-controlled eye tracking to record and analyze the positioning of the participant&#39;s eye and the forms of eye movements during viewing of visual material.
Many theoretical perspectives regarding eye movements have matured over the years. These include the vision buffer processing model, the parafoveally processing model, the E-Z Reader model, the immediately processing model and the eye-mind processing model8,9,10. The immediately processing model holds that the processing of viewing content at all levels is not delayed, but occurs in real-time. The eye-mind processing model focuses on text information and holds that as long as one is processing a word, one would look at it. Put in another way, the word that one processes is exactly the very word one is looking at. The processing time of a word is the total fixation time of the participant&#8217;s eye.
There are three basic types of human eye movements: fixation, saccades and pursuit movements8,11,12. The fixation duration and fixation count usually reflect the extent to which the participant exerts cognitive resources to the content being viewed. Saccade refers to the movement from one gaze point to another. Retrospective saccade can be used as an indicator of the processing sophistication in the encoding process. Regression saccade indicates a deeper processing of an area after the first gaze of the key areas, reflecting the difficulty or interest of the content in that area. Pursuit movements are usually performed when there is visual noise, and the eye seeks out a point of interest.
On the other hand, we can also measure pupil size and blink frequency; both reflect people&#8217;s psychological activities13,14,15. For example, there is a relationship between pupil size and the specific task difficulty, motivation, interest, attitude, and fatigue. At present, the relationship between pupil size and emotional valence is not clear16. However, if a researcher combines pupil size with other indicators, such as electroencephalogram (EEG), the accuracy would be greatly improved17. For blink indicators, according to the hedonic-blink hypothesis, the decrease of blink frequency is usually associated with happy emotional thoughts, while the increase of blink frequency is associated with unhappy emotional states18.
The application of eye-tracking technology is extremely broad, including reading strategies, visual information processing, compulsive behaviors, to even artistic intentions. The application in the field of reading is the most mature. In communication, eye-tracking is useful in news consumption studies and advertising effectiveness research. For example, a large number of eye movement experiments have explored both exogenous and endogenous factors in advertising19,20, with the former exploring the physical characteristics of advertisements such as size, pattern, color, position, originality and repeated presentations21,22, and the latter exploring individual factors such as product involvement, product motivation, prior knowledge and brand familiarity23,24,25,26,27.
In addition, eye tracking technology is widely used in many other fields, such as human-computer interaction and usability research28,29,30,31; skills transfer32; infant and child development research33,34,35,36; marketing and online consumer behavior research37,38,39; and packaging design40,41, among others.
In addition to being used alone, eye tracking is often combined with other multimodal measurement techniques. For example, researchers can combine eye movement data with other physiological indicators such as EEG, skin electrical response, heart rate, skin temperature, facial expression, etc. In this way, users&#39; emotional responses to different kinds of information can be studied more effectively42.
Eye movement technology can also be integrated into other technologies. For example, it is being used with augmented reality technology30. At present, the integration of eye movement technology and virtual reality (VR) technology is worthy of attention. On the one hand, such integration can promote the rapid development of VR related equipment. For example, the graphics processing unit (GPU) of most VR equipment is overburdened and consumes a lot of energy. Eye tracking technology can detect the audience&#39;s fixation point in real time. VR devices only need to focus on rendering this area and ignore other areas. This can significantly reduce power consumption and GPU rendering load. On the other hand, the combination can enhance its interaction features, and improve the immersion and involvement of VR users. For example, players can use eye movement instead of hands to complete operations in a game43. In addition, researchers can implement eye movement testing in a simulated environment. For example, in consumer behavior research, researchers do not need to take subjects to a real shopping mall or use real products, but test them in virtual research scenarios, as shown in Figure 1.
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/61089/61089fig01.jpg" /> 
Figure 1: Virtual research scenario <a href="https://www.jove.com/files/ftp_upload/61089/61089fig01large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
Barrage video 
Barrage is a term originating from the military. In order to intercept moving targets, multiple artillery guns are used to fire and to form a curtain composed of high-density projectiles before the moving targets44. The term &#34;barrage&#34; is used in video viewing to describe a format and a phenomenon in which audience members of a particular video comment and displaying such comments on the video screen while watching a video, thus forming a comment wall curtain, as shown in Figure 2. This format of interacting and commenting is also referred to as overlaid comments45,46. Barrage videos first appeared on the &#34;niconico&#34; video website in Japan. The most popular websites in China are AcFun (Station A, www.acfun.cn) and Bilibili (Station B, www.bilibili.com). Many other streaming video sites in China, such as Youku, Tencent, LeTV and iQiyi, have also added the barrage sending and viewing functions. Barrage videos have attracted many Chinese viewers, and the attention of researchers in many disciplines.
<img alt="Figure 2" class="xfigimg" src="/files/ftp_upload/61089/61089fig02.jpg" /> 
Figure 2: Comment wall curtain <a href="https://www.jove.com/files/ftp_upload/61089/61089fig02large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
Viewers of barrage videos can watch and interact with others by commenting to possibly meet a few psychological needs. The audience of such videos may share similarities in age, background and interests, so they may develop a sense of identity and belonging. There is research, for example, that shows AcFun and Bilibili are home to the &#8220;quadratic element&#8221; genre enthusiasts, with its audience being mainly between 17 and 25 years old47,48. These two websites have set up an extremely strict membership review system in that membership is only granted after acing an entrance exam by answering questions such as &#34;What is the height of the heroine of the Thousands of Defenders in Hurricane Butler?&#34; &#34;How many cruisers have been dispatched by Gonon in the Battle of Rum?&#34; Such criteria basically weed out most of the &#34;heterogeneous&#34; audience who want to &#34;invade&#34; the group.
Related to this study, barrage video creates a sense of crowd watching, which is particularly meaningful for lonely individuals. Because the content of the barrage is highly related to the plot, and comments are immediate responses, it gives the audience an illusion of watching with others even though they may be watching alone physically. This sense of companionship has been shown to alleviate loneliness49.
Barrage videos can also afford entertainment of a different kind in that viewers are not only consuming but also creating video content by blending sometime serious film making with fun plays. Barrage viewers can also find a sanctuary away from reality50,51 in which they can vent their anxiety and engage in emotional catharsis in a safe environment52, or showcase their personalities, demonstrate narcissism by getting others&#8217; attention, and even bypass conformist norms in the real world53.
Viewing mechanism of lonely audience on barrage video 
Barrage video viewing is an ideal platform to study media use and loneliness for its emotional support afforded by the venue. In this study, the researchers find previous conceptualizations of media exposure to be inadequate and therefore offer an activation and match satisfaction model (AMSM) to explain the psychological underpinnings of viewing barrage, especially by loners. In previous research, there are two perspectives that explain media use. The deficiency paradigm holds that loners, for lack of companionship, would devote more cognitive resources to the barrage content during watching, to seek company and to compensate loneliness. The global-use paradigm maintains that media use is prevalent, and that it satisfies generic and interpersonal needs in general. So regardless of emotional state, all viewers would pay attention to barrage, and they obtain different gratifications54. However, AMSM posits that emotional content serves to activate lonely audiences&#39; interpersonal needs, and they would take initiatives to search for interactive and interpersonal communication elements, such as barrage content in the viewing process, and devote more attention to such components. The degree that these elements satisfy their emotional needs determines the degree of satisfaction they obtain.
In order to understand the mechanism of barrage video watching by lonely people, one needs to know the amount of cognitive resources that loners invest in different media content and how they satisfy their needs. However, such data cannot be reliably obtained by traditional participative reporting methods. Cognitive resources allocation operates consciously and subconsciously. It is a tall order that audience members articulate to which parts of content they invest more cognitive resources. To accomplish this, a suitable research methodology is needed to record the viewing process, and to distinguish the amount of attention to different parts of content, in addition to measuring the corresponding satisfaction from the viewing process.
For these reasons, this project tracked participants&#8217; eye movements, as measurements of attention and the degree of cognitive resources allocated. The follow-up Likert scale questions were designed to measure participants&#39; degree of satisfaction from exposure. Eye-tracking is a noninvasive technology that has high temporal and spatial resolutions, allowing recording while participants process continuous visual stimuli without distraction55,56. In this study, duration time and fixation counts are used as measures of attention. Duration time refers to the length of attention, and fixation count refers to the number of gazes at a particular area of the video material. Both of these eye movement indicators have been shown to be valid measures of processing thoroughness, reflecting the cognitive resources allocated by individuals57,58. Results of gaze probability, for example, allow researchers to infer attributes in the video that were important to the participant. For self-reported measures, the researchers use a 7-point Likert scale, asked and answered immediately after watching.
Based on arguments from previous explicated perspectives, the researchers designed a 2 (audience type) x 2 (ad appeal) x 2 (barrage) mixed experimental study and hypothesized the amount of attention that normal and lonely participants would pay to barrage video. Audience type (lonely and normal) was a between factor. Ad appeal had two levels, with either emotional ads or rational ads. Barrage also had two levels, denoting video that either had barrage or not. The last two were within subject factors. The general hypotheses were that lonely audience would pay more attention to emotional ads than to rational ads, and they would pay more attention to barrage than to non-barrage, whereas for normal audience, there was no such difference. Satisfaction ratings followed the same patterns. The Chen et al.59 original paper detailed all these hypotheses.

<table>
	<tbody>
		<tr>
			<td>eyetracker</td>
			<td>Tobii</td>
			<td>T60</td>
			<td></td>
		</tr>
		<tr>
			<td>questionnaire</td>
			<td></td>
			<td></td>
			<td></td>
		</tr>
	</tbody>
</table>

loneliness assuaged: eye-tracking an audience watching barrage videos

Researchers usually theorize media exposure based on assumptions of legacy media. However, a new interactive video viewing format, in this case barrage video where viewers&#8217; comments are overlaid over visual content, challenges past perspectives. This study proposes an activation and match satisfaction model to study viewing behaviors of lonely people and to challenge previous claims. It presents a protocol to examine the mechanism of how loners use barrage videos by combining eye tracking and self-report measures. Eye tracking documents the audience&#39;s conscious and subconscious watching behaviors in real time and allows for inference of the amount of allocated cognitive resources in response to rational and emotional content. The self-report gauges the amount of satisfaction obtained. Overall, results from the measures supported an activation and match satisfaction model regarding loners and their barrage video viewing behaviors. Implications are discussed.

Repeated measures MANOVAs were conducted using duration and fixation as dependent variables, which indicated attention. Results confirmed proposed hypotheses that lonely participants&#8217; gaze stayed on barrage longer than on non-barrage areas when the emotional ads were present. However, when the rational ads were viewed, there was no difference. Such data pattern did not replicate for the low loneliness participants, whose attention remained statistically non-significant when watching emotional ads and rational ads. ...

Watch this Scientific Journal Video about Loneliness Assuaged: Eye-Tracking an Audience Watching Barrage Videos at JoVE.com

Loneliness Assuaged: Eye-Tracking an Audience Watching Barrage Videos

The study proposes an activation-match model to study how loneliness is mitigated when a lonely audience watches barrage videos of rational and emotional appeals. The protocol uses eye tracking to document duration and fixation, accounting for the degree of satisfaction when emotional needs are appeased by content and barrage.

Researchers usually theorize media exposure based on assumptions of legacy media. However, a new interactive video viewing format, in this case barrage ...

loneliness-assuaged-eye-tracking-an-audience-watching-barrage-videos

Research

JoVE Journal

Behavior

4.1K Views.  Jinan University. The study proposes an activation-match model to study how loneliness is mitigated when a lonely audience watches barrage videos of rational and emotional appeals. The protocol uses eye tracking to document duration and fixation, accounting for the degree of satisfaction when emotional needs are appeased by content and barrage.

Video: Loneliness Assuaged: Eye-Tracking an Audience Watching Barrage Videos

Eye Tracking, Cortisol, and a Sleep vs. Wake Consolidation Delay: Combining Methods to Uncover an Interactive Effect of Sleep and Cortisol on Memory

Although rises in cortisol can benefit memory consolidation, as can sleep soon after encoding, there is currently a paucity of literature as to how these two factors may interact to influence consolidation. Here we present a protocol to examine the interactive influence of cortisol and sleep on memory consolidation, by combining three methods: eye tracking, salivary cortisol analysis, and behavioral memory testing across sleep and wake delays. To assess resting cortisol levels, participants gave a saliva sample before viewing negative and neutral objects within scenes. To measure overt attention, participants&#8217; eye gaze was tracked during encoding. To manipulate whether sleep occurred during the consolidation window, participants either encoded scenes in the evening, slept overnight, and took a recognition test the next morning, or encoded scenes in the morning and remained awake during a comparably long retention interval. Additional control groups were tested after a 20&#160;min&#160;delay in the morning or evening, to control for time-of-day effects. Together, results showed that there is a direct relation between resting cortisol at encoding and subsequent memory, only following a period of sleep. Through eye tracking, it was further determined that for negative stimuli, this beneficial effect of cortisol on subsequent memory may be due to cortisol strengthening the relation between where participants look during encoding and what they are later able to remember. Overall, results obtained by a combination of these methods uncovered an interactive effect of sleep and cortisol on memory consolidation.

We present a protocol used to discover an interactive effect between sleep and cortisol on memory consolidation, particularly for negative arousing images. Specifically, the experimental design utilizes eye tracking, salivary cortisol analysis, and behavioral memory testing &#8211; methods that can be used with both healthy and clinical participants.

Although rises in cortisol can benefit memory consolidation, as can sleep soon after encoding, there is currently a paucity of literature as to how these ...

A Method to Quantify Visual Information Processing in Children Using Eye Tracking

Visual problems that occur early in life can have major impact on a child's development. Without verbal communication and only based on observational methods, it is difficult to make a quantitative assessment of a child's visual problems. This limits accurate diagnostics in children under the age of 4 years and in children with intellectual disabilities. Here we describe a quantitative method that overcomes these problems. The method uses a remote eye tracker and a four choice preferential looking paradigm to measure eye movement responses to different visual stimuli. The child sits without head support in front of a monitor with integrated infrared cameras. In one of four monitor quadrants a visual stimulus is presented. Each stimulus has a specific visual modality with respect to the background, e.g., form, motion, contrast or color. From the reflexive eye movement responses to these specific visual modalities, output parameters such as reaction times, fixation accuracy and fixation duration are calculated to quantify a child's viewing behavior. With this approach, the quality of visual information processing can be assessed without the use of communication. By comparing results with reference values obtained in typically developing children from 0-12 years, the method provides a characterization of visual information processing in visually impaired children. The quantitative information provided by this method can be advantageous for the field of clinical visual assessment and rehabilitation in multiple ways. The parameter values provide a good basis to: (i) characterize early visual capacities and consequently to enable early interventions; (ii) compare risk groups and follow visual development over time; and (iii), construct an individual visual profile for each child.

A method is described to quantify the quality of visual information processing based on reflexive eye movements in response to specific visual modalities. Reaction times and fixation output parameters are used to characterize visual performance in children with and without visual impairments from 6 months of age.

Visual problems that occur early in life can have major impact on a child's development. Without verbal communication and only based on observational ...

Eye-Tracking Control to Assess Cognitive Functions in Patients with Amyotrophic Lateral Sclerosis

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disorder with pathological involvement of upper and lower motoneurons, subsequently leading to progressive loss of motor and speech abilities. In addition, cognitive functions are impaired in a subset of patients. To evaluate these potential deficits in severely physically impaired ALS patients, eye-tracking is a promising means to conduct cognitive tests. The present article focuses on how eye movements, an indirect means of communication for physically disabled patients, can be utilized to allow for detailed neuropsychological assessment. The requirements, in terms of oculomotor parameters that have to be met for sufficient eye-tracking in ALS patients are presented. The properties of stimuli, including type of neuropsychological tests and style of presentation, best suited to successfully assess cognitive functioning, are also described. Furthermore, recommendations regarding procedural requirements are provided. Overall, this methodology provides a reliable, easy to administer and fast approach for assessing cognitive deficits in patients who are unable to speak or write such as patients with severe ALS. The only confounding factor might be deficits in voluntary eye movement control in a subset of ALS patients.

Cognitive deficits are common in about one third of patients with amyotrophic lateral sclerosis, a neurological condition leading to progressive impairments in speech and movement abilities. To conduct cognitive tests in patients unable to speak or write a reliable and easy to administer eye-tracking paradigm was developed.

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disorder with pathological involvement of upper and lower motoneurons, subsequently leading to ...

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

The present work is a description and an assessment of a methodology designed to quantify different aspects of the interaction between language processing and the perception of the visual world. The recording of eye-gaze patterns has provided good evidence for the contribution of both the visual context and linguistic/world knowledge to language comprehension. Initial research assessed object-context effects to test&#160;theories of modularity in language processing. In the introduction, we describe how subsequent investigations have taken the role of the wider visual context in language processing as a research topic in its own right, asking questions such as how our visual perception of events and of speakers contributes to&#160;comprehension informed by&#160;comprehenders&#39; experience. Among the examined aspects of the visual context are actions, events, a speaker&#39;s gaze, and emotional facial expressions, as well as spatial object configurations. Following an overview of the eye-tracking method and its different applications, we list the key steps of the methodology in the protocol, illustrating how to successfully use it to study visually-situated language comprehension. A final section presents three sets of representative results and illustrates the benefits and limitations of eye tracking for investigating the interplay between the perception of the visual world and language comprehension.

The present article reviews an eye-tracking methodology for studies on language comprehension. To obtain reliable data, key steps of the protocol must be followed. Among these are the correct set-up of the eye tracker (e.g., ensuring good quality of the eye and head images) and accurate calibration.

The present work is a description and an assessment of a methodology designed to quantify different aspects of the interaction between language processing ...

Eye-tracking to Distinguish Comprehension-based and Oculomotor-based Regressive Eye Movements During Reading

Regressive eye movements are eye movements that move backwards through the text and comprise approximately 10-25% of eye movements during reading. As such, understanding the causes and mechanisms of regressions plays an important role in understanding eye movement behavior. Inhibition of return (IOR) is an oculomotor effect that results in increased latency to return attention to a previously attended target versus a target that was not previously attended. Thus, IOR may affect regressions. This paper describes how to design materials to distinguish between regressions caused by comprehension-related and oculomotor processes; the latter is subject to IOR. The method allows researchers to identify IOR and control the causes of regressions. While the method requires tightly controlled materials and large numbers of participants and materials, it allows researchers to distinguish and control the types of regressions that occur in their reading studies.

The method was designed to investigate the role of inhibition of return (IOR) in regressive eye movements during reading. The focus is on differentiating between regressions triggered as a result of comprehension difficulty versus those triggered from oculomotor error, including the role of IOR in the two types of regressions.

Regressive eye movements are eye movements that move backwards through the text and comprise approximately 10-25% of eye movements during reading. As ...

Gaze in Action: Head-mounted Eye Tracking of Children's Dynamic Visual Attention During Naturalistic Behavior

Young children's visual environments are dynamic, changing moment-by-moment as children physically and visually explore spaces and objects and interact with people around them. Head-mounted eye tracking offers a unique opportunity to capture children's dynamic egocentric views and how they allocate visual attention within those views. This protocol provides guiding principles and practical recommendations for researchers using head-mounted eye trackers in both laboratory and more naturalistic settings. Head-mounted eye tracking complements other experimental methods by enhancing opportunities for data collection in more ecologically valid contexts through increased portability and freedom of head and body movements compared to screen-based eye tracking. This protocol can also be integrated with other technologies, such as motion tracking and heart-rate monitoring, to provide a high-density multimodal dataset for examining natural behavior, learning, and development than previously possible. This paper illustrates the types of data generated from head-mounted eye tracking in a study designed to investigate visual attention in one natural context for toddlers: free-flowing toy play with a parent. Successful use of this protocol will allow researchers to collect data that can be used to answer questions not only about visual attention, but also about a broad range of other perceptual, cognitive, and social skills and their development.

Gaze in Action: Head-mounted Eye Tracking of Children&#039;s Dynamic Visual Attention During Naturalistic Behavior

Young children do not passively observe the world, but rather actively explore and engage with their environment. This protocol provides guiding principles and practical recommendations for using head-mounted eye trackers to record infants' and toddlers' dynamic visual environments and visual attention in the context of natural behavior.

Young children's visual environments are dynamic, changing moment-by-moment as children physically and visually explore spaces and objects and interact ...

Comparing Eye-tracking Data of Children with High-functioning ASD, Comorbid ADHD, and of a Control Watching Social Videos

Children with autism spectrum disorders (ASD) are known to have sensory-perceptual processing deficits that weaken their abilities to attend and perceive social stimuli in daily living contexts. Since daily social episodes consist of subtle dynamic changes in social information, any failure to attend to or process subtle human nonverbal cues, such as facial expression, postures, and gestures, might lead to inappropriate social interaction. Traditional behavioral rating scales or assessment tools based on static social scenes have limitations in capturing the moment-to-moment changes in social scenarios. An eye-tracking assessment, which can be administered in a video-based mode, is therefore preferred, to augment clinical observation. In this study, using the single-case comparison design, the eye-tracking data of three participants, a child with autism spectrum disorder (ASD), another with comorbid attention deficit-hyperactive disorder (ADHD), and a neurotypical control, are captured while they view a video of social scenarios. The eye-tracking experiment has helped answer the research question: How does social attention differ between the three participants? By predefining areas of interest (AOIs), their visual attention on relevant or irrelevant social stimuli, how fast each participant attends to the first social stimuli appearing in the videos, for how long each participant continues to attend to those stimuli within the AOIs, and the gaze shifts between multiple social stimuli appearing concurrently in the same social scene are captured, compared, and analyzed in a video-based eye-tracking experiment.

This is a qualitative comparative case study analysis of eye-tracking data on the first moments of social video scenes as viewed by three participants: one with autism spectrum disorder, one with comorbid attention deficit-hyperactive disorder, and one neurotypical control.

Children with autism spectrum disorders (ASD) are known to have sensory-perceptual processing deficits that weaken their abilities to attend and perceive ...

Simultaneous Eye Tracking and Single-Neuron Recordings in Human Epilepsy Patients

Intracranial recordings from patients with intractable epilepsy provide a unique opportunity to study the activity of individual human neurons during active behavior. An important tool for quantifying behavior is eye tracking, which is an indispensable tool for studying visual attention. However, eye tracking is challenging to use concurrently with invasive electrophysiology and this approach has consequently been little used. Here, we present a proven experimental protocol to conduct single-neuron recordings with simultaneous eye tracking in humans. We describe how the systems are connected and the optimal settings to record neurons and eye movements. To illustrate the utility of this method, we summarize results that were made possible by this setup. This data shows how using eye tracking in a memory-guided visual search task allowed us to describe a new class of neurons called target neurons, whose response was reflective of top-down attention to the current search target. Lastly, we discuss the significance and solutions to potential problems of this setup. Together, our protocol and results suggest that single-neuron recordings with simultaneous eye tracking in humans are an effective method to study human brain function. It provides a key missing link between animal neurophysiology and human cognitive neuroscience.

We describe a method to conduct single-neuron recordings with simultaneous eye tracking in humans. We demonstrate the utility of this method and illustrate how we used this approach to obtain neurons in the human medial temporal lobe that encode targets of a visual search.

Intracranial recordings from patients with intractable epilepsy provide a unique opportunity to study the activity of individual human neurons during ...

Investigating the Deployment of Visual Attention Before Accurate and Averaging Saccades via Eye Tracking and Assessment of Visual Sensitivity

This experimental protocol was designed to investigate whether visual attention is obligatorily deployed at the endpoint of saccades. To this end, we recorded the eye position of human participants engaged in a saccade task via eye tracking and assessed visual orientation discrimination performance at various locations during saccade preparation. Importantly, instead of using a single saccade target paradigm for which the saccade endpoint typically coincides roughly with the target, this protocol comprised the presentation of two nearby saccade targets, leading to a distinct spatial dissociation between target locations and saccade endpoint on a substantial number of trials. The paradigm allowed us to compare presaccadic visual discrimination performance at the endpoint of accurate saccades (landing at one of the saccade targets) and of averaging saccades (landing at an intermediate location in between the two targets). We observed a selective enhancement of visual sensitivity at the endpoint of accurate saccades but not at the endpoint of averaging saccades. Rather, before the execution of averaging saccades, visual sensitivity was equally enhanced at both targets, suggesting that saccade averaging follows from unresolved attentional selection among the saccade targets. These results argue against a mandatory coupling between visual attention and saccade programming based on a direct measure of presaccadic visual sensitivity rather than saccadic reaction times, which have been used in other protocols to draw similar conclusions. While our protocol provides a useful framework to investigate the relationship between visual attention and saccadic eye movements at the behavioral level, it can also be combined with electrophysiological measures to extend insights at the neuronal level.

This experimental protocol combined eye tracking and the assessment of presaccadic visual sensitivity in a dual task paradigm, consisting of a free choice saccade task and a visual discrimination task, to investigate the deployment of visual spatial attention before both, accurate and averaging saccades.

This experimental protocol was designed to investigate whether visual attention is obligatorily deployed at the endpoint of saccades. To this end, we ...

Defining the Role Of Language in Infants' Object Categorization with Eye-tracking Paradigms

Assessing infant category learning is a challenging but vital aspect of studying infant cognition. By employing a familiarization-test paradigm, we straightforwardly measure infants&#8217; success in learning a novel category while relying only on their looking behavior. Moreover, the paradigm can directly measure the impact of different auditory signals on the infant categorization across a range of ages. For instance, we assessed how 2-year-olds learn categories in a variety of labeling environments: in our task, 2-year-olds successfully learned categories when all exemplars were labeled&#160;or the first two exemplars were labeled, but they failed to categorize when no exemplars were labeled or only the final two exemplars were labeled. To determine infants&#8217; success in such tasks, researchers can examine both the overall preference displayed by infants in each condition and infants&#8217; pattern of looking over the course of the test phase, using an eye-tracker to provide fine-grained time-course data. Thus, we present a powerful paradigm for identifying the role of language, or any auditory signal, in infants&#8217; object category learning.

Here we present a protocol for familiarization-test paradigms which provide a direct test of infant categorization and help to define the role of language in early category learning.

Assessing infant category learning is a challenging but vital aspect of studying infant cognition. By employing a familiarization-test paradigm, we ...