A Methodology for Capturing Joint Visual Attention Using Mobile Eye-Trackers

Summary

Using multimodal sensors is a promising way to understand the role of social interactions in educational settings. This paper describes a methodology for capturing joint visual attention from colocated dyads using mobile eye-trackers.

Abstract

With the advent of new technological advances, it is possible to study social interactions at a microlevel with unprecedented accuracy. High frequency sensors, such as eye-trackers, electrodermal activity wristbands, EEG bands, and motion sensors provide observations at the millisecond level. This level of precision allows researchers to collect large datasets on social interactions. In this paper, I discuss how multiple eye-trackers can capture a fundamental construct in social interactions, joint visual attention (JVA). JVA has been studied by developmental psychologists to understand how children acquire language, learning scientists to understand how small groups of learners work together, and social scientists to understand interactions in small teams. This paper describes a methodology for capturing JVA in colocated settings using mobile eye-trackers. It presents some empirical results and discusses implications of capturing microobservations to understand social interactions.

Introduction

JVA has been extensively studied over the last century, especially by developmental psychologists studying language acquisition. It was quickly established that joint attention is more than just a way to learn words but rather a precursor to children's theories of mind¹. Thus, it plays a significant role in many social processes, such as communicating with others, collaborating, and developing empathy. Autistic children, for instance, lack the ability to coordinate their visual attention with their caregivers, which is associated with significant social impairments². Humans need joint attention to become functional m....

Protocol

1. Participant Screening

1. Ensure that participants with normal or corrected-to-normal vision are recruited. Because participants will be asked to wear a mobile eye-tracker, they can wear contact lenses but not regular eyeglasses.

2. Preparation for the Experiment

1. Eye-tracking devices
   1. Use any mobile eye-tracker capable of capturing eye movement in real world environments.
      NOTE: The mobile eye-trackers used here were two Tobii Pro Glasses 2 (see .......

Discussion

The methodology described in this paper provides a rigorous way to capture JVA in colocated dyads. With the emergence of affordable sensing technology and improved computer vision algorithms, it is now possible to study collaborative interactions with an accuracy that was previously unavailable. This methodology leverages fiducial markers disseminated in the environment and uses homographies as a way to remap participants' gazes onto a common plane. This allows researchers to rigorously study JVA in colocated groups........

Materials

Name	Company	Catalog Number	Comments
Tobii Glasses 2	Tobii	N/A	https://www.tobiipro.com/product-listing/tobii-pro-glasses-2/
Fiducial markers	Chili lab – EPFL, Switzerland	N/A	https://github.com/chili-epfl/chilitags