Name: methods to test visual attention online
Uploaded: 2015-02-19T14:00:00
Description: Watch this Scientific Journal Video about Methods to Test Visual Attention Online at JoVE.com

The authors have nothing to disclose.

The protocol was approved by the institutional review board at the University of Wisconsin-Madison. The following steps have been written as a guide for programmers to replicate the automated process of the web application described.
1. Login Participant
<ol>
	<li>Instruct the participant to use an internet-enabled computer and navigate to the web application using an HTML5 compatible browser: <a href="http://brainandlearning.org/jove" target="_blank">http://brainandlearning.org/jove</a>. Ha...

<ol>
	<li>Henrich, J., Heine, S. J., Norenzayan, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+weirdest+people+in+the+world.">The weirdest people in the world.</a> The Behavioral And Brain Sciences. 33, 61-135 (2010).</li><li>Buhrmester, M., Kwang, T., Gosling, S. D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Amazon's+Mechanical+Turk:+A+new+source+of+inexpensive,+yet+high-quality,+data.">Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data.</a> Perspectives on Psychological Science. 6 (1), 3-5 (2011).</li><li>Goodman, J. K., Cryder, C. E., Cheema, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Data+collection+in+a+flat+world:+the+strengths+and+weaknesses+of+mechanical+turk+samples.">Data collection in a flat world: the strengths and weaknesses of mechanical turk samples.</a> Journal of Behavioral Decision Making. 26 (3), 213-224 (2013).</li><li>Mason, W., Suri, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Conducting+behavioral+research+on+Amazon's+Mechanical+Turk.">Conducting behavioral research on Amazon's Mechanical Turk.</a> Behavior Research Methods. 44 (1), 1-23 (2012).</li><li>Crump, M. J., McDonnell, J. V., Gureckis, T. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Evaluating+Amazon's+Mechanical+Turk+as+a+tool+for+experimental+behavioral+research.">Evaluating Amazon's Mechanical Turk as a tool for experimental behavioral research.</a> PLoS One. 8, e57410 (1371).</li><li>Lupyan, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+difficulties+of+executing+simple+algorithms:+why+brains+make+mistakes+computers+don't.">The difficulties of executing simple algorithms: why brains make mistakes computers don't.</a> Cognition. 129, 615-636 (2013).</li><li>Ball, K., Owsley, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+useful+field+of+view+test:+a+new+technique+for+evaluating+age-related+declines+in+visual+function.">The useful field of view test: a new technique for evaluating age-related declines in visual function.</a> J Am Optom Assoc. 64 (1), 71-79 (1993).</li><li>Pylyshyn, Z. W., Storm, R. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Tracking+multiple+independent+targets:+Evidence+for+a+parallel+tracking+mechanism.">Tracking multiple independent targets: Evidence for a parallel tracking mechanism.</a> Spatial Vision. 3 (3), 179-197 (1988).</li><li>Dye, M. W. G., Bavelier, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Differential+development+of+visual+attention+skills+in+school-age+children.">Differential development of visual attention skills in school-age children.</a> Vision Research. 50 (4), 452-459 (2010).</li><li>Green, C. S., Bavelier, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Enumeration+versus+multiple+object+tracking:+The+case+of+action+video+game+players.">Enumeration versus multiple object tracking: The case of action video game players.</a> Cognition. 101 (1), 217-245 (2006).</li><li>Chengalur, S. N., Rodgers, S. H., Bernard, T. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Chapter+1.+Kodak+Company.+Ergonomics+Design+Philosophy.">Chapter 1. Kodak Company. Ergonomics Design Philosophy.</a> Kodak&#8217;s Ergonomic Design for People at Work. , (2004).</li><li>Elze, T., Bex, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=P1-7:+Modern+display+technology+in+vision+science:+Assessment+of+OLED+and+LCD+monitors+for+visual+experiments.">P1-7: Modern display technology in vision science: Assessment of OLED and LCD monitors for visual experiments.</a> i-Perception. 3 (9), 621 (2012).</li><li>Elze, T., Tanner, T. G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Temporal+Properties+of+Liquid+Crystal+Displays:+Implications+for+Vision+Science+Experiments.">Temporal Properties of Liquid Crystal Displays: Implications for Vision Science Experiments.</a> PLoS One. 7 (9), e44048 (2012).</li></ol>

Online data collection has a number of advantages over standard laboratory-based data collection. These include the potential to sample far more representative populations than the typical college undergraduate pool utilized in the field, and the ability to obtain far greater sample sizes in less time than it takes to obtain sample sizes that are an order of magnitude smaller in the lab1-6 (e.g., the data points collected from 1,700+ participants in the current paper were obtained in less than one wee...

Over the past five years there has been a surge of interest in the use of online behavioral data collection methods. While the vast majority of publications in the domain of psychology have utilized potentially non-representative subject populations1 (i.e., primarily college undergraduates) and often reasonably small sample sizes as well (i.e., typically in the range of tens of subjects), online methods offer the promise of far more diverse and larger samples. For instance, Amazon&#8217;s Mechanical Turk service has been the subject of a number of recent studies, both describing the characteristics of the &#8220;worker&#8221; population and the use of this population in behavioral research2-6.
However, one significant concern related to such methods is the relative lack of control over critical stimulus variables. For example, in most visual psychophysics tasks, stimuli are described in terms of visual angle. The calculation of visual angles requires precise measurements of viewing distance, screen size, and screen resolution. While these parameters are trivial to measure and control in a lab setting (where there is a known monitor and participants view stimuli while in a chin rest placed a known distance from the monitor), the same is not true of online data collection. In an online environment, not only will participants inevitably use a wide variety of monitors of different sizes with different software settings, they also may not have easy access to rulers/tape measures that would allow them to determine their monitor size or have the knowledge necessary to determine their software and hardware settings (e.g., refresh rate, resolution).
Here we describe a set of methods to collect data on two well-known tests of visual attention &#8211; the Useful Field of View (UFOV) paradigm7 and the multiple object tracking (MOT) task8 &#8211; while avoiding as much as possible the sources of variability that are inherent in online measurements. These tasks can be run by any participant with an internet connection and an HTML5 compatible browser. Participants who do not know their screen size are walked through a measurement process utilizing commonly available items of standard size (i.e., credit card/CD &#8211; see Figure 1).
Data on these two tasks were collected from over 1,700 participants in a Massive Online Open Course. Average performance of this online sample was highly consistent with results obtained in tightly controlled laboratory-based measures of the exact same tasks9,10. Our results are thus consistent with the growing body of literature demonstrating the efficacy of online data collection methods, even in tasks that require specific control over viewing conditions.

<table border="0" cellpadding="0" cellspacing="0" width="573">
	<tbody>
		<tr height="21">
			<td height="21" style="height:21px;width:216px;">Name of Reagent/ Equipment</td>
			<td style="width:71px;">Company</td>
			<td style="width:119px;">Catalog Number</td>
			<td style="width:167px;">Comments/Description</td>
		</tr>
		<tr height="21">
			<td height="21" style="height:21px;width:216px;">Computer/tablet</td>
			<td style="width:71px;">N/A</td>
			<td style="width:119px;">N/A</td>
			<td>It must have an internet connection and an HTML5 compatible browser</td>
		</tr>
		<tr height="21">
			<td height="21" style="height:21px;width:216px;">CD or credit card</td>
			<td style="width:71px;">N/A</td>
			<td style="width:119px;">N/A</td>
			<td>May not be needed if participant already knows the monitor size</td>
		</tr>
	</tbody>
</table>

methods to test visual attention online

Online data collection methods have particular appeal to behavioral scientists because they offer the promise of much larger and much more representative data samples than can typically be collected on college campuses. However, before such methods can be widely adopted, a number of technological challenges must be overcome &#8211; in particular in experiments where tight control over stimulus properties is necessary. Here we present methods for collecting performance data on two tests of visual attention. Both tests require control over the visual angle of the stimuli (which in turn requires knowledge of the viewing distance, monitor size, screen resolution, etc.) and the timing of the stimuli (as the tests involve either briefly flashed stimuli or stimuli that move at specific rates). Data collected on these tests from over 1,700 online participants were consistent with data collected in laboratory-based versions of the exact same tests. These results suggest that with proper care, timing/stimulus size dependent tasks can be deployed in web-based settings.

Outlier Removal
A total of 1,779 participants completed the UFOV task. Of those, 32 participants had UFOV thresholds that were greater than 3 standard deviations from the mean, suggesting that they were unable to perform the task as instructed. As such, the UFOV data from these participants were removed from the final analysis, leaving a total of 1,747 participants.
Data were obtained from 1,746 participants for the MOT task. Two participants had mean accuracy scores t...

Watch this Scientific Journal Video about Methods to Test Visual Attention Online at JoVE.com

Methods to Test Visual Attention Online

To replicate laboratory settings, online data collection methods for visual tasks require tight control over stimulus presentation. We outline methods for the use of a web application to collect performance data on two tests of visual attention.

Online data collection methods have particular appeal to behavioral scientists because they offer the promise of much larger and much more representative ...

Research

JoVE Journal

Behavior

Methods to Explore the Influence of Top-down Visual Processes on Motor Behavior

Kinesthetic awareness is important to successfully navigate the environment. When we interact with our daily surroundings, some aspects of movement are ...

A Dual Task Procedure Combined with Rapid Serial Visual Presentation to Test Attentional Blink for Nontargets

When viewers search for targets in a rapid serial visual presentation (RSVP) stream, if two targets are presented within about 500 msec of each other, the ...

Visualizing Visual Adaptation

Many techniques have been developed to visualize how an image would appear to an individual with a different visual sensitivity: e.g., because of optical ...

Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments

This protocol describes how to conduct temporal-order experiments to measure visual processing speed and the attentional resource distribution. The ...

Central and Divided Visual Field Presentation of Emotional Images to Measure Hemispheric Differences in Motivated Attention

Two dominant theories on lateralized processing of emotional information exist in the literature. One theory posits that unpleasant emotions are processed ...

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

In a typical eye tracking study using the visual world paradigm, participants' eye movements to objects or pictures in the visual workspace are recorded ...

Gaze in Action: Head-mounted Eye Tracking of Children's Dynamic Visual Attention During Naturalistic Behavior

Young children's visual environments are dynamic, changing moment-by-moment as children physically and visually explore spaces and objects and interact ...

Enactive Phenomenological Approach to the Trier Social Stress Test: A Mixed Methods Point of View

The single Trier Social Stress Test (TSST) and the TSST for groups (TSST-G) are the most used protocols to experimentally induce psychosocial stress. ...

Spotlighting Customers' Visual Attention at the Stock, Shelf and Store Levels with the 3S Model

Several models of the in-store search process exist in the fields of retailing, marketing, and consumer-based research. The present article presents a new ...

Investigating the Deployment of Visual Attention Before Accurate and Averaging Saccades via Eye Tracking and Assessment of Visual Sensitivity

This experimental protocol was designed to investigate whether visual attention is obligatorily deployed at the endpoint of saccades. To this end, we ...