Name: visualizing visual adaptation
Uploaded: 2017-04-24T13:00:00
Description: Watch this Scientific Journal Video about Visualizing Visual Adaptation at JoVE.com

Supported by National Institutes of Health (NIH) grant EY-10834.

NOTE: The protocol illustrated uses a program that allows one to select images and then adapt them using options selected by different drop-down menus.
1. Select the Image to Adapt
<ol>
	<li>Click on the image and browse for the filename of the image to work with. Observe the original image in the upper left pane.</li>
</ol>
2. Specify the Stimulus and the Observer
<ol>
	<li>Click the &#34;format&#34; menu to choose how to represent the image and the observer.</li>
...

The illustrated protocol demonstrates how the effects of adaptation to a change in the environment or the observer can be portrayed in images. The form this portrayal takes will depend on the assumptions made for the model &#8212; for example, how color is encoded, and how the encoding mechanisms respond and adapt. Thus the most important step is deciding on the model for color vision &#8212; for example what the properties of the hypothesized channels are, and how they are assumed to adapt. The other important steps are...

What might the world look like to others, or to ourselves as we change? Answers to these questions are fundamentally important for understanding the nature and mechanisms of perception and the consequences of both normal and clinical variations in sensory coding. A wide variety of techniques and approaches have been developed to simulate how images might appear to individuals with different visual sensitivities. For example, these include simulations of the colors that can be discriminated by different types of color deficiencies1,2,3,4, the spatial and chromatic differences that can be resolved by infants or older observers5,6,7,8,9, how images appear in peripheral vision10, and the consequences of optical errors or disease11,12,13,14. They have also been applied to visualize the discriminations that are possible for other species15,16,17. Typically, such simulations use measurements of the sensitivity losses in different populations to filter an image and thus reduce or remove the structure they have difficulty seeing. For instance, common forms of color blindness reflect a loss of one of the two photoreceptors sensitive to medium or long wavelengths, and images filtered to remove their signals typically appear devoid of &#34;reddish-greenish&#34; hues1. Similarly, infants have poorer acuity, and thus the images processed for their reduced spatial sensitivity appear blurry5. These techniques provide invaluable illustrations of what one person can see that another may not. However, they do not &#8212; and often are not intended to &#8212; portray the actual perceptual experience of the observer, and in some cases may misrepresent the amount and types of information available to the observer.
This article describes a novel technique developed to simulate differences in visual experience which incorporates a fundamental characteristic of visual coding &#8212; adaptation18,19. All sensory and motor systems continuously adjust to the context they are exposed to. A pungent odor in a room quickly fades, while vision accommodates to how bright or dim the room is. Importantly, these adjustments occur for almost any stimulus attribute, including &#34;high-level&#34; perceptions such as the characteristics of someone&#39;s face20,21 or their voice22,23, as well as calibrating the motor commands made when moving the eyes or reaching for an object24,25. In fact, adaptation is likely an essential property of almost all neural processing. This paper illustrates how to incorporate these adaptation effects into simulations of the appearance of images, by basically &#34;adapting the image&#34; to predict how it would appear to a specific observer under a specific state of adaptation26,27,28,29. Many factors can alter the sensitivity of an observer, but adaptation can often compensate for important aspects of these changes, so that the sensitivity losses are less conspicuous than would be predicted without assuming that the system adapts. Conversely, because adaptation adjusts sensitivity according to the current stimulus context, these adjustments are also important to incorporate for predicting how much perception might vary when the environment varies.
The following protocol illustrates the technique by adapting the color content of images. Color vision has the advantage that the initial neural stages of color coding are relatively well understood, as are the patterns of adaptation30. The actual mechanisms and adjustments are complex and varied, but the main consequences of adaptation can be captured using a simple and conventional two-stage model (Figure 1a). In the first stage, color signals are initially encoded by three types of cone photoreceptors that are maximally sensitive to short, medium or long wavelengths (S, M, and L cones). In the second stage, the signals from different cones are combined within post-receptoral cells to form &#34;color-opponent&#34; channels that receive antagonistic inputs from the different cones (and thus convey &#34;color&#34; information), and &#34;non-opponent&#34; channels that sum together the cone inputs (thus coding &#34;brightness&#34; information). Adaptation occurs at both stages, and adjusts to two different aspects of the color &#8212; the mean (in the cones) and the variance (in post-receptoral channels)30,31. The goal of the simulations is to apply these adjustments to the model mechanisms and then render the image from their adapted outputs.
The process of adapting images involves six primary components. These are 1) choosing the images; 2) choosing the format for the image spectra; 3) defining the change in color of the environment; 4) defining the change in the sensitivity of the observer; 5) using the program to create the adapted images; and 6) using the images to evaluate the consequences of the adaptation. The following considers each of these steps in detail. The basic model and mechanism responses are illustrated in Figure 1, while Figures 2 - 5 show examples of images rendered with the model.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Computer</td>
		</tr>
		<tr>
			<td>Images to adapt</td>
		</tr>
		<tr>
			<td>Programming language (e.g. Visual Basic or Matlab)</td>
		</tr>
		<tr>
			<td>Program for processing the images</td>
		</tr>
		<tr>
			<td>Observer spectral sensitivities (for applications involving observer-specific adaptation)</td>
		</tr>
		<tr>
			<td>Device emmission spectra (for device-dependent applications)</td>
		</tr>
	</tbody>
</table>

visualizing visual adaptation

Many techniques have been developed to visualize how an image would appear to an individual with a different visual sensitivity: e.g., because of optical or age differences, or a color deficiency or disease. This protocol describes a technique for incorporating sensory adaptation into the simulations. The protocol is illustrated with the example of color vision, but is generally applicable to any form of visual adaptation. The protocol uses a simple model of human color vision based on standard and plausible assumptions about the retinal and cortical mechanisms encoding color and how these adjust their sensitivity to both the average color and range of color in the prevailing stimulus. The gains of the mechanisms are adapted so that their mean response under one context is equated for a different context. The simulations help reveal the theoretical limits of adaptation and generate &#34;adapted images&#34; that are optimally matched to a specific environment or observer. They also provide a common metric for exploring the effects of adaptation within different observers or different environments. Characterizing visual perception and performance with these images provides a novel tool for studying the functions and consequences of long-term adaptation in vision or other sensory systems.

Figures 2 - 4 illustrate the adaptation simulations for changes in the observer or the environment. Figure 2 compares the predicted appearance of Cezanne&#39;s Still Life with Apples for a younger and older observer who differ only in the density of the lens pigment28. The original image as seen through the younger eye (Figure 2a) appears much yellower and dimmer through the more densely pigmented lens (<s...

Watch this Scientific Journal Video about Visualizing Visual Adaptation at JoVE.com

Visualizing Visual Adaptation

This article describes a novel method for simulating and studying adaptation in the visual system.

Many techniques have been developed to visualize how an image would appear to an individual with a different visual sensitivity: e.g., because of optical ...

The overall goal of this software program is to simulate the effects of adaptation on visual perception by using a model of how the human visual system adapts to encode color. This model calculates how that encoding changes when the same observer is adapted to different color environments, or when different observers are adapted to the same environment. This method addresses key questions in cognitive science and neuroscience, such as why sensory systems adapt and what effects adaptation has on perception.The main advantage of this technique is that it incorporates plausible assumptions about adaptation and color vision and uses these to more reliably predict color perception across different environments and observers. Begin by selecting the desired image to work with and observe it pop up in the upper left pane. Then, click the format menu to choose how to represent the image and the observer.Click on standard observer to model a standard or average observer adapting to a specific color distribution. Next, click on Natural Spectra or RGB to approximate actual light spectra or the red, green, blue pixel spectra of the image on the monitor. Begin by adapting the same observer to different environments, such as the colors of the current image versus the color distribution typical of a natural outdoor environment.To set these environments, select the reference and test environments from the dropdown menus;then click the reference menu to choose the starting environment, which is the environment the observer is assumed to be adapted to while viewing the original image. Next, click the test menu to choose the new environment that the observer will become adapted to. Click the Adapt button and observe two new images filtered for cone adaptation and adapted for contrast adaptation.Alternatively, to adapt different observers to the same environment, such as the spectral sensitivity of a younger or older observer;select the spectral sensitivity of the observer and click on the lens menu to choose the density of the lens pigment. Then, click on the macular menu to similarly select the density of the macular pigment;and click on the Cones menu to choose between observers with normal trichromacy or different types of anomalous trichromacy. Again, click the Adapt button and observe three new images on the screen, this time showing the original image, the image filtered for the new sensitivity, and the image after adapting to the new sensitivity.Finally, click the Save Images button to save the three images. This protocol demonstrates how the effects of visual adaptation to a change in the environment or the observer can be visualized in images. As an illustration of how adaptation adjusts to the changes in the observer, the model simulated the consequences of lens aging by comparing the predicted appearance of an image as seen through the eyes of a younger or older observer who differ only in the density of the lens pigment.As an illustration of how adaptation adjusts to a change in the environment, the model was used to simulate how we might adapt to a low contrast, hazy environment by adjusting color coding to match the encoding expected for a typical higher contrast environment. This technique can be applied to any image ensembles or observers with different spectral sensitivities. It can also be generalized to examine adaptation and other characteristics of images, such as their spectral properties.The rendered images model the theoretical limits of long-term adaptation and can be used to examine the function of adaptation by testing what subjects could perceive in the adapted images that they could not perceive in the original images. The rendered images also provide more realistic simulations of the effects of the sensitivity change on our visual perception and show how adaptation tends to discount or compensate for these sensitivity changes.

Research

Education

JoVE Journal

JoVE Core

Biology

Behavior

Human Biology

Sensory Systems

SCIENTISTS IN ACTION

Topographical Estimation of Visual Population Receptive Fields by fMRI

Visual cortex is retinotopically organized so that neighboring populations of cells map to neighboring parts of the visual field. Functional magnetic ...

Methods to Test Visual Attention Online

Online data collection methods have particular appeal to behavioral scientists because they offer the promise of much larger and much more representative ...

Visualizing Motion Patterns in Acupuncture Manipulation

Acupuncture manipulation varies widely among practitioners in clinical settings, and it is difficult to teach novice students how to perform acupuncture ...

Using a Split-belt Treadmill to Evaluate Generalization of Human Locomotor Adaptation

Understanding the mechanisms underlying locomotor learning helps researchers and clinicians optimize gait retraining as part of motor rehabilitation. ...

A Method to Study Adaptation to Left-Right Reversed Audition

An unusual sensory space is one of the effective tools to uncover the mechanism of adaptability of humans to a novel environment. Although most of the ...

Visual Classical Conditioning in Wood Ants

Several species of insects have become model systems for studying learning and memory formation. Although many studies focus on freely moving animals, ...

Investigating the Deployment of Visual Attention Before Accurate and Averaging Saccades via Eye Tracking and Assessment of Visual Sensitivity

This experimental protocol was designed to investigate whether visual attention is obligatorily deployed at the endpoint of saccades. To this end, we ...

Integrating Visual Psychophysical Assays within a Y-Maze to Isolate the Role that Visual Features Play in Navigational Decisions

Collective animal behavior arises from individual motivations and social interactions that are critical for individual fitness. Fish have long inspired ...

Using Looming Visual Stimuli to Evaluate Mouse Vision

The visual system in the central nervous system processes diverse visual signals. Although the overall structure has been characterized from the retina ...

A Methodology for Capturing Joint Visual Attention Using Mobile Eye-Trackers

With the advent of new technological advances, it is possible to study social interactions at a microlevel with unprecedented accuracy. High frequency ...