Name: visualizing visual adaptation
Uploaded: 2017-04-24T13:00:00
Description: Watch this Scientific Journal Video about Visualizing Visual Adaptation at JoVE.com

Supported by National Institutes of Health (NIH) grant EY-10834.

NOTE: The protocol illustrated uses a program that allows one to select images and then adapt them using options selected by different drop-down menus.
1. Select the Image to Adapt
<ol>
	<li>Click on the image and browse for the filename of the image to work with. Observe the original image in the upper left pane.</li>
</ol>
2. Specify the Stimulus and the Observer
<ol>
	<li>Click the &#34;format&#34; menu to choose how to represent the image and the observer.</li>
...

The illustrated protocol demonstrates how the effects of adaptation to a change in the environment or the observer can be portrayed in images. The form this portrayal takes will depend on the assumptions made for the model &#8212; for example, how color is encoded, and how the encoding mechanisms respond and adapt. Thus the most important step is deciding on the model for color vision &#8212; for example what the properties of the hypothesized channels are, and how they are assumed to adapt. The other important steps are...

What might the world look like to others, or to ourselves as we change? Answers to these questions are fundamentally important for understanding the nature and mechanisms of perception and the consequences of both normal and clinical variations in sensory coding. A wide variety of techniques and approaches have been developed to simulate how images might appear to individuals with different visual sensitivities. For example, these include simulations of the colors that can be discriminated by different types of color deficiencies1,2,3,4, the spatial and chromatic differences that can be resolved by infants or older observers5,6,7,8,9, how images appear in peripheral vision10, and the consequences of optical errors or disease11,12,13,14. They have also been applied to visualize the discriminations that are possible for other species15,16,17. Typically, such simulations use measurements of the sensitivity losses in different populations to filter an image and thus reduce or remove the structure they have difficulty seeing. For instance, common forms of color blindness reflect a loss of one of the two photoreceptors sensitive to medium or long wavelengths, and images filtered to remove their signals typically appear devoid of &#34;reddish-greenish&#34; hues1. Similarly, infants have poorer acuity, and thus the images processed for their reduced spatial sensitivity appear blurry5. These techniques provide invaluable illustrations of what one person can see that another may not. However, they do not &#8212; and often are not intended to &#8212; portray the actual perceptual experience of the observer, and in some cases may misrepresent the amount and types of information available to the observer.
This article describes a novel technique developed to simulate differences in visual experience which incorporates a fundamental characteristic of visual coding &#8212; adaptation18,19. All sensory and motor systems continuously adjust to the context they are exposed to. A pungent odor in a room quickly fades, while vision accommodates to how bright or dim the room is. Importantly, these adjustments occur for almost any stimulus attribute, including &#34;high-level&#34; perceptions such as the characteristics of someone&#39;s face20,21 or their voice22,23, as well as calibrating the motor commands made when moving the eyes or reaching for an object24,25. In fact, adaptation is likely an essential property of almost all neural processing. This paper illustrates how to incorporate these adaptation effects into simulations of the appearance of images, by basically &#34;adapting the image&#34; to predict how it would appear to a specific observer under a specific state of adaptation26,27,28,29. Many factors can alter the sensitivity of an observer, but adaptation can often compensate for important aspects of these changes, so that the sensitivity losses are less conspicuous than would be predicted without assuming that the system adapts. Conversely, because adaptation adjusts sensitivity according to the current stimulus context, these adjustments are also important to incorporate for predicting how much perception might vary when the environment varies.
The following protocol illustrates the technique by adapting the color content of images. Color vision has the advantage that the initial neural stages of color coding are relatively well understood, as are the patterns of adaptation30. The actual mechanisms and adjustments are complex and varied, but the main consequences of adaptation can be captured using a simple and conventional two-stage model (Figure 1a). In the first stage, color signals are initially encoded by three types of cone photoreceptors that are maximally sensitive to short, medium or long wavelengths (S, M, and L cones). In the second stage, the signals from different cones are combined within post-receptoral cells to form &#34;color-opponent&#34; channels that receive antagonistic inputs from the different cones (and thus convey &#34;color&#34; information), and &#34;non-opponent&#34; channels that sum together the cone inputs (thus coding &#34;brightness&#34; information). Adaptation occurs at both stages, and adjusts to two different aspects of the color &#8212; the mean (in the cones) and the variance (in post-receptoral channels)30,31. The goal of the simulations is to apply these adjustments to the model mechanisms and then render the image from their adapted outputs.
The process of adapting images involves six primary components. These are 1) choosing the images; 2) choosing the format for the image spectra; 3) defining the change in color of the environment; 4) defining the change in the sensitivity of the observer; 5) using the program to create the adapted images; and 6) using the images to evaluate the consequences of the adaptation. The following considers each of these steps in detail. The basic model and mechanism responses are illustrated in Figure 1, while Figures 2 - 5 show examples of images rendered with the model.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Computer</td>
		</tr>
		<tr>
			<td>Images to adapt</td>
		</tr>
		<tr>
			<td>Programming language (e.g. Visual Basic or Matlab)</td>
		</tr>
		<tr>
			<td>Program for processing the images</td>
		</tr>
		<tr>
			<td>Observer spectral sensitivities (for applications involving observer-specific adaptation)</td>
		</tr>
		<tr>
			<td>Device emmission spectra (for device-dependent applications)</td>
		</tr>
	</tbody>
</table>

visualizing visual adaptation

Many techniques have been developed to visualize how an image would appear to an individual with a different visual sensitivity: e.g., because of optical or age differences, or a color deficiency or disease. This protocol describes a technique for incorporating sensory adaptation into the simulations. The protocol is illustrated with the example of color vision, but is generally applicable to any form of visual adaptation. The protocol uses a simple model of human color vision based on standard and plausible assumptions about the retinal and cortical mechanisms encoding color and how these adjust their sensitivity to both the average color and range of color in the prevailing stimulus. The gains of the mechanisms are adapted so that their mean response under one context is equated for a different context. The simulations help reveal the theoretical limits of adaptation and generate &#34;adapted images&#34; that are optimally matched to a specific environment or observer. They also provide a common metric for exploring the effects of adaptation within different observers or different environments. Characterizing visual perception and performance with these images provides a novel tool for studying the functions and consequences of long-term adaptation in vision or other sensory systems.

Figures 2 - 4 illustrate the adaptation simulations for changes in the observer or the environment. Figure 2 compares the predicted appearance of Cezanne&#39;s Still Life with Apples for a younger and older observer who differ only in the density of the lens pigment28. The original image as seen through the younger eye (Figure 2a) appears much yellower and dimmer through the more densely pigmented lens (<s...

Watch this Scientific Journal Video about Visualizing Visual Adaptation at JoVE.com

Visualizing Visual Adaptation

This article describes a novel method for simulating and studying adaptation in the visual system.

Many techniques have been developed to visualize how an image would appear to an individual with a different visual sensitivity: e.g., because of optical ...

The overall goal of this software program is to simulate the effects of adaptation on visual perception by using a model of how the human visual system adapts to encode color. This model calculates how that encoding changes when the same observer is adapted to different color environments, or when different observers are adapted to the same environment. This method addresses key questions in cognitive science and neuroscience, such as why sensory systems adapt and what effects adaptation has on perception.The main advantage of this technique is that it incorporates plausible assumptions about adaptation and color vision and uses these to more reliably predict color perception across different environments and observers. Begin by selecting the desired image to work with and observe it pop up in the upper left pane. Then, click the format menu to choose how to represent the image and the observer.Click on standard observer to model a standard or average observer adapting to a specific color distribution. Next, click on Natural Spectra or RGB to approximate actual light spectra or the red, green, blue pixel spectra of the image on the monitor. Begin by adapting the same observer to different environments, such as the colors of the current image versus the color distribution typical of a natural outdoor environment.To set these environments, select the reference and test environments from the dropdown menus;then click the reference menu to choose the starting environment, which is the environment the observer is assumed to be adapted to while viewing the original image. Next, click the test menu to choose the new environment that the observer will become adapted to. Click the Adapt button and observe two new images filtered for cone adaptation and adapted for contrast adaptation.Alternatively, to adapt different observers to the same environment, such as the spectral sensitivity of a younger or older observer;select the spectral sensitivity of the observer and click on the lens menu to choose the density of the lens pigment. Then, click on the macular menu to similarly select the density of the macular pigment;and click on the Cones menu to choose between observers with normal trichromacy or different types of anomalous trichromacy. Again, click the Adapt button and observe three new images on the screen, this time showing the original image, the image filtered for the new sensitivity, and the image after adapting to the new sensitivity.Finally, click the Save Images button to save the three images. This protocol demonstrates how the effects of visual adaptation to a change in the environment or the observer can be visualized in images. As an illustration of how adaptation adjusts to the changes in the observer, the model simulated the consequences of lens aging by comparing the predicted appearance of an image as seen through the eyes of a younger or older observer who differ only in the density of the lens pigment.As an illustration of how adaptation adjusts to a change in the environment, the model was used to simulate how we might adapt to a low contrast, hazy environment by adjusting color coding to match the encoding expected for a typical higher contrast environment. This technique can be applied to any image ensembles or observers with different spectral sensitivities. It can also be generalized to examine adaptation and other characteristics of images, such as their spectral properties.The rendered images model the theoretical limits of long-term adaptation and can be used to examine the function of adaptation by testing what subjects could perceive in the adapted images that they could not perceive in the original images. The rendered images also provide more realistic simulations of the effects of the sensitivity change on our visual perception and show how adaptation tends to discount or compensate for these sensitivity changes.

Research

Education

JoVE Journal

JoVE Core

Biology

Behavior

Human Biology

Sensory Systems

SCIENTISTS IN ACTION

Topographical Estimation of Visual Population Receptive Fields by fMRI

Visual cortex is retinotopically organized so that neighboring populations of cells map to neighboring parts of the visual field. Functional magnetic resonance imaging allows us to estimate voxel-based population receptive fields (pRF), i.e., the part of the visual field that activates the cells within each voxel. Prior, direct, pRF estimation methods1 suffer from certain limitations: 1) the pRF model is chosen a-priori and may not fully capture the actual pRF shape, and 2) pRF centers are prone to mislocalization near the border of the stimulus space. Here a new topographical pRF estimation method2 is proposed that largely circumvents these limitations. A linear model is used to predict the Blood Oxygen Level-Dependent (BOLD) signal by convolving the linear response of the pRF to the visual stimulus with the canonical hemodynamic response function. PRF topography is represented as a weight vector whose components represent the strength of the aggregate response of voxel neurons to stimuli presented at different visual field locations. The resulting linear equations can be solved for the pRF weight vector using ridge regression3, yielding the pRF topography. A pRF model that is matched to the estimated topography can then be chosen post-hoc, thereby improving the estimates of pRF parameters such as pRF-center location, pRF orientation, size, etc. Having the pRF topography available also allows the visual verification of pRF parameter estimates allowing the extraction of various pRF properties without having to make a-priori assumptions about the pRF structure. This approach promises to be particularly useful for investigating the pRF organization of patients with disorders of the visual system.

It is important to obtain unbiased estimates of visual population receptive fields (pRFs) by functional magnetic resonance imaging. We use mild regularization constraints to estimate pRF topography without a-priori assumptions about pRF shape, allowing us to choose specific pRF models post-hoc. This is particularly advantageous in subjects with visual-pathway lesions.

Visual cortex is retinotopically organized so that neighboring populations of cells map to neighboring parts of the visual field. Functional magnetic ...

Methods to Test Visual Attention Online

Online data collection methods have particular appeal to behavioral scientists because they offer the promise of much larger and much more representative data samples than can typically be collected on college campuses. However, before such methods can be widely adopted, a number of technological challenges must be overcome &#8211; in particular in experiments where tight control over stimulus properties is necessary. Here we present methods for collecting performance data on two tests of visual attention. Both tests require control over the visual angle of the stimuli (which in turn requires knowledge of the viewing distance, monitor size, screen resolution, etc.) and the timing of the stimuli (as the tests involve either briefly flashed stimuli or stimuli that move at specific rates). Data collected on these tests from over 1,700 online participants were consistent with data collected in laboratory-based versions of the exact same tests. These results suggest that with proper care, timing/stimulus size dependent tasks can be deployed in web-based settings.

To replicate laboratory settings, online data collection methods for visual tasks require tight control over stimulus presentation. We outline methods for the use of a web application to collect performance data on two tests of visual attention.

Online data collection methods have particular appeal to behavioral scientists because they offer the promise of much larger and much more representative ...

Visualizing Motion Patterns in Acupuncture Manipulation

Acupuncture manipulation varies widely among practitioners in clinical settings, and it is difficult to teach novice students how to perform acupuncture manipulation techniques skillfully. The Acupuncture Manipulation Education System (AMES) is an open source software system designed to enhance acupuncture manipulation skills using visual feedback. Using a phantom acupoint and motion sensor, our method for acupuncture manipulation training provides visual feedback regarding the actual movement of the student's acupuncture manipulation in addition to the optimal or intended movement, regardless of whether the manipulation skill is lifting, thrusting, or rotating. Our results show that students could enhance their manipulation skills by training using this method. This video shows the process of manufacturing phantom acupoints and discusses several issues that may require the attention of individuals interested in creating phantom acupoints or operating this system.

Here, we present a protocol for using the Acupuncture Manipulation Education System (AMES) in the training of acupuncture manipulation skills using phantom acupoints.

Acupuncture manipulation varies widely among practitioners in clinical settings, and it is difficult to teach novice students how to perform acupuncture ...

Using a Split-belt Treadmill to Evaluate Generalization of Human Locomotor Adaptation

Understanding the mechanisms underlying locomotor learning helps researchers and clinicians optimize gait retraining as part of motor rehabilitation. However, studying human locomotor learning can be challenging. During infancy and childhood, the neuromuscular system is quite immature, and it is unlikely that locomotor learning during early stages of development is governed by the same mechanisms as in adulthood. By the time humans reach maturity, they are so proficient at walking that it is difficult to come up with a sufficiently novel task to study de novo locomotor learning. The split-belt treadmill, which has two belts that can drive each leg at a different speed, enables the study of both short- (i.e., immediate) and long-term (i.e., over minutes-days; a form of motor learning) gait modifications in response to a novel change in the walking environment. Individuals can easily be screened for previous exposure to the split-belt treadmill, thus ensuring that all experimental participants have no (or equivalent) prior experience. This paper describes a typical split-belt treadmill adaptation protocol that incorporates testing methods to quantify locomotor learning and generalization of this learning to other walking contexts. A discussion of important considerations for designing split-belt treadmill experiments follows, including factors like treadmill belt speeds, rest breaks, and distractors. Additionally, potential but understudied confounding variables (e.g., arm movements, prior experience) are considered in the discussion.

We describe a protocol for investigating human locomotor adaptation using the split-belt treadmill, which has two belts that can drive each leg at a different speed. We specifically focus on a paradigm designed to test the generalization of adapted locomotor patterns to different walking contexts (e.g., gait speeds, walking environments).

Understanding the mechanisms underlying locomotor learning helps researchers and clinicians optimize gait retraining as part of motor rehabilitation. ...

A Method to Study Adaptation to Left-Right Reversed Audition

An unusual sensory space is one of the effective tools to uncover the mechanism of adaptability of humans to a novel environment. Although most of the previous studies have used special spectacles with prisms to achieve unusual spaces in the visual domain, a methodology for studying the adaptation to unusual auditory spaces has yet to be fully established. This study proposes a new protocol to set-up, validate, and use a left-right reversed stereophonic system using only wearable devices, and to study the adaptation to left-right reversed audition with the help of neuroimaging. Although individual acoustic characteristics are not yet implemented, and slight spillover of unreversed sounds is relatively uncontrollable, the constructed apparatus shows high performance in a 360&#176; sound source localization coupled with hearing characteristics with little delay. Moreover, it looks like a mobile music player and enables a participant to focus on daily life without arousing curiosity or drawing attention of other individuals. Since the effects of adaptation were successfully detected at the perceptual, behavioral, and neural levels, it is concluded that this protocol provides a promising methodology for studying adaptation to left-right reversed audition, and is an effective tool for uncovering the adaptability of humans to a novel environments in the auditory domain.

The present study proposes a protocol to investigate the adaptation to left-right reversed audition achieved only by wearable devices, using neuroimaging, which can be an effective tool for uncovering the adaptability of humans to a novel environment in the auditory domain.

An unusual sensory space is one of the effective tools to uncover the mechanism of adaptability of humans to a novel environment. Although most of the ...

Visual Classical Conditioning in Wood Ants

Several species of insects have become model systems for studying learning and memory formation. Although many studies focus on freely moving animals, studies implementing classical conditioning paradigms with harnessed insects have been important for investigating the exact cues that individuals learn and the neural mechanisms underlying learning and memory formation. Here we present a protocol for evoking visual associative learning in wood ants through classical conditioning. In this paradigm, ants are harnessed and presented with a visual cue (a blue cardboard), the conditional stimulus (CS), paired with an appetitive sugar reward, the unconditional stimulus (US). Ants perform a Maxilla-Labium Extension Reflex (MaLER), the unconditional response (UR), which can be used as a readout for learning. Training consists of 10 trials, separated by a 5-minute intertrial interval (ITI). Ants are also tested for memory retention 10 minutes or 1 hour after training. This protocol has the potential to allow researchers to analyze, in a precise and controlled manner, the details of visual memory formation and the neural basis of learning and memory formation in wood ants.

We present a protocol for the classical conditioning of harnessed ants that permits researchers to study visual learning and memory formation at a level of analysis not possible with freely moving individuals.

Several species of insects have become model systems for studying learning and memory formation. Although many studies focus on freely moving animals, ...

Investigating the Deployment of Visual Attention Before Accurate and Averaging Saccades via Eye Tracking and Assessment of Visual Sensitivity

This experimental protocol was designed to investigate whether visual attention is obligatorily deployed at the endpoint of saccades. To this end, we recorded the eye position of human participants engaged in a saccade task via eye tracking and assessed visual orientation discrimination performance at various locations during saccade preparation. Importantly, instead of using a single saccade target paradigm for which the saccade endpoint typically coincides roughly with the target, this protocol comprised the presentation of two nearby saccade targets, leading to a distinct spatial dissociation between target locations and saccade endpoint on a substantial number of trials. The paradigm allowed us to compare presaccadic visual discrimination performance at the endpoint of accurate saccades (landing at one of the saccade targets) and of averaging saccades (landing at an intermediate location in between the two targets). We observed a selective enhancement of visual sensitivity at the endpoint of accurate saccades but not at the endpoint of averaging saccades. Rather, before the execution of averaging saccades, visual sensitivity was equally enhanced at both targets, suggesting that saccade averaging follows from unresolved attentional selection among the saccade targets. These results argue against a mandatory coupling between visual attention and saccade programming based on a direct measure of presaccadic visual sensitivity rather than saccadic reaction times, which have been used in other protocols to draw similar conclusions. While our protocol provides a useful framework to investigate the relationship between visual attention and saccadic eye movements at the behavioral level, it can also be combined with electrophysiological measures to extend insights at the neuronal level.

This experimental protocol combined eye tracking and the assessment of presaccadic visual sensitivity in a dual task paradigm, consisting of a free choice saccade task and a visual discrimination task, to investigate the deployment of visual spatial attention before both, accurate and averaging saccades.

This experimental protocol was designed to investigate whether visual attention is obligatorily deployed at the endpoint of saccades. To this end, we ...

Integrating Visual Psychophysical Assays within a Y-Maze to Isolate the Role that Visual Features Play in Navigational Decisions

Collective animal behavior arises from individual motivations and social interactions that are critical for individual fitness. Fish have long inspired investigations into collective motion, specifically, their ability to integrate environmental and social information across ecological contexts. This demonstration illustrates techniques used for quantifying behavioral responses of fish, in this case, Golden Shiner (Notemigonus crysoleucas), to visual stimuli using computer visualization and digital image analysis. Recent advancements in computer visualization allow for empirical testing in the lab where visual features can be controlled and finely manipulated to isolate the mechanisms of social interactions. The purpose of this method is to isolate visual features that can influence the directional decisions of the individual, whether solitary or with groups. This protocol provides specifics on the physical Y-maze domain, recording equipment, settings and calibrations of the projector and animation, experimental steps and data analyses. These techniques demonstrate that computer animation can elicit biologically-meaningful responses. Moreover, the techniques are easily adaptable to test alternative hypotheses, domains, and species for a broad range of experimental applications. The use of virtual stimuli allows for the reduction and replacement of the number of live animals required, and consequently reduces laboratory overhead.
This demonstration tests the hypothesis that small relative differences in the movement speeds (2 body lengths per second) of virtual conspecifics will improve the speed and accuracy with which shiners follow the directional cues provided by the virtual silhouettes. Results show that shiners directional decisions are significantly affected by increases in the speed of the visual cues, even in the presence of background noise (67% image coherency). In the absence of any motion cues, subjects chose their directions at random. The relationship between decision speed and cue speed was variable and increases in cue speed had a modestly disproportionate influence on directional accuracy.

Here, we present a protocol to demonstrate a behavioral assay that quantifies how alternative visual features, such as motion cues, influence directional decisions in fish. Representative data are presented on the speed and accuracy where Golden Shiner (Notemigonus crysoleucas) follow virtual fish movements.

Collective animal behavior arises from individual motivations and social interactions that are critical for individual fitness. Fish have long inspired ...

Applying Incongruent Visual-Tactile Stimuli during Object Transfer with Vibro-Tactile Feedback

The application of incongruent sensory signals that involves disrupted tactile feedback is rarely explored, specifically with the presence of vibrotactile feedback (VTF). This protocol aims to test the effect of VTF on the response to incongruent visual-tactile stimuli. The tactile feedback is acquired by grasping a block and moving it across a partition. The visual feedback is a real-time virtual presentation of the moving block, acquired using a motion capture system. The congruent feedback is the reliable presentation of the movement of the block, so that the subject feels that the block is grasped and see it move along with the path of the hand. The incongruent feedback appears as the movement of the block diverts from the actual movement path, so that it seems to drop from the hand when it is actually still held by the subject, thereby contradicting the tactile feedback. Twenty subjects (age 30.2 &#177; 16.3) repeated 16 block transfers, while their hand was hidden. These were repeated with VTF and without VTF (total of 32 block transfers). Incongruent stimuli were presented randomly twice within the 16 repetitions in each condition (with and without VTF). Each subject was asked to rate the difficulty level of performing the task with and without the VTF. There were no statistically significant differences in the length of the hand paths and durations between transfers recorded with congruent and incongruent visual-tactile signals &#8211; with and without the VTF. The perceived difficulty level of performing the task with the VTF significantly correlated with the normalized path length of the block with VTF (r = 0.675, p = 0.002). This setup is used to quantify the additive or reductive value of VTF during motor function that involves incongruent visual-tactile stimuli. Possible applications are prosthetics design, smart sport-wear, or any other garments that incorporate VTF.

We present a protocol to apply incongruent visual-tactile stimuli during an object transfer task. Specifically, during block transfers, performed while the hand is hidden, a virtual presentation of the block shows random occurrences of false block drops. The protocol also describes adding vibrotactile feedback while performing the motor task.

The application of incongruent sensory signals that involves disrupted tactile feedback is rarely explored, specifically with the presence of vibrotactile ...

Using Looming Visual Stimuli to Evaluate Mouse Vision

The visual system in the central nervous system processes diverse visual signals. Although the overall structure has been characterized from the retina through the lateral geniculate nucleus to the visual cortex, the system is complex. Cellular and molecular studies have been conducted to elucidate the mechanisms underpinning visual processing and, by extension, disease mechanisms. These studies may contribute to the development of artificial visual systems. To validate the results of these studies, behavioral vision testing is necessary. Here, we show that the looming stimulation experiment is a reliable mouse vision test that requires a relatively simple setup. The looming experiment was conducted in a large enclosure with a shelter in one corner and a computer monitor located on the ceiling. A CCD camera positioned next to the computer monitor served to observe mouse behavior. A mouse was placed in the enclosure for 10 minutes and allowed to acclimate to and explore the surroundings. Then, the monitor projected a program-derived looming stimulus 10 times. The mouse responded to the stimuli either by freezing or by fleeing to the hiding place. The mouse&#39;s behavior before and after the looming stimuli was recorded, and the video was analyzed using motion tracking software. The velocity of the mouse movement significantly changed after the looming stimuli. In contrast, no reaction was observed in blind mice. Our results demonstrate that the simple looming experiment is a reliable test of mouse vision.

To examine mouse vision, we conducted a looming test. Mice were placed in a large square arena with a monitor on its ceiling. The looming visual stimulus consistently evoked freezing or flight reactions in the mice.

The visual system in the central nervous system processes diverse visual signals. Although the overall structure has been characterized from the retina ...