Object Substitution Masking

Source: Laboratory of Jonathan Flombaum—Johns Hopkins University

Visual masking is a term used by perceptual scientists to refer to a wide range of phenomena in which in an image is presented but not perceived by an observer because of the presentation of a second image. There are several different kinds of masking, many of them relatively intuitive and unsurprising. But one surprising and important type of masking is called Object Substitution Masking. It has been a focus of research in vision science since it was discovered, relatively recently, around 1997 by Enns and Di Lollo.¹

This video will demonstrate standard procedures for how to conduct an object substitution experiment, how to analyze the results, and it will also explain the hypothesized causes for this unusual form of masking.

1. Stimuli and design

1. To run this experiment, you will need a programming environment such as MATLAB or experimental sequencing software such as E-Prime.
2. Each trial will consist of four basic components: A fixation cross, a shape stimulus display (called the target display), a mask (four dots), and a response display. Figure 1 depicts the four primary elements in each trial.
   1. The display background will always be white.
   2. The fixation cross is a small cross in the center of the display, with each arm of the cross measuring 0.25 cm. The fixation cross will be present at all times, except for when the response display appears.
   3. The response display will consist of a black outline of a square, a circle, a diamond, and a triangle arranged horizontally and in the center of the screen. The shapes should each be inscribed within an invisible circle with a radius of 0.75 cm.
   4. The target display will include eight shapes, randomly selected on each trial from the set of four: a black outline of a circle, square, diamond, and triangle. Not all shapes need to appear in each trial, and of course, shapes can (and will be repeated) in a given trial. Again, the shapes will each be 0.75 cm in radius. They will appear in random positions on an invisible circle with a radius of 1.5 cm from the fixation cross in the center of the screen.
   5. The mask will be four black discs (dots) with a radius of 0.25 cm. The dots should be arranged to form the four corners of a square just large enough to include all of the four target shapes within.

Figure 1: Primary elements of an object substation display. Every trial will begin with a fixation display, and participants will be instructed to fixate the cross before initiating a trial. Every trial will end with a response display, in which the participant will select the shape she remembers seeing between the four dots. Between the fixation and response display, a target display will show a ring of eight shapes and a mask of four dots will also appear, in a position surrounding one of the shapes. As described in more detail in the procedure, the mask and target display can appear in different orders, but each will remain present for 30 ms.

3. The sequence of events in each trial will begin with a fixation screen that will remain present until the participant presses the spacebar, and for 200 ms after.
4. A trial will always end with a response display that will remain present until the participant enters a response (by clicking on the shape she believes she saw between the four dots on that trial.)
5. Between the fixation and response displays a target display and a mask will each be shown for 30 ms. The mask will surround the position of a randomly selected target shape in each trial.
6. The critical independent variable in this experiment is the stimulus onset asynchrony (SOA). This is defined as the time difference between the onset of two stimuli, in the case of this experiment, the mask and the target display. Thus, the SOA is the time of onset of the target display minus the time of onset of the mask display. Figures 2-4 schematize several trials of the experiment with different SOAs.
   1. If the SOA is 0, it means that the stimuli appear together (and since each last for 30 ms, they disappear together as well).
   2. If the SOA is negative, it means that the mask appeared before the target display.
   3. If the SOA is positive, it means that the mask appeared after the target display.
   4. The experiment will include 15 different SOAs, 0 ms as well as positive and negative: 10, 30, 50, 70, 90, 150, and 300 ms.

Figure 2: Stimulus onset asynchrony of 0 ms. In a trial with an SOA of 0 ms, the mask and the targets appear simultaneously. Since each is programmed to remain present for 30 ms, they are present and expire together as well.

Figure 3: Stimulus onset asynchrony of 50 ms. An SOA of 50 ms, whether negative or positive, means that one stimulus will onset 50 ms after the other. But these stimuli are programmed to last only 30 ms, meaning that an SOA of 50 will leave 20 ms with an empty display (except for the fixation cross). For this experiment, we've defined SOA as target onset-mask onset, meaning that negative SOA values are associated with the mask appearing first, and positive values are when the mask appears second.

Figure 4: Stimulus onset asynchrony of 10 ms. With masks and targets programmed to remain present for 30 ms, SOAs of 10 ms leave 20 ms during which the mask and the target overlap.

7. Sequence the experiment to include 20 trials with each of the 15 SOAs, randomly intermixing all the different kinds of trials.
8. A critical factor is that the participant does not know where the dots will appear in each trial, so be sure that the dots are appearing unpredictably from trial to trial.
9. Be sure that the experimental program outputs sufficient information about each trial to allow you to analyze the results later. The relevant information for each trial is: the trial number, the SOA on that trial, the shape shown between the mask, and the shape response given by the participant.

2. Running the experiment

1. To run this experiment, you'll want to recruit 10 participants, testing each individually in a quiet room.
2. When a participant arrives for the experiment, seat them 60 cm away from the monitor of the testing computer.
3. The image on the screen should be of a target display with a mask. Use this image to explain the instructions as follows:
   1. In each trial of this experiment you will see a ring of shapes, like the one shown here [pointing]. Additionally, you will see four dots, like the ones here. Sometimes, they will overlap, as in this image. But sometimes, one might precede the other. Regardless of their order of appearance, your task is simply to try to see and remember the shape that appears in the space between those four dots. Does this make sense?"
   2. Great. Sometimes, it might feel hard-you might feel like you really don't know the answer. Just guess in those cases. A few other things. Each trial will start with a display of just a cross, like the one in the middle, here. Before you start a trial, make sure you are fixating the cross, and try your best not to move your eyes. When you are ready, you'll press the spacebar to start a trial."
4. After explaining the instructions to the participant, start the experiment, and observe the participant for four or five trials to make sure they have understood the instructions. Then leave them to complete the experiment.

3. Analyzing the results

1. To analyze the results, you'll first want to compute response accuracy individually for each subject and SOA.
2. Next, you can average together response accuracy by SOA, across subjects.
3. To determine whether there is a significant effect of SOA on performance, perform a repeated measures ANOVA on accuracy as a function of SOA using MATLAB or a statistical package such as SPSS.

Figure 5 graphs average response accuracy across participants as a function of SOA. As the graph should make clear, an ANOVA analysing these results would probably show a significant effect of SOA. What kind of effect does it show? It appears that with very large SOAs, negative or positive, the mask does nothing-performance in the task is very good. When separated by 150 or 300 ms, the mask and the target stimulus are really just separate events. But these are critical conditions because they show that the target shapes can be perceived, even related to the position of the four dots, in the 30 ms of exposure. In other words, they show that there is nothing inherently too fast about these presentation times. Similarly, with negative SOAs, performance is mostly pretty good. These are trials in which the mask precedes the target stimulus. Even with an SOA of -10 and 0, performance is 50% or better, and the mask and stimulus overlap during these SOAs for 20 to 30 ms.

Figure 5: Results of Object Substitution Masking experiment. The results plotted are of average response accuracy as a function of SOA. Since there were four shapes in each trial, guessing would produce an average accuracy of 25%, labeled by the red line marked chance. When the mask precedes the target stimulus (when the SOA is negative), performance tends to be very good, better than 50% and often better than 80%. This is because this kind of mask needs to come after a target in order to mask it. With SOAs in the range of 10 to 90 ms, however, accuracy is surprisingly low, at times dropping to 25%. This is the range of SOAs during which an Object Substitution Mask works.

The critical SOAs are the ones between 10 and 90 ms. At these SOAs, performance is very bad, dropping as low as chance-what someone would do if they were just guessing. Performance at these SOAs demonstrates that Object Substitution Masking is taking place. Why?

Remember that the four dots do not overlap or cover up any of the masked shape. But the space that they surround entirely includes the shape. The explanation for this phenomenon is that for a stimulus to be perceived consciously it needs to do more than just stimulate the retina; it needs to be processed and reprocessed. Conscious perception is something that takes time for our brains to create. The four dots appearing to surround a position that was just occupied serve to effectively confuse the brain; they halt the reprocessing of the original stimulus that would be necessary for it to make it into conscious awareness.

Among the many applications of Object Substitution Masking in recent years are studies that have utilized it in conjunction with neurophysiological techniques in order to isolate brain circuits involved in the production of conscious experience. Hirose and colleagues² in 2005 conducted an experiment using a technique known as repeated Transcranial Magnetic Stimulation (rTMS): Researchers use a magnetic coil to induce small electrical potentials in the brain of a subject, and repeated induction can cause a small portion of cortex to deactivate for a brief period of time. In the Hirose et al. study, they deactivated a region of visual cortex called V5/MT+. The effect was that this prevented object substitution masking-positive SOA dot presentations did not prevent perception of the stimuli. V5/MT+ is known to play a large role in the perception of motion. This study suggested that its role might be broader, participating in connecting moments together in perceptual experience. When disrupted, the mask and the target stimulus can't be seen as part of the same event, and as a result, the mask fails to mask.

Another way that Object Substitution Masking has been used is to investigate questions about whether stimuli need to make it into conscious awareness in order to influence behaviour. For example, a word that is masked is not reportable by an observer. Will it have a priming effect however? Some research suggests that it does.³

1. Enns, J. T., & Di Lollo, V. (1997). Object substitution: A new form of masking in unattended visual locations. Psychological Science, 8(2), 135-139.
2. Hirose, N., Kihara, K., Tsubomi, H., Mima, T., Ueki, Y., Fukuyama, H., & Osaka, N. (2005). Involvement of V5/MT+ in object substitution masking: evidence from repetitive transcranial magnetic stimulation. Neuroreport,16(5), 491-494.
3. Goodhew, S. C., Visser, T. A., Lipp, O. V., & Dux, P. E. (2011). Implicit semantic perception in object substitution masking. Cognition, 118(1), 130-134.