Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods

Podsumowanie

Using functional MRI and behavioral methods to determine the neural representation of the subjective value of risky and ambiguous options in the human brain.

Streszczenie

Most of the choices we make have uncertain consequences. In some cases the probabilities for different possible outcomes are precisely known, a condition termed "risky". In other cases when probabilities cannot be estimated, this is a condition described as "ambiguous". While most people are averse to both risk and ambiguity^1,2, the degree of those aversions vary substantially across individuals, such that the subjective value of the same risky or ambiguous option can be very different for different individuals. We combine functional MRI (fMRI) with an experimental economics-based method³ to assess the neural representation of the subjective values of risky and ambiguous options⁴. This technique can be now used to study these neural representations in different populations, such as different age groups and different patient populations.

In our experiment, subjects make consequential choices between two alternatives while their neural activation is tracked using fMRI. On each trial subjects choose between lotteries that vary in their monetary amount and in either the probability of winning that amount or the ambiguity level associated with winning. Our parametric design allows us to use each individual's choice behavior to estimate their attitudes towards risk and ambiguity, and thus to estimate the subjective values that each option held for them. Another important feature of the design is that the outcome of the chosen lottery is not revealed during the experiment, so that no learning can take place, and thus the ambiguous options remain ambiguous and risk attitudes are stable. Instead, at the end of the scanning session one or few trials are randomly selected and played for real money. Since subjects do not know beforehand which trials will be selected, they must treat each and every trial as if it and it alone was the one trial on which they will be paid. This design ensures that we can estimate the true subjective value of each option to each subject. We then look for areas in the brain whose activation is correlated with the subjective value of risky options and for areas whose activation is correlated with the subjective value of ambiguous options.

Protokół

1. Preparing the Experiment

1. The first step is to design visual stimuli representing risky and ambiguous choices that will be presented on the screen in the scanner. We use images such as those presented in Figure 1 to represent bags filled with poker chips which we call "lottery bags". Graphically, these images can be thought of as stacks of poker chips before they are placed in a bag. Importantly, these images represent real containers, in our case envelopes, filled with 100 red and blue poker chips which the subject will see before starting the experiment. This insures that subjects both understand the lotteries they will face and believe that the computer display accurately presents those lotteries. For risky lotteries the winning probability, the ratio of red to blue chips in a given envelope, is precisely stated using both numbers and a graphic stimulus (Figure 1A). For ambiguous lotteries part of the information about the probability is missing (Figure 1B), such that the possible ratio of red to blue chips is bounded but not specified, rendering the winning probability partially ambiguous.
2. For risky lotteries the red and blue areas of each image are proportional to the number of red and blue chips in the envelope. We recommend using a minimum of 3 outcome probabilities⁴ (Figure 1A). The exact probabilities used can vary according to the specific requirements of the experiment, but experimenters should be cautious about using very high and very low probabilities. It is known that human subjects typically misrepresent probabilities below 10% or above 90%⁵. Unless one intends to study this systematic misrepresentation, these extremes should be avoided.
3. To convey ambiguity the central part of the stack of chips in the computer display is obscured with a gray occluder (Figure 1B). In the gray area the number of chips of each color will be unknown, and thus the probability of drawing a red or a blue chip will not be precisely known. For example, in the middle bag in Figure 1B the occluder covers 50% of the bag, and thus the number of red chips can be anywhere between 25 (if all the chips behind the occluder are blue) and 75 (if all the chips behind the occluder are red). Of course, the number of blue chips can also be anywhere between these two values.
4. Increasing the occluder size increases the ambiguity level (the range of possible probabilities for drawing a red or blue chip). We recommend using at least 3 levels of occlusion, covering ~25, 50 or 75% of the bag (Figure 1B).
5. When subjects perform the task on the computer, we present each winning probability/ambiguity level with a range of possible outcome amounts. We recommend using 5 reward levels⁴, spanning a wide range of amounts, for example: 5, 9.5, 18, 34 and 65 dollars. In the display, we present the outcome amount next to the winning color and display "0" next to the other color. For example, in Figure 2 drawing a red chip would result in winning $18 while drawing a blue chip would result in a zero outcome.
6. Critically, on each trial subjects will choose between two lotteries. For simplicity we keep one of the options constant throughout the experiment (in this example a 50% chance of winning $5) and only vary the other option. This has two advantages. First, the constant option does not have to appear on the screen, simplifying the visual display (although a reminder every now and then may be helpful). Second, because one option never changes, regression-based analyses of the fMRI signal can effectively neglect this parameter. Note that in order to have a "common currency" for the subjective values of risky and ambiguous lotteries the reference option has to be the same for both types of trials.
7. Each combination of probability or ambiguity level and amount should be presented a total of at least 4 times, and preferably more, to ensure sufficient statistical power in both the behavioral and fMRI analyses. In half of the repetitions red should be associated with a non-zero outcome and in the other half blue, to avoid color bias and to insure symmetry in the ambiguity.
8. We chose to use a slow event-related design (Figure 3), in which the hemodynamic response to different trials is well separated in time. In such a design each lottery should be presented as a stimulus briefly, in our case for 2 s, followed by a delay period (in our case, 6 s), to allow time for the decision-related neural activation to build up. Responses should be made within a brief time window (1-2 s). Use an image such as the one shown in Figure 3 as a brief feedback, so that the subject knows their response has been recorded. Separate the trials by long rest periods (10 s or more) to allow for the hemodynamic response to go back to baseline. Shorter rest periods can be used with appropriate jitter. Group the trials in blocks of up to 30 trials each, but be sure not to let the blocks take longer than about 10 minutes. This allows for rest periods in the scanner which maximize performance and minimize subject fatigue. To allow for at least 4 repetitions of each combination of probability/ambiguity and amount the total number of trials will be at least 120, i.e. 4 blocks.
9. Prepare the physical bags (in our case envelopes) so they can be shown to subjects before they perform the task. They will be used later to play the randomly chosen trial(s) for pay off. Prepare a bag for each lottery image used in the experiment. Fill each bag with a total of 100 poker/bingo red and blue chips, with proportions corresponding to the probability of drawing a chip of each color from that bag shown in the display. For ambiguous bags use a random number generator to decide on the actual numbers of red and blue chips, corresponding to each ambiguity level. Prepare the reference bag with 50 red and 50 blue chips. Preparing physical bags and showing them to the subjects is particularly important for subjects recruited in psychology departments. These subjects are likely to suspect some kind of deception and should they suspect deception their responses will be uninterpretable.

2. Preparing the Subject

1. Each subject must fill out a consent form and a screening questionnaire. The screening form verifies that the subject does not have any metal in their body, that they are not pregnant or claustrophobic, and that they can be safely scanned. Subjects must also remove all metals from their body to insure safety in the scanner environment. This is critical.
2. Provide the subject with detailed instructions about the experiment. Ask them a few simple questions to make sure they understand how probabilities and amounts are conveyed in each image, and to make sure they understand their task. Make sure not to reveal any information that could influence their choices. For example, do not frame the choice problem they face in such a way as to bias the subjects towards a particular risk attitude. Show them the physical bags and stress that each image in the experiment refers to a single specific physical bag that you cannot and will not tamper with. Also explain that in half the trials blue will be the winning color and in half red. Explain the payment mechanism, so that the subject understands that they will be paid according to their choices. Encourage subjects to ask questions about anything they do not understand. This is a critical period when the beliefs of the subjects about the experiment are being established. It is essential that subjects be confident that the experiment does not involve any type of deception or the behavioral and neural results will be uninterpretable.
3. Seal the bags and have the subject sign their name across the seal. Explain that this will enable them to verify at the end of the experiment that you did not change the contents of the bags during the experiment. This helps reassure subjects that they are playing an entirely fair game. Explain also that after the lotteries are played at the end of the experiment they will be allowed to look into the contents of the bags to make sure they conform to the stated probability or ambiguity level.

3. Scanning

1. We use a 3T MRI scanner with a head coil (4 channels or more) to get Blood Oxygenation Level-Dependent (BOLD) signals from the whole brain.
2. Use a 2-button response box to record subjects choices.
3. Anatomical scan: We use a T1-weighted MPRAGE sequence to get a clear high-resolution (1x1x1 mm) image of the subject's brain that can be used for 3D reconstruction. Any high-resolution sequence can be used for this purpose.
4. Functional scans: We use a T2*-weighted EPI sequence, with a TR of 2 s, and 3x3x3 mm voxels. Make sure to position the slices such that they include the brain areas you are most interested in, typically the prefrontal cortex, parietal cortex and the basal ganglia. Scanning parameters should be optimized for the specific scanner, we used: TE 30ms, flip angle 75°, 36 3 mm slices with no inter-slice gap, parallel to the AC-PC line, in-plane resolution 3x3 mm, FOV 192 mm. Other studies have suggested that positioning the slices at 30° to the AC-PC line may reduce signal dropout in the orbitofrontal cortex⁶.

4. Payment Procedure

1. After removing the subject from the scanner, retrieve the behavioral data from the computer that has recorded the subject's responses.
2. Randomly select one or few trials for payment. It is best to let the subject do the selection, for example by drawing a numbered poker chip out of an opaque bag that contains chips with all trial numbers. This will ensure to the subject that the selection is indeed random.
3. For each selected trial show the subject the presented option and the choice they made on that trial. Ask them to draw a chip from the bag chosen on that trial, and pay them according to the drawn color and the amount presented on the trial.

For example, if the selected trial presented the lottery depicted in Figure 2 (an ambiguous lottery, offering $18 if a red chip is drawn) and the subject chose this lottery (rather than the reference lottery), then the subject should draw a chip out of the physical bag corresponding to the lottery image. If a red chip is drawn the subject will receive $18, if a blue chip is drawn they will receive nothing.

5. Analyzing the Behavioral Data

1. Using maximum likelihood we fit the choice data of each subject to a logistic function of the form:

Where Pv is the probability that the subject chose the variable lottery, SV_F and SV_V are the subjective values of the fixed and variable options respectively, and γ is the slope of the logistic function, which is a subject-specific parameter. An alternative approach is to use a probit distribution.

2. To model the subjective value of each option for each subject you can use one of a number of models that take into account the amount, probability and ambiguity level of the option and the attitudes of the individual subject towards risk and ambiguity. We chose to use a power function⁵ that includes a linear effect of ambiguity on the perceived probability⁷:

Where p is the objective probability (by definition 0.5 for this class of ambiguous lotteries), A is the ambiguity level (the fraction of the total probability that is unknown, 0 for risky lotteries), V is the amount, and α and βare subject-specific risk and ambiguity attitude parameters respectively. One of several alternative approaches is to include ambiguity as an exponential effect⁸:

Fitting the choice data with the choice function thus provides estimates for the risk attitude (α) and ambiguity attitude (β) for each subject.

6. Analyzing the Neural Data

1. Perform standard preprocessing of the data, including: 1) slice scan-time correction to account for the slight differences in scanning times of different slices; 2) motion correction to account for intra- and inter-run subject movement; and 3) removal of low frequencies that are typically related to physiological noise and scanner drifts.
2. Register the functional data of each subject to their anatomical data.
3. For analysis at the single subject level model the activity of each voxel as a sustained response during the entire trial (in our case 10 s), convolved with a standard hemodynamic response function⁹. Use a General Linear Model with the following predictors:

• Two predictors of subjective value (SV), one for risky trials and one for ambiguous trials. Use equation 2 and the individual subject specific parameters derived from the behavioral fit to calculate the SV of each lottery. Since the reference lottery is the same for all trials we can use the SV of the variable lottery alone in each trial. For the risky SV predictor insert the SV for each risky trial, and 0 for each ambiguous trial, and vice versa for the ambiguous predictor.
• Two dummy predictors, one for risky trials and one for ambiguous trials, to capture general activation, such as visual and motor activations.

4. Look for voxels in which the coefficients of SV under risk and/or under ambiguity are significant. The test for significance should take into account the multiple comparisons performed. The method we used was limiting the minimum cluster size to 6 contiguous functional voxels¹⁰. Alternatively, other methods, such as the False Discovery Rate (FDR)¹¹, can be used to correct for multiple corrections.

7. Representative Results

Behavior

Figure 4 presents the behavioral results of three representative subjects. Each panel presents the choice data and model fit results for one subject under either risk (left) or ambiguity (right). The graphs depict the proportion of trials in which the subject chose the variable lottery as a function of amount, separately for each level of probability or ambiguity. As can be seen, subjects may vary a lot in their attitudes towards risk and ambiguity.

To examine the goodness of the fit, check the r², which should ideally be over 0.5, and also inspect the curves visually. While all our three example subjects had lawful behavior that enabled reasonable fits, note that subject 2 hardly chose the variable option in the risk condition with the lowest probability (0.13). This suggests that expanding the range of amounts and/or using higher probabilities may provide better results, because it will ensure that subjects choose the variable options on at least some of the trials. Another option is to pre-test each subject on a wide range of amounts and choose those amounts that ensure a comparable number of reference and variable option choices for each individual.

fMRI

Figure 5 presents the imaging results in one representative subject. Highlighted voxels are ones in which the coefficient of the subjective value predictor under ambiguity (top) or risk (bottom) was significantly different from 0. In this typical subject, significant correlation was found in medial prefrontal cortex (MPFC) and the striatum under both conditions. These areas are the most consistent across subjects, but significant correlations may also be expected in areas in medial and lateral parietal cortex, as well as the amygdala. As activity in this type of tasks is usually weak and noisy you should expect high variability across subjects with many subjects exhibiting significant correlations only in a subset of areas.

Figure 1. Risky and ambiguous stimuli. A) In risky stimuli the red and blue areas of each image on the screen are proportional to the number of red and blue chips in the envelope. Three outcome probabilities were used here: 0.13, 0.25 and 0.38. B) In ambiguous stimuli the central part of the image is obscured with a gray occluder. In the gray area the number of chips of each color is unknown, and thus the probability of drawing a chip of a certain color is not precisely known. Three levels of ambiguity are used here, where 25, 50 or 75% of the image are occluded.

Figure 2. A lottery example. This is an ambiguous lottery, at a 50% ambiguity level. At least 25 of the chips in the envelope are red and at least 25 are blue. If a red chip is drawn the subject will win $18, while they will win nothing if a blue chip is drawn.

Figure 3. The trial structure. A lottery is briefly presented, followed by a delay period. A response cue then prompts subjects to indicate their choice between the lottery on the screen and the reference lottery (in this case a 50% chance of winning $5). Trials are interleaved with long rest periods.

Figure 4. Examples of single subject choice behavior. The graphs present the proportion of trials in which each subject chose the variable option over the reference, as a function of the offered amount, in risky (left) and ambiguous (right) trials. Different curves are for different risk or ambiguity levels. α, risk attitude parameter; β, ambiguity attitude parameter; r², McFadden's pseudo R-squared, a measure of the goodness of fit of the behavioral model, equivalent to the portion of the variance that is explained by the model; n, number of trials in which response was made (out of a total of 180).

Figure 5. Example of single subject activation maps. Activation maps are presented on a high resolution anatomical image. Highlighted areas are those whose activation was significantly correlated with subjective value under risk (top) or under ambiguity (bottom). In most subjects the medial prefrontal cortex (MPFC) and the striatum represent subjective value under both risk and ambiguity. Corrected p-values are based on a minimum cluster size of 6 functional voxels. Click here to view larger figure.

Access restricted. Please log in or start a trial to view this content.

Dyskusje

We have used a method from experimental economics to characterize subjects' behavior and estimate individual attitudes towards risk and ambiguity. We then used these estimates to analyze neural data.

Other methods for examining fMRI activity while subjects make choices under risk and ambiguity have been used before^8,12. Our approach, however, combines several important features. First, it uses a parametric design, in which different parameters (amount, probability and ambiguity leve...

Access restricted. Please log in or start a trial to view this content.

Materiały

Name	Company	Catalog Number	Comments
Allegra Head Only 3.0 T MRI Scanner	Siemens AG	A whole body scanner can also be used
NM-011 transmit head coil	Nova Medical
E-prime	Psychology Software Tools	Stimuli presentation software
Matlab	Mathworks