Choosing the correct experimental design for the specific scientific question at hand is essential to obtain reliable results. A within-subjects design is an experimental paradigm where all the participants receive every level of the treatment; there is one independent variable with several variations, or conditions. In this way, the amount of error arising from natural variance between individuals—typical of a between-subjects design—is reduced.
One caveat of this type of experiment is that the order in which treatments are given can influence the results. To minimize order effects, counterbalancing is used to ensure all possible orders of the conditions occur in the study.
This video demonstrates a within-subjects experiment that examines how different motivational messages influence willingness to exert physical effort. As a within-subjects design, the participant will read each of four types of motivational messages and then have their physical effort measured via lifting weights.
First, several concepts to consider when designing a within-subjects experiment are introduced. The video will then go on to demonstrate how to conduct the experiment using proper counterbalancing. Finally, analysis of data from a large number of participants will be discussed.
To begin, create an operational definition of a motivational message. An operational definition is an unambiguous description of a variable for the purpose of the experiment.
For the purposes of this experiment, a motivational message is any combination of image and phrase designed to energize a person’s behavior. The person’s behavior is manipulated here by viewing a series of images accompanied by empowering quotes focusing on one of 4 areas: hard work, self-affirmation, outcomes and success, and general positive feelings and emotions.
Next, create an operational definition of effort. For purposes of this experiment, effort is defined as the participant’s willingness to exert physical strength on a weight-lifting task.
Then determine the order of conditions through counterbalance. It is important to address order effects because if conditions were always in the same order, the participants would likely perform worse on the later conditions due to fatigue.
Counterbalance conditions involve a systematic approach by which the researcher ensures that every order occurs during the study. Determine all possible orders of the four conditions. Here, H equals Hard Work, S equals Self-Affirmation, O equals Outcomes, and P equals Positive Emotion.
To conduct the study, first meet the participant at the lab. Provide the participant with informed consent. This is a brief description of the research, a sense of the procedure, an indication of potential risks and benefits, the freedom of withdrawal at any time, and a manner to get help if they experience discomfort.
Next, write each of the 24 possible orders of the four conditions on a slip of paper. Then, place all slips in a bowl.
Select one slip and proceed to run the experiment in that order; this randomly selects one of the counterbalanced orders. Do not replace the order in the bowl so that every order gets done once before repeating any one order a second time.
To run the conditions, show the participant a page-sized printout of the quote while the participant sits at a table. Tell the participant to read the quote over and take a minute to reflect on what it means to them.
After a minute instruct the participant to stand and take a 10 pound dumbbell in his or her dominant hand. Tell the participant to complete as many curls as they would like to in the next 30 sec, and to count aloud as they complete each one.
After 30 sec, note the participant’s number of completed curls on a sheet as the participant takes a brief 10 to 15 sec rest.
Proceed to run all four conditions using the same steps, with the only difference being the image the participant views.
To debrief the participant, tell them the nature of the study.
Researcher: “Thank you for participating. In this study I was trying to determine if different types of motivational messages would increase the amount of physical effort participants were willing to exert. There were four types of messages: one emphasizing hard work, one emphasizing what a good person you are, one emphasizing successful outcomes, and one that was generally positive. We hypothesized that the message emphasizing hard work would result in exerting more physical effort.”
Then, explain explicitly why deception was necessary for the experiment.
Researcher: “We couldn’t tell you about our hypotheses ahead of time because we wanted you to act as naturally as possible.”
The procedure was run for 24 counterbalanced orders three times. Accordingly, data was collected from 72 total participants; a large number of participants are necessary to ensure that the results are reliable and reflective of the greater population.
The numbers shown here reflect the number of times participants in each condition lifted the weight. The results are the means for the 72 participants in each condition.
The outcome indicates that participants who read the hard work motivational message exerted more physical effort by doing more curls of the 10 pound weight in 30 sec.
Within-subjects designs are particularly common in functional magnetic resonance imaging, or fMRI, research where participants lay in an fMRI machine while experiencing several conditions to see how the brain reacts to different experiences.
For example, an fMRI study was used to investigate which areas of the brain correlate with feelings of long-term intense romantic love. To test this, images were shown to participants that represented a highly familiar acquaintance, a close long-term friend, a low-familiar person, and their long-term romantic partner.
Analyses indicated that the long-term romantic partner activated areas of the brain associated with the dopamine reward system, and areas associated with emotional attachments.
You’ve just watched JoVE’s introduction to within-subjects experimental design. Now you should have a good understanding of the concept of counterbalancing to generate proper controls for this type of experiment, how to set up a within-subjects experiment through creating operational definitions, and how to conduct the study. You’ve also been introduced to research performed using fMRI as an application of this type of experiment.
Remember, proper counterbalancing and use of a large number of participants is critical to obtain reliable results when performing within-subjects experiments.
