Name: operant protocols for assessing the cost-benefit analysis during reinforced decision making by rodents
Uploaded: 2018-09-10T13:30:00
Description: Watch this Scientific Journal Video about Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents at JoVE.com

This research was supported by RMH Neuroscience Foundation, Australia; the Australian Brain Foundation; the RACP Thyne Reid Fellowship, Australia; and by a project grant from the Cognitive Sciences and Technologies Council, Iran to Abbas Haghparast.

All procedures explained here were approved and carried out in accordance with the Guide for the Care and Use of Laboratory Animals and were approved by the Florey Institute Animal Ethics Committee or the Neuroscience Research Center.
1. Housing, Handling, and Food Restriction
<ol>
	<li>Use adult (normally 8 weeks old) male rats (any strains) and keep them in the room with a 12-h light/dark cycle.</li>
	<li>Restrict their food access to encourage the animals to perform the task. 
	Note:...

<ol>
	<li>Gold, J. I., Shadlen, M. N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+neural+basis+of+decision+making.">The neural basis of decision making.</a> Annual Review of Neuroscience. 30, 535-574 (2007).</li><li>Shi, Z., M&#252;ller, H. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+perception+and+action:+development,+decision-making,+and+neural+mechanisms.">Multisensory perception and action: development, decision-making, and neural mechanisms.</a> Frontiers in Integrative Neuroscience. 7, 81 (2013).</li><li>Sutton, R. S., Barto, A. G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Reinforcement Learning: An Introduction. 1, (1998).</li><li>Khani, A., Rainer, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Neural+and+neurochemical+basis+of+reinforcement-guided+decision+making.">Neural and neurochemical basis of reinforcement-guided decision making.</a> Journal of Neurophysiology. 116, 724-741 (2016).</li><li>Fatahi, Z., Haghparast, A., Khani, A., Kermani, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Functional+connectivity+between+anterior+cingulate+cortex+and+orbitofrontal+cortex+during+value-based+decision+making.">Functional connectivity between anterior cingulate cortex and orbitofrontal cortex during value-based decision making.</a> Neurobiology of Learning and Memory. 147, 74-78 (2018).</li><li>Khani, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Activation+of+cannabinoid+system+in+anterior+cingulate+cortex+and+orbitofrontal+cortex+modulates+cost-benefit+decision+making.">Activation of cannabinoid system in anterior cingulate cortex and orbitofrontal cortex modulates cost-benefit decision making.</a> Psychopharmacology. 232, 2097-2112 (2015).</li><li>Rudebeck, P. H., Walton, M. E., Smyth, A. N., Bannerman, D. M., Rushworth, M. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Separate+neural+pathways+process+different+decision+costs.">Separate neural pathways process different decision costs.</a> Nature Neuroscience. 9, 1161-1168 (2006).</li><li>Gage, G. J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Surgical+implantation+of+chronic+neural+electrodes+for+recording+single+unit+activity+and+electrocorticographic+signals.">Surgical implantation of chronic neural electrodes for recording single unit activity and electrocorticographic signals.</a> Journal of Visualized Experiments. (60), e3565 (2012).</li><li>Paxinos, G., Watson, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> The Rat Brain in Stereotaxic Coordinates. , (1998).</li><li>Bokil, H., Andrews, P., Kulkarni, J. E., Mehta, S., Mitra, P. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Chronux:+a+platform+for+analyzing+neural+signals.">Chronux: a platform for analyzing neural signals.</a> Journal of Neuroscience Methods. 192, 146-151 (2010).</li><li>Cohen, M. X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Analyzing Neural Time Series Data: Theory and Practice. , (2014).</li><li>Luk, C. -. H., Wallis, J. D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Choice+coding+in+frontal+cortex+during+stimulus-guided+or+action-guided+decision-making.">Choice coding in frontal cortex during stimulus-guided or action-guided decision-making.</a> Journal of Neuroscience. 33, 1864-1871 (2013).</li><li>Rudebeck, P. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Frontal+cortex+subregions+play+distinct+roles+in+choices+between+actions+and+stimuli.">Frontal cortex subregions play distinct roles in choices between actions and stimuli.</a> Journal of Neuroscience. 28, 13775-13785 (2008).</li><li>Goshadrou, F., Kermani, M., Ronaghi, A., Sajjadi, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+effect+of+ghrelin+on+MK-801+induced+memory+impairment+in+rats.">The effect of ghrelin on MK-801 induced memory impairment in rats.</a> Peptides. 44, 60-65 (2013).</li><li>Haghparast, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Intrahippocampal+administration+of+D2+but+not+D1+dopamine+receptor+antagonist+suppresses+the+expression+of+conditioned+place+preference+induced+by+morphine+in+the+ventral+tegmental+area.">Intrahippocampal administration of D2 but not D1 dopamine receptor antagonist suppresses the expression of conditioned place preference induced by morphine in the ventral tegmental area.</a> Neuroscience Letters. 541, 138-143 (2013).</li><li>Esmaeili, M. -. H., Kermani, M., Parvishan, A., Haghparast, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Role+of+D1/D2+dopamine+receptors+in+the+CA1+region+of+the+rat+hippocampus+in+the+rewarding+effects+of+morphine+administered+into+the+ventral+tegmental+area.">Role of D1/D2 dopamine receptors in the CA1 region of the rat hippocampus in the rewarding effects of morphine administered into the ventral tegmental area.</a> Behavioural Brain Research. 231, 111-115 (2012).</li><li>Chaleek, N., Kermani, M., Eliassi, A., Haghparast, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Effects+of+orexin+and+glucose+microinjected+into+the+hypothalamic+paraventricular+nucleus+on+gastric+acid+secretion+in+conscious+rats.">Effects of orexin and glucose microinjected into the hypothalamic paraventricular nucleus on gastric acid secretion in conscious rats.</a> Neurogastroenterology &amp; Motility. 24, e94-e102 (2012).</li><li>Kermani, M., Eliassi, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Gastric+acid+secretion+induced+by+paraventricular+nucleus+microinjection+of+orexin+A+is+mediated+through+activation+of+neuropeptide+Yergic+system.">Gastric acid secretion induced by paraventricular nucleus microinjection of orexin A is mediated through activation of neuropeptide Yergic system.</a> Neuroscience. 226, 81-88 (2012).</li><li>Kermani, M., Azizi, P., Haghparast, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+role+of+nitric+oxide+in+the+effects+of+cumin+(Cuminum+Cyminum+L.)+fruit+essential+oil+on+the+acquisition+of+morphine-induced+conditioned+place+preference+in+adult+male+mice.">The role of nitric oxide in the effects of cumin (Cuminum Cyminum L.) fruit essential oil on the acquisition of morphine-induced conditioned place preference in adult male mice.</a> Chinese Journal of Integrative Medicine. , 1-6 (2012).</li><li>Ahmadi, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Synthesis+and+antinociceptive+behaviors+of+new+methyl+and+hydroxyl+derivatives+of+phencyclidine.">Synthesis and antinociceptive behaviors of new methyl and hydroxyl derivatives of phencyclidine.</a> Current Medicinal Chemistry. 19, 763-769 (2012).</li></ol>

Rodents have long been used in neuroscientific studies that deal with different topics, from cognitive abilities such as learning and memory2,14 and reinforced behavior7,15,16 to the central control of organs17,18 and neuropharmacology19,20. The proposed ...

Decision making is the process of recognizing and selecting choices based on the values and preferences of the decision maker and the consequences of the selected action1. Although decision making has been extensively studied in different fields (i.e., economics, psychology, and neuroscience), neural mechanisms underlying such cognitive abilities are not yet fully understood. Two subcategories of decision making are perceptual decision making and reinforcement-guided decision making. Though they incorporate considerable overlapping elements and concepts, perceptual decision making relies on the available sensory information1,2, whereas reinforcement-guided decision-making deals with the relative value of actions gained over a specific timescale3. One important aspect of reinforced decision making is the cost-benefit analysis which is performed intuitively by the brain by computing the benefits of the given choices and subtracting the associated costs of each alternative1.
The T-maze (or the variant Y-maze) is one of the most-used mazes in cognitive experiments using rodents. Animals are placed in the start arm (the base of the T) and permitted to choose the goal arm (one of the side arms). Tasks such as a forced alternation or left-right discrimination are mainly used with rodents in the T-maze to test reference and working memory4. T-mazes are also widely used in decision-making experiments5,6,7. In the simplest design, the reward is placed in only one goal arm. The choice is predictable, and animals would certainly prefer the reward rather than nothing, regardless of the reward value. Another option is to place rewards in both goal arms and then let the animals make a choice of which path to take depending on several parameters (i.e., the natural preference of the animal, the difference in the value of the rewards, and the costs to be paid). In the value-based design, the task is more complicated by having weighing-scale properties. In this way, an animal receives differently valued rewards by choosing between the two alternatives, as well as between the costs of the actions [i.e., the amount of waiting (delay-based) or the amount of effort (effort-based) needed to receive rewards], each contributing to the decision that is made5,6.
In traditional delay-based T-maze decision making, animals are trained to select the high reward arm (HRA) and avoid the opposite low reward arm (LRA). The sides of the HRA and the LRA remain unchanged throughout the experiment. Although the task described above has been well documented in the literature, it suffers from several procedural drawbacks. Firstly, by having a fixed goal arm, the animal knows which arm to choose from the beginning of each trial. In this scenario, animals may select the goal arm based on their memory rather than on decision making. Hence, in a delay-based decision-making paradigm, if an animal selects the low reward because of the study intervention, it will not be clear whether this is due to a loss of memory or to the study intervention. A memory control group to segregate the observed behavior from the memory problem might be considered, but this burdens researchers and animals alike because of the additional work7. A second concern is the moment of decision making by the animal: once animals reach the decision zone (the junction of all three arms), they usually look to the left and to the right, weigh the costs and benefits regarding each arm, and then make their decision. However, after a few trials, they perform such a computation prior to arriving at the decision zone and simply run directly to the reward arm. As a result, these two drawbacks&#8212;a pre-bias to one arm and finding the moment of decision making&#8212;both highly interrupt the interpretation of electrophysiological and neuroimaging data.
In the method explained in this paper, the preferred arm (HRA) is cued by an auditory cue and may vary from trial to trial. Animals initiate the trials by entering the test zone (Figure 1) and triggering the auditory cue by &#34;nose-poking&#34; an infrared gate that has been placed at the junction of the three arms. The audio signal (20 dB, between 500 and 1,000 ms) is played from a speaker at the end of the goal arm.

<table>
	<tbody>
		<tr>
			<td>T-maze</td>
			<td></td>
			<td></td>
			<td>Self made</td>
		</tr>
		<tr>
			<td>Dustless Precision Sugar Pellets</td>
			<td>TSE Systems Intl. Group</td>
			<td>F0023</td>
			<td>45 mg, Sucrose</td>
		</tr>
		<tr>
			<td>Ketamine Hydrochloride Injection, USP</td>
			<td>Sigma-Aldrich</td>
			<td>6740-87-0</td>
			<td></td>
		</tr>
		<tr>
			<td>Xylazine</td>
			<td>Sigma-Aldrich</td>
			<td>7361-61-7</td>
			<td></td>
		</tr>
		<tr>
			<td>stereotaxic device</td>
			<td>Stoelting</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Isofluran</td>
			<td>Santa Cruz Biotechnology</td>
			<td>sc-363629Rx</td>
			<td></td>
		</tr>
		<tr>
			<td>PFA-coated stainless-steel wires</td>
			<td>A-M systems</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>acrylic cement</td>
			<td>Vertex, MA, USA</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>(wooden or PVC (polyvinyl chloride)-made)</td>
			<td>local suppliers</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Mini-Fit Power Connector</td>
			<td>Molex</td>
			<td>15243048</td>
			<td></td>
		</tr>
		<tr>
			<td>ethannol 70%</td>
			<td>Local suppliers</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>buprenorphine</td>
			<td>diamondback drugs</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Arduino UNO</td>
			<td>Arduino</td>
			<td></td>
			<td>https://www.arduino.cc/</td>
		</tr>
		<tr>
			<td>Infrared emitting diode</td>
			<td>Sharp</td>
			<td>GL480E00000F</td>
			<td>http://www.sharp-world.com/</td>
		</tr>
		<tr>
			<td>Chronux Toolbox</td>
			<td></td>
			<td></td>
			<td>Chronux.org</td>
		</tr>
		<tr>
			<td>Arduino codes</td>
			<td></td>
			<td></td>
			<td>https://github.com/dechuans/arduino-maze</td>
		</tr>
	</tbody>
</table>

operant protocols for assessing the cost-benefit analysis during reinforced decision making by rodents

Reinforcement-guided decision making is the ability to choose between competing courses of action based on the relative value of the benefits and their consequences. This process is integral to the normal human behavior and has been shown to be disrupted by neurological and psychiatric disorders such as addiction, schizophrenia, and depression. Rodents have long been used to uncover the neurobiology of human cognition. To this end, several behavioral tasks have been developed; however, most are non-automated and are labor-intensive. The recent development of the open-source microcontroller has enabled researchers to automate operant-based tasks for assessing a variety of cognitive tasks, standardizing the stimulus presentation, improving the data recording and consequently, improving the research output. Here, we describe an automated delay-based reinforcement-guided decision-making task, using an operant T-maze controlled by custom-written software programs. Using these decision-making tasks, we show the changes in the local field potential activities in the anterior cingulate cortex of a rat whilst it performs a delay-based cost-and-benefit decision-making task.

The data presented here is the recorded LFP from the left orbitofrontal cortex (OFC) and the anterior cingulate cortex (ACC) of six male Wistar rats using bipolar electrodes (of PFA-coated stainless steel). Table 1 shows the behavioral acquisition length for each training stage. The coordinates for the target locations were determined from a rat brain atlas9 and are as follows: for the AAC, 1.2 mm anterior to the bregma, 0.8 mm lateral to the midli...

Watch this Scientific Journal Video about Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents at JoVE.com

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents

A cost-benefit analysis is a weighing-scale approach that the brain performs during the course of decision making. Here, we propose a protocol to train rats on an operant-based decision-making paradigm where rats choose higher rewards at the expense of waiting for 15 s to receive them.

Reinforcement-guided decision making is the ability to choose between competing courses of action based on the relative value of the benefits and their ...

This technique can answer key questions in cognitive neurosciences such as how comparing consequences and causes can affect making a decision. The main advantage of this technique is that the special elements such as the location of the board do not affect the animal's decision. Begin by restricting food access to adult male rats in a room with a 12-hour light/dark cycle.On days one to three, handle the animals for five minutes twice a day. Then, weigh the animals after each handling session to determine the free-feeding amount. Next, obtain a T maze equipped with three partitions or arms, two stimulus speakers, five retractable doors, and five infrared motion sensors or Infrared Beam or IRB sensors.Copy the experimental software code into the computer. Then, download the experimental software and install it on a computer. Connect the microcontroller to the computer using a USB port.Next, on the computer, click the software icon, go to Tools and then select Port. From the dropdown menu, choose the COM port which connects the software to the computer. Go to Tools and select Boards.From the dropdown menu, select the type of software that controls the T maze. Then, click Upload on the top left of the interface window. Select the experimental software code, wait until the process finishes.After the process has finished, click Serial Monitor at the top right of the interface window. Then, in a new popup window, change the baud rate to 115, 200. For five days prior to the experimental day, bring the animals to the experiment room for at least one hour.Leave 10 sugar pellets at each goal arm and leave all doors open. During days one through three, place the animals individually in the maze for periods of 10 minutes once per day to explore the maze with no restrictions. During days four through five, place the animals individually in the maze.Leave two pellets in each arm and allow them to sample the food from both sides. Remove the animal from the maze immediately after eating the pellets from both sides or after five minutes of being in the maze. Finally, after each habituation session, clean the maze's floors with 70%ethanol and make sure the ethanol has evaporated before placing the next animal in the maze.On day six, bring the animals to the experiment room at least one hour prior to the experiment. Run the software program and set the trial number to 14 of which the first four trials will be forced choice and the rest will be choice trials. Before each trial begins, place four pellets in the goal arm which is to be cued as the High Reward Arm, HRA, and two pellets in the other arm which is the Low Reward Arm, LRA.For the forced choice trial training, pseudo randomly block one arm before each trial so that the animal is forced to select the other arm. Place an animal in the start box. After five to seven seconds, simultaneously open the start gate and click Start in the software's interface window.Collect the animal immediately after eating pellets or after five minutes of being in the maze and leave the animal in the home cage for two minutes. Prior to each trial, close door B in the arm which is selected by the software to be the HRA. Leave door A open in the opposite arm, the LRA.Then, place an animal in the start box. After five to seven seconds, simultaneously open the start gate and click Start. Let the animal freely choose either arm.If the animal chooses the HRA, open door A, let the animal enter the chamber, shut door A and open door B immediately to give the animal access to the selected food well. If the animal chooses the LRA, open door B in order to give the animal access to the food well. Next, remove the animal after it has eaten all the food in the selected food well and leave it in the home cage for two minutes.Complete the 10 choice trials for each animal and record the animal's choice, either HRA or LRA in each trial. Calculate the percentage of High Reward Choice, HRC, for all choice trials after each training session. Finally, after each animal has reached 80%of HRC in the discrimination training, complete the delay training by opening door B after a five-second delay once the animal chooses the HRA.Run 10 trials per day for each animal. Results indicated that there was a decrease in both the low and the high frequency powers in the ACC from the onset to the end of the stimulus. Comparing the time spent outside the chamber with the time in the chamber, the spectral analysis showed no changes in the oscillatory activities in the ACC.Further, mid to high frequency band activities in the OFC increased following the stimulus onset. No significant changes were observed in the pre-chamber and chamber time windows when they were compared together. While attempting this procedure, it is important to remove any source of anxiety as it can affect the animal's decision and impair the reliability of the result.

Research

JoVE Journal

Neuroscience

Combining Behavioral Endocrinology and Experimental Economics: Testosterone and Social Decision Making

Behavioral endocrinological research in humans as well as in animals suggests that testosterone plays a key role in social interactions. Studies in ...

Operant Sensation Seeking in the Mouse

Operant methods are powerful behavioral tools for the study of motivated behavior.  These 'self-administration' methods have been used extensively in drug ...

Isolating Nasal Olfactory Stem Cells from Rodents or Humans

The olfactory mucosa, located in the nasal cavity, is in charge of detecting odours. It is also the only nervous tissue that is exposed to the external ...

T-maze Forced Alternation and Left-right Discrimination Tasks for Assessing Working and Reference Memory in Mice

Forced alternation and left-right discrimination tasks using the T-maze have been widely used to assess working and reference memory, respectively, in ...

Large-scale Recording of Neurons by Movable Silicon Probes in Behaving Rodents

A major challenge in neuroscience is linking behavior to the collective activity of neural assemblies. Understanding of input-output relationships of ...

A Fully Automated and Highly Versatile System for Testing Multi-cognitive Functions and Recording Neuronal Activities in Rodents

We have developed a fully automated system for operant behavior testing and neuronal activity recording by which multiple cognitive brain functions can be ...

A Polished and Reinforced Thinned-skull Window for Long-term Imaging of the Mouse Brain

In vivo imaging of cortical function requires optical access to the brain without disruption of the intracranial environment. We present a method to form ...

Microdialysis of Ethanol During Operant Ethanol Self-administration and Ethanol Determination by Gas Chromatography

Operant self-administration methods are commonly used to study the behavioral and pharmacological effects of many drugs of abuse, including ethanol.  ...

Recording and Analysis of Circadian Rhythms in Running-wheel Activity in Rodents

When rodents have free access to a running wheel in their home cage, voluntary use of this wheel will depend on the time of day1-5. Nocturnal rodents, ...

Effects of Transcranial Alternating Current Stimulation on the Primary Motor Cortex by Online Combined Approach with Transcranial Magnetic Stimulation

Transcranial Alternating Current Stimulation (tACS) is a neuromodulatory technique able to act through sinusoidal electrical waveforms in a specific ...