Publications

Dopaminergic signals play a mathematically precise role in reward-related learning, and variations in dopaminergic signaling have been implicated in vulnerability to addiction. Here, we provide a detailed overview of the relationship between theoretical, mathematical, and experimental accounts of phasic dopamine signaling, with implications for the role of learning-related dopamine signaling in addiction and related disorders. We describe the theoretical and behavioral characteristics of model-free learning based on errors in the prediction of reward, including step-by-step explanations of the underlying equations. We then use recent insights from an animal model that highlights individual variation in learning during a Pavlovian conditioning paradigm to describe overlapping aspects of incentive salience attribution and model-free learning. We argue that this provides a computationally coherent account of some features of addiction.

Gaining a better understanding of the biological mechanisms underlying the individual variation observed in response to rewards and reward cues could help to identify and treat individuals more prone to disorders of impulsive control, such as addiction. Variation in response to reward cues is captured in rats undergoing autoshaping experiments where the appearance of a lever precedes food delivery. Although no response is required for food to be delivered, some rats (goal-trackers) learn to approach and avidly engage the magazine until food delivery, whereas other rats (sign-trackers) come to approach and engage avidly the lever. The impulsive and often maladaptive characteristics of the latter response are reminiscent of addictive behaviour in humans. In a previous article, we developed a computational model accounting for a set of experimental data regarding sign-trackers and goal-trackers. Here we show new simulations of the model to draw experimental predictions that could help further validate or refute the model. In particular, we apply the model to new experimental protocols such as injecting flupentixol locally into the core of the nucleus accumbens rather than systemically, and lesioning of the core of the nucleus accumbens before or after conditioning. In addition, we discuss the possibility of removing the food magazine during the inter-trial interval. The predictions from this revised model will help us better understand the role of different brain regions in the behaviours expressed by sign-trackers and goal-trackers.

There is ample evidence to suggest that the paraventricular nucleus of the thalamus (PVT) mediates cue-reward learning, especially as it relates to drug-seeking behavior. However, its exact role in these complex processes remains unknown. Here we will present and discuss data from our own laboratory which suggests that the PVT plays a role in multiple forms of stimulus-reward learning, and does so via distinct neurobiological systems. Using an animal model that captures individual variation in response to reward-associated cues, we are able to parse the incentive from the predictive properties of reward cues and to elucidate the neural circuitry underlying these different forms of cue-reward learning. When rats are exposed to a classical Pavlovian conditioning paradigm, wherein a cue predicts food reward, some rats, termed sign-trackers, approach and manipulate the cue upon its presentation. This behavior is indicative of attributing incentive salience to the cue. That is, the cue gains excessive control over behavior for sign-trackers. In contrast, other rats, termed goal-trackers, treat the cue as a mere predictor, and upon its presentation go to the location of reward delivery. Based on our own data utilizing this model, we hypothesize that the PVT represents a common node, but differentially regulates the sign- vs. goal-tracking response. We postulate that the PVT regulates sign-tracking behavior, or the attribution of incentive salience, via subcortical, dopamine-dependent mechanisms. In contrast, we propose that goal-tracking behavior, or the attribution of predictive value, is the product of “top-down” glutamatergic processing between the prelimbic cortex (PrL) and the PVT. Together, data from our laboratory and others support a role for the PVT in cue-motivated behaviors and suggest that it may be an important locus within the neural circuitry that goes awry in addiction and related disorders.

Reinforcement Learning has greatly influenced models of conditioning, providing powerful explanations of acquired behaviour and underlying physiological observations. However, in recent autoshaping experiments in rats, variation in the form of Pavlovian conditioned responses (CRs) and associated dopamine activity, have questioned the classical hypothesis that phasic dopamine activity corresponds to a reward prediction error-like signal arising from a classical Model-Free system, necessary for Pavlovian conditioning. Over the course of Pavlovian conditioning using food as the unconditioned stimulus (US), some rats (sign-trackers) come to approach and engage the conditioned stimulus (CS) itself – a lever – more and more avidly, whereas other rats (goal-trackers) learn to approach the location of food delivery upon CS presentation. Importantly, although both sign-trackers and goal-trackers learn the CS-US association equally well, only in sign-trackers does phasic dopamine activity show classical reward prediction error-like bursts. Furthermore, neither the acquisition nor the expression of a goal-tracking CR is dopamine-dependent. Here we present a computational model that can account for such individual variations. We show that a combination of a Model-Based system and a revised Model-Free system can account for the development of distinct CRs in rats. Moreover, we show that revising a classical Model-Free system to individually process stimuli by using factored representations can explain why classical dopaminergic patterns may be observed for some rats and not for others depending on the CR they develop. In addition, the model can account for other behavioural and pharmacological results obtained using the same, or similar, autoshaping procedures. Finally, the model makes it possible to draw a set of experimental predictions that may be verified in a modified experimental protocol. We suggest that further investigation of factored representations in computational neuroscience studies may be useful.

Human genetic and epidemiological studies provide evidence that only a subset of individuals who experiment with potentially addictive drugs become addicts. What renders some individuals susceptible to addiction remains to be determined, but most would agree that there is no single trait underlying the disorder. However, there is evidence in humans that addiction liability has a genetic component, and that certain personality characteristics related to temperament (e.g. the sensation-seeking trait) are associated with individual differences in addiction liability. Consequently, we have used a selective breeding strategy based on locomotor response to a novel environment to generate two lines of rats with distinct behavioral characteristics. We have found that the resulting phenotypes differ on a number of neurobehavioral dimensions relevant to addiction. Relative to bred low-responder (bLR) rats, bred high-responder (bHR) rats exhibit increased exploratory behavior, are more impulsive, more aggressive, seek stimuli associated with rewards, and show a greater tendency to relapse. We therefore utilize this unique animal model to parse the genetic, neural and environmental factors that contribute to addiction liability. Our work shows that the glucocorticoid receptor (GR), dopaminergic molecules, and members of the fibroblast growth factor family are among the neurotransmitters and neuromodulators that play a role in both the initial susceptibility to addiction as well as the altered neural responses that follow chronic drug exposure. Moreover, our findings suggest that the hippocampus plays a major role in mediating vulnerability to addiction. It is hoped that this work will emphasize the importance of personalized treatment strategies and identify novel therapeutic targets for humans suffering from addictive disorders.This article is part of a Special Issue entitled ‘NIDA 40th Anniversary Issue’.

Due in part to their rich behavioral repertoire rats have been widely used in behavioral studies of drug abuse-related traits for decades. However, the mouse became the model of choice for researchers exploring the genetic underpinnings of addiction after the first mouse study was published demonstrating the capability of engineering the mouse genome through embryonic stem cell technology. The sequencing of the mouse genome and more recent re-sequencing of numerous inbred mouse strains have further cemented the status of mice as the premier mammalian organism for genetic studies. As a result, many of the behavioral paradigms initially developed and optimized for rats have been adapted to mice. However, numerous complex and interesting drug abuse-related behaviors that can be studied in rats are very difficult or impossible to adapt for use in mice, impeding the genetic dissection of those traits. Now, technological advances have removed many of the historical limitations of genetic studies in rats. For instance, the rat genome has been sequenced and many inbred rat strains are now being re-sequenced and outbred rat stocks are being used to fine-map QTLs. In addition, it is now possible to create “knockout” rats using zinc finger nucleases (ZFN), transcription activator-like effector nucleases (TALENs) and related techniques. Thus, rats can now be used to perform quantitative genetic studies of sophisticated behaviors that have been difficult or impossible to study in mice.This article is part of a Special Issue entitled ‘NIDA 40th Anniversary Issue’.