When Sigmund Freud first presented his theories of the superego and the id to an unsuspecting world at the dawn of the 20th century, it quickly met with an outcry – controversial even in the very young pioneering field that was psychology. Not only were we just some species that was next in line, the latest subset of primates, as biology of the day would tell us, but we don’t even know why we do half the things we do.
Things have gotten a bit mo
re sophisticated over the last century, as the beginning of the 21st century saw the growing field of neuroscience make some even more surprising discoveries – both in understanding the brain’s mechanisms, as well as how its various chemical reactions play a role in our thought patterns as well as neurological disorders. Dopamine, for example, has attracted the scrutiny of researchers for many years – even after pharmaceuticals were able to synthesize it.
Over the last few decades, a growing body of neuroscience studies has indicated that transient boosts in the brain’s supply of dopamine actually work as critical signals that help the brain learn about reward, and give it the motivation necessary to obtain further rewards. Researchers at University of California San Francisco recently conducted a study that explores the effect of transient increases in dopamine to designated subregions of the brain’s striatum, a region of the brain that is essential to both our reward-based learning as well as our decision-making.
These new findings, published by the journal Nature Neuroscience, take things a step further. They suggest that the dopamine transient fluctuations throughout three specific parts of the striatum reflect an individual’s reward predictions across various time horizons (that is, whether the reaction occurs within a fraction of a second, as opposed to tens of seconds or even hundreds of seconds later, with trials for all three).
“A dopamine pulse may indicate that we have found ourselves in a better situation than we expected, and so our prior estimates of reward need to be updated,” according to Josh Berke, a neuroscientist at the UC – San Francisco, who was the lead author of this paper. “This ‘reward prediction error’ theory has been very influential, in part because it connects brain activity patterns to certain computations in machine learning, yet there are several aspects of this theory that are, at best, incomplete.”
This new study from Berke and his research team was targeted at solving two problematic facets of the ‘reward prediction error’ theory. First of all, the researchers had to deal with a rather broad definition of the phrase ‘reward prediction,’ since it cannot specify when the brain is expected to find its reward (Does it occur within one second, one minute, one hour, etc.?).
“A second problem is that dopamine signals were originally thought to be broadcast uniformly throughout the forebrain, but more recent studies have found different dopamine signals at different places in the brain,” says Berke. “So, does this indicate that we need many different theories to explain these different dopamine signals?”
To further clarify the ambiguity on aspects of their reward prediction error theory, Berke and the research team performed a series of experiments with lab rats with the aid of a recently developed molecular sensor. The sensor uses a genetically programmed protein signal that alters its fluorescence whenever a bond with dopamine is formed.
“We deployed this sensor in three different areas within the rat striatum, the brain area that receives the strongest dopamine input,” Berke explains. “These different areas are part of distinct, large-scale loop circuits in the brain, which process different types of information.”
The researchers probed any fluctuations of dopamine, highlighted with the aid of the new molecular sensor, as their rats partook in a wide array of simple behavioral tasks. The rats were then given a small reward at varying rates of intensity and these rewards were prefaced with auditory cues, each of which indicated that rewards were soon coming, but with alternating delays and probabilities. One sound distinguished that a reward was imminent for example, whereas another suggested that the reward was coming within 60 seconds, and so forth.
Curiously, they discovered that the dopamine fluctuations occurring within each of the three striatal subregions they analyzed could correlate with the different time scales the rats’ internally memorized for their expectation of the rewards.
“In one area, most concerned with motor control, dopamine fluctuates frequently, and the response to a reward-predictive cue is strong only if it predicts reward delivery within a fraction of a second,” says Berke. “A second striatal area seems to care about rewards within tens of seconds, and a third about hundreds of seconds. We think there may be a continuous gradient of reward prediction time scales, involving parallel circuits within the brain.”
The difference in the time scales for reward processing and the striatal subregions associated with this function, as revealed by Berke and the UCSF research team could be able to explain the bizarre experimental observations previously recorded in literature when it comes to recording divergent dopamine signals, all making use of just a single theoretical framework. In addition to this, the fact that different reward prediction time scales exist could be crucial when it comes to underpinning some of the more complex and sometimes what seem like incoherent behaviors that are commonly observed in animals.
“For example, when singing a song, there is very little time separation between moving our vocal cords and hearing the pleasant (or otherwise) result,” suggests Berke. “This feedback must be fast for effective learning. However, sometimes we make choices and don’t find out the results for a considerable time. We need brain mechanisms to overcome this time gap, to determine whether the choice was a good one.”
The researchers’ new findings could be catalytic to helping us deepen our understanding of the correlation between both dopamine transients in our striatal subregions and the nature of reward-based learning. In addition to that, they could also play a critical role in revealing why we often make the decisions we make and determine how much of the brain plays an active role in any given moment.
“Often, we make resolutions about how to act over the long term (e.g., to lose weight), but when faced with an immediate choice we don’t act accordingly,” Berke explains.
“This mismatch has long been studied and was considered ‘irrational,’ However, it’s possible that this is an inevitable consequence of having multiple decision sub-systems operating in parallel, each concerned with a different duration of the future. As potential outcomes get closer in time, more sub-systems get involved, and push for short-term results.”
In the immediate future, Berke’s new study could aid in the conception and development of new and better detailed theoretical models that demonstrate the prediction of rewards offered at alternate time scales. For the time being, however, Berke and his research team hope to continue building on their findings by conducting further experiments that analyze dopamine signals and the way in which they interact with their neighboring neural circuits throughout the brain.
“We’re now investigating how these dopamine signals interact with other circuit components, as part of developing a richer understanding of how these circuits work and how they go wrong in disorders such as addiction, Parkinson’s Disease, and Tourette Syndrome,” says Berke. “We also have an active program studying how we imagine future possibilities and adjust our behavior accordingly.
