Integrative Center for Learning and Memory (ICLM) Journal Club – Tanya Gupta, Ph.D.

“Mesolimbic Dopamine Adapts the Rate of Learning from Action“
Nature, 18 Jan 2023

Tanya Gupta, Ph.D.
Postdoctoral Fellow
Laboratory of Dr. S. Wilke
Department ofPsychiatry
UCLA College of Life Sciences

In-Person
Gonda Research Center 1357

For more information, please contact the ICLM Journal Club at ICLM.JournalClub@gmail.com.

==============================

Abstract: Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction4; however, so far there hasbeen little consideration of how direct policy learning might inform our understanding. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encodingfor a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning.