April 21, 2023
9:30 am / 10:30 am
“Arithmetic Value Representation for Hierarchical Behavior Composition“
H. Makino, Nature (2022)
Laboratory of Dr. W. Hong
Department of Biological Chemistry
David Geffen School of Medicine at UCLA
Gonda ResearchCenter 1357
For more information, please contact the ICLM Journal Club at ICLM.JournalClub@gmail.com.
Abstract: The ability to compose new skills from a preacquired behavior repertoire is a hallmark of biological intelligence. Although artificialagents extract reusable skills from past experience and recombine them in a hierarchical manner, whether the brain similarly composes a novel behavior is largely unknown. In the present study, I show that deep reinforcement learning agents learn to solve a novel composite task by additively combining representations of prelearned action values of constituent subtasks. Learning efficacy in the composite task was further augmented by the introduction of stochasticity in behavior during pretraining. These theoretical predictions were empirically tested in mice, where subtask pretraining enhanced learning of the composite task. Cortex-wide, two-photon calcium imaging revealed analogous neural representations of combined action values, with improved learning when the behavior variability was amplified. Together,these results suggest that the brain composes a novel behavior with a simple arithmetic operation of preacquired action-value representations with stochastic policies.