Integrative Center for Learning and Memory (ICLM) Journal Club – Linfan Gu

April 21, 2023

9:30am – 10:30am


Gonda 1357

Arithmetic Value Representation for Hierarchical Behavior Composition
H. Makino, Nature (2022)

Linfan Gu
Graduate Student
Laboratory of Dr. W. Hong
Department of Biological Chemistry
David Geffen School of Medicine at UCLA

Gonda ResearchCenter 1357

For more information, please contact the ICLM Journal Club at


Abstract: The ability to compose new skills from a preacquired behavior repertoire is a hallmark of biological intelligence. Although artificialagents extract reusable skills from past experience and recombine them in a hierarchical manner, whether the brain similarly composes a novel behavior is largely unknown. In the present study, I show that deep reinforcement learning agents learn to solve a novel composite task by additively combining representations of prelearned action values of constituent subtasks. Learning efficacy in the composite task was further augmented by the introduction of stochasticity in behavior during pretraining. These theoretical predictions were empirically tested in mice, where subtask pretraining enhanced learning of the composite task. Cortex-wide, two-photon calcium imaging revealed analogous neural representations of combined action values, with improved learning when the behavior variability was amplified. Together,these results suggest that the brain composes a novel behavior with a simple arithmetic operation of preacquired action-value representations with stochastic policies.