Dopamine transients do not act as model-free prediction errors during associative learning
MetadataShow full item record
AbstractDopamine neurons are proposed to signal the reward prediction error in model-free reinforcement learning algorithms. This term represents the unpredicted or 'excess' value of the rewarding event, value that is then added to the intrinsic value of any antecedent cues, contexts or events. To support this proposal, proponents cite evidence that artificially-induced dopamine transients cause lasting changes in behavior. Yet these studies do not generally assess learning under conditions where an endogenous prediction error would occur. Here, to address this, we conducted three experiments where we optogenetically activated dopamine neurons while rats were learning associative relationships, both with and without reward. In each experiment, the antecedent cues failed to acquire value and instead entered into associations with the later events, whether valueless cues or valued rewards. These results show that in learning situations appropriate for the appearance of a prediction error, dopamine transients support associative, rather than model-free, learning. Copyright 2020, The Author(s).
SponsorsNational Institute on Drug Abuse, NIDA: ZIA-DA000587
Identifier to cite or link to this itemhttps://www.scopus.com/inward/record.uri?eid=2-s2.0-85077558744&doi=10.1038%2fs41467-019-13953-1&partnerID=40&md5=654e5697ba5dc3ed9fd422150f4ba829; http://hdl.handle.net/10713/11671
- Optogenetic Blockade of Dopamine Transients Prevents Learning Induced by Changes in Reward Features.
- Authors: Chang CY, Gardner M, Di Tillio MG, Schoenbaum G
- Issue date: 2017 Nov 20
- Dopamine transients are sufficient and necessary for acquisition of model-based associations.
- Authors: Sharpe MJ, Chang CY, Liu MA, Batchelor HM, Mueller LE, Jones JL, Niv Y, Schoenbaum G
- Issue date: 2017 May
- A causal link between prediction errors, dopamine neurons and learning.
- Authors: Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH
- Issue date: 2013 Jul
- Causal evidence supporting the proposal that dopamine transients function as temporal difference prediction errors.
- Authors: Maes EJP, Sharpe MJ, Usypchuk AA, Lozzi M, Chang CY, Gardner MPH, Schoenbaum G, Iordanova MD
- Issue date: 2020 Feb
- Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework.
- Authors: Sadacca BF, Jones JL, Schoenbaum G
- Issue date: 2016 Mar 7