A Reinforcement Learning Model of Temporal Difference Variations for Action-Selection and Action-Execution in the Human Brain

Abstract

Temporal difference (TD) prediction error signal models are instrumental in simulating brain function during reinforcement learning (RL). Recent evidence suggests a significant role of TD prediction error signals in the actionselection and action-execution brain networks. We introduce a neurocomputational model that explores TD prediction error signal variations for action-selection and action-execution. The TD prediction error signal represents the dopamine neurotransmitter the basal ganglia and prefrontal cortex brain regions. The model incorporates dopamine genetic parameters in the two networks (COMT gene for action-selection; DAT1 gene for action-execution) to generate four different parameter combinations. The model simulation showed that TD signaling in both networks plays a significant role in RL under optimal conditions of medium, not high, TD signals. Moreover, each parameter combination showed a unique pattern of RL, corresponding with experimental data obtained using a computer-based RL task