Neural activity in the primary motor cortex is reminiscent of temporal difference reinforcement learning.

Abstract

A key component of Reinforcement Learning (RL) is the value function, which captures the expected, temporally discounted reward, from a given state. The value function can also be modified by the animal’s knowledge and certainty of its environment. Here we show that motor cortex (M1) encodes a value function in line with temporal difference RL. M1 responds to the delivery of unpredictable reward, and shifts its value related response earlier in atrial, becoming predictive of expected reward, when reward is predictable. M1also encodes multiple levels of reward. This is observed in tasks performed manually or observed passively. The Microstimulus Temporal Difference (MSTD)model, known to accurately capture RL related dopaminergic activity, extends to account for M1 reward related neural activity.