Training a spiking neuronal network model of visual-motor cortex to play a virtual racket-ball game using reinforcement learning.
Cast your vote
You can rate an item by clicking the amount of stars they wish to award to this item.
When enough users have cast their vote on this item, the average rating will also be shown.
Your vote was cast
Thank you for your feedback
Thank you for your feedback
Chadderdon, George L
Kerr, Cliff C
Lytton, William W
Neymotin, Samuel A
Journal titlePloS one
Publication Begin pagee0265808
MetadataShow full item record
AbstractRecent models of spiking neuronal networks have been trained to perform behaviors in static environments using a variety of learning rules, with varying degrees of biological realism. Most of these models have not been tested in dynamic visual environments where models must make predictions on future states and adjust their behavior accordingly. The models using these learning rules are often treated as black boxes, with little analysis on circuit architectures and learning mechanisms supporting optimal performance. Here we developed visual/motor spiking neuronal network models and trained them to play a virtual racket-ball game using several reinforcement learning algorithms inspired by the dopaminergic reward system. We systematically investigated how different architectures and circuit-motifs (feed-forward, recurrent, feedback) contributed to learning and performance. We also developed a new biologically-inspired learning rule that significantly enhanced performance, while reducing training time. Our models included visual areas encoding game inputs and relaying the information to motor areas, which used this information to learn to move the racket to hit the ball. Neurons in the early visual area relayed information encoding object location and motion direction across the network. Neuronal association areas encoded spatial relationships between objects in the visual scene. Motor populations received inputs from visual and association areas representing the dorsal pathway. Two populations of motor neurons generated commands to move the racket up or down. Model-generated actions updated the environment and triggered reward or punishment signals that adjusted synaptic weights so that the models could learn which actions led to reward. Here we demonstrate that our biologically-plausible learning rules were effective in training spiking neuronal network models to solve problems in dynamic environments. We used our models to dissect the circuit architectures and learning rules most effective for learning. Our model shows that learning mechanisms involving different neural circuits produce similar performance in sensory-motor tasks. In biological networks, all learning mechanisms may complement one another, accelerating the learning capabilities of animals. Furthermore, this also highlights the resilience and redundancy in biological systems.
CitationAnwar H, Caby S, Dura-Bernal S, D'Onofrio D, Hasegan D, Deible M, Grunblatt S, Chadderdon GL, Kerr CC, Lakatos P, Lytton WW, Hazan H, Neymotin SA. Training a spiking neuronal network model of visual-motor cortex to play a virtual racket-ball game using reinforcement learning. PLoS One. 2022 May 11;17(5):e0265808. doi: 10.1371/journal.pone.0265808. PMID: 35544518; PMCID: PMC9094569.
The following license files are associated with this item:
- Creative Commons
Except where otherwise noted, this item's license is described as Attribution-NonCommercial-NoDerivatives 4.0 International
- Reinforcement learning of targeted movement in a spiking neuronal model of motor cortex.
- Authors: Chadderdon GL, Neymotin SA, Kerr CC, Lytton WW
- Issue date: 2012
- A Simple Network Architecture Accounts for Diverse Reward Time Responses in Primary Visual Cortex.
- Authors: Huertas MA, Hussain Shuler MG, Shouval HZ
- Issue date: 2015 Sep 16
- Mirrored STDP Implements Autoencoder Learning in a Network of Spiking Neurons.
- Authors: Burbank KS
- Issue date: 2015 Dec
- A Scalable Weight-Free Learning Algorithm for Regulatory Control of Cell Activity in Spiking Neuronal Networks.
- Authors: Zhang X, Foderaro G, Henriquez C, Ferrari S
- Issue date: 2018 Mar
- Reinforcement Learning of Linking and Tracing Contours in Recurrent Neural Networks.
- Authors: Brosch T, Neumann H, Roelfsema PR
- Issue date: 2015 Oct