Show simple item record

dc.contributor.authorAnwar, Haroon
dc.contributor.authorCaby, Simon
dc.contributor.authorDura-Bernal, Salvador
dc.contributor.authorD'Onofrio, David
dc.contributor.authorHasegan, Daniel
dc.contributor.authorDeible, Matt
dc.contributor.authorGrunblatt, Sara
dc.contributor.authorChadderdon, George L
dc.contributor.authorKerr, Cliff C
dc.contributor.authorLakatos, Peter
dc.contributor.authorLytton, William W
dc.contributor.authorHazan, Hananel
dc.contributor.authorNeymotin, Samuel A
dc.date.accessioned2023-04-10T15:35:41Z
dc.date.available2023-04-10T15:35:41Z
dc.date.issued2022-05-11
dc.identifier.citationAnwar H, Caby S, Dura-Bernal S, D'Onofrio D, Hasegan D, Deible M, Grunblatt S, Chadderdon GL, Kerr CC, Lakatos P, Lytton WW, Hazan H, Neymotin SA. Training a spiking neuronal network model of visual-motor cortex to play a virtual racket-ball game using reinforcement learning. PLoS One. 2022 May 11;17(5):e0265808. doi: 10.1371/journal.pone.0265808. PMID: 35544518; PMCID: PMC9094569.en_US
dc.identifier.eissn1932-6203
dc.identifier.doi10.1371/journal.pone.0265808
dc.identifier.pmid35544518
dc.identifier.urihttp://hdl.handle.net/20.500.12648/8558
dc.description.abstractRecent models of spiking neuronal networks have been trained to perform behaviors in static environments using a variety of learning rules, with varying degrees of biological realism. Most of these models have not been tested in dynamic visual environments where models must make predictions on future states and adjust their behavior accordingly. The models using these learning rules are often treated as black boxes, with little analysis on circuit architectures and learning mechanisms supporting optimal performance. Here we developed visual/motor spiking neuronal network models and trained them to play a virtual racket-ball game using several reinforcement learning algorithms inspired by the dopaminergic reward system. We systematically investigated how different architectures and circuit-motifs (feed-forward, recurrent, feedback) contributed to learning and performance. We also developed a new biologically-inspired learning rule that significantly enhanced performance, while reducing training time. Our models included visual areas encoding game inputs and relaying the information to motor areas, which used this information to learn to move the racket to hit the ball. Neurons in the early visual area relayed information encoding object location and motion direction across the network. Neuronal association areas encoded spatial relationships between objects in the visual scene. Motor populations received inputs from visual and association areas representing the dorsal pathway. Two populations of motor neurons generated commands to move the racket up or down. Model-generated actions updated the environment and triggered reward or punishment signals that adjusted synaptic weights so that the models could learn which actions led to reward. Here we demonstrate that our biologically-plausible learning rules were effective in training spiking neuronal network models to solve problems in dynamic environments. We used our models to dissect the circuit architectures and learning rules most effective for learning. Our model shows that learning mechanisms involving different neural circuits produce similar performance in sensory-motor tasks. In biological networks, all learning mechanisms may complement one another, accelerating the learning capabilities of animals. Furthermore, this also highlights the resilience and redundancy in biological systems.
dc.language.isoenen_US
dc.relation.urlhttps://journals.plos.org/plosone/article?id=10.1371/journal.pone.0265808en_US
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.titleTraining a spiking neuronal network model of visual-motor cortex to play a virtual racket-ball game using reinforcement learning.en_US
dc.typeArticle/Reviewen_US
dc.source.journaltitlePloS oneen_US
dc.source.volume17
dc.source.issue5
dc.source.beginpagee0265808
dc.source.endpage
dc.source.countryUnited States
dc.source.countryUnited States
dc.source.countryUnited States
dc.source.countryUnited States
dc.description.versionVoRen_US
refterms.dateFOA2023-04-10T15:35:41Z
html.description.abstractRecent models of spiking neuronal networks have been trained to perform behaviors in static environments using a variety of learning rules, with varying degrees of biological realism. Most of these models have not been tested in dynamic visual environments where models must make predictions on future states and adjust their behavior accordingly. The models using these learning rules are often treated as black boxes, with little analysis on circuit architectures and learning mechanisms supporting optimal performance. Here we developed visual/motor spiking neuronal network models and trained them to play a virtual racket-ball game using several reinforcement learning algorithms inspired by the dopaminergic reward system. We systematically investigated how different architectures and circuit-motifs (feed-forward, recurrent, feedback) contributed to learning and performance. We also developed a new biologically-inspired learning rule that significantly enhanced performance, while reducing training time. Our models included visual areas encoding game inputs and relaying the information to motor areas, which used this information to learn to move the racket to hit the ball. Neurons in the early visual area relayed information encoding object location and motion direction across the network. Neuronal association areas encoded spatial relationships between objects in the visual scene. Motor populations received inputs from visual and association areas representing the dorsal pathway. Two populations of motor neurons generated commands to move the racket up or down. Model-generated actions updated the environment and triggered reward or punishment signals that adjusted synaptic weights so that the models could learn which actions led to reward. Here we demonstrate that our biologically-plausible learning rules were effective in training spiking neuronal network models to solve problems in dynamic environments. We used our models to dissect the circuit architectures and learning rules most effective for learning. Our model shows that learning mechanisms involving different neural circuits produce similar performance in sensory-motor tasks. In biological networks, all learning mechanisms may complement one another, accelerating the learning capabilities of animals. Furthermore, this also highlights the resilience and redundancy in biological systems.
dc.description.institutionSUNY Downstateen_US
dc.description.departmentPhysiology and Pharmacologyen_US
dc.description.departmentNathan Kline Institute for Psychiatric Researchen_US
dc.description.degreelevelN/Aen_US
dc.identifier.journalPloS one


Files in this item

Thumbnail
Name:
journal.pone.0265808.pdf
Size:
6.070Mb
Format:
PDF

This item appears in the following Collection(s)

Show simple item record

Attribution-NonCommercial-NoDerivatives 4.0 International
Except where otherwise noted, this item's license is described as Attribution-NonCommercial-NoDerivatives 4.0 International