Thumbnail Image
Publication

Application of hemodynamic prefrontal cortex desirability signals as reinforcers in machine learning

Journal Title
Keywords
Readers/Advisors
Francis, Joseph T.
Journal Title
Term and Year
Spring 2012
Publication Date
2012-06-11
Book Title
Publication Volume
Publication Issue
Publication Begin
Publication End
Number of pages
Collections
Downstate School of Graduate Studies Theses and Dissertations
Show all metadata
Files
Thumbnail Image
Doctoral Dissertation
Adobe PDF, 9.85 MB
Research Projects
Organizational Units
Journal Issue
URI
http://hdl.handle.net/20.500.12648/15885
Abstract
Decision-making ability in the frontal lobe (among other brain structures) relies on the assignment of value to states of the organism and its environment. Then higher valued states can be pursued and lower valued (or negative) states avoided. The same principle forms the basis for computational reinforcement learning controllers, which have been fruitfully applied both as models of value estimation in the brain, and as artificial controllers in their own right. This work shows how state value signals decoded from frontal lobe hemodynamics, as measured with near-infrared spectroscopy (NIRS), can be applied as reinforcers to an adaptable artificial learning agent in order to guide its acquisition of skills. A set of experiments carried out on an alert macaque demonstrate that both oxy- and deoxyhemoglobin concentrations in the frontal lobe show differences in response to both primarily and secondarily desirable (versus undesirable) stimuli. This difference allows a NIRS signal classifier to serve successfully as a reinforcer for an adaptive controller performing a virtual tool-retrieval task. The agent’s adaptability allows its performance to exceed the limits of the NIRS classifier decoding accuracy. We also show that decoding state desirabilities is more accurate when using relative concentrations of both oxyhemoglobin and deoxyhemoglobin, rather than either species alone. This suggests that studies of value-related brain function that infer patterns of neural activity based solely on deoxyhemoglobin concentration (as in fMRI) may yet be incomplete. This work is part of a continuing investigation into the use of reinforcement learning agents as controllers in brain-machine interfaces.
Citation
DiStasio, M. (2012). Application of hemodynamic prefrontal cortex desirability signals as reinforcers in machine learning. [Doctoral dissertation, SUNY Downstate Health Sciences University]. SUNY Open Access Repository. https://soar.suny.edu/handle/20.500.12648/15885
DOI
Description
Doctoral Dissertation
Accessibility Statement
Embedded videos