- -

Attentional selection can be predicted by reinforcement learning of task-relevant stimulus features weighted by value-independent stickiness

RiuNet: Institutional repository of the Polithecnic University of Valencia

Share/Send to

Cited by

Statistics

Attentional selection can be predicted by reinforcement learning of task-relevant stimulus features weighted by value-independent stickiness

Show full item record

Balcarras, M.; Ardid-Ramírez, JS.; Kaping, D.; Everling, S.; Womelsdorf, T. (2016). Attentional selection can be predicted by reinforcement learning of task-relevant stimulus features weighted by value-independent stickiness. Journal of Cognitive Neuroscience. 28(2):333-349. https://doi.org/10.1162/jocn_a_00894

Por favor, use este identificador para citar o enlazar este ítem: http://hdl.handle.net/10251/167007

Files in this item

Item Metadata

Title: Attentional selection can be predicted by reinforcement learning of task-relevant stimulus features weighted by value-independent stickiness
Author: Balcarras, M. Ardid-Ramírez, Joan Salvador Kaping, D. Everling, S. Womelsdorf, T.
UPV Unit: Universitat Politècnica de València. Instituto de Investigación para la Gestión Integral de Zonas Costeras - Institut d'Investigació per a la Gestió Integral de Zones Costaneres
Issued date:
Abstract:
[EN] Attention includes processes that evaluate stimuli relevance, select the most relevant stimulus against less relevant stimuli, and bias choice behavior toward the selected information. It is not clear how these processes ...[+]
Subjects: Matching behavior , Economic choice , Frontal-Cortex , Reward , Decision , Information , Representations , Computations , Networks , Models
Copyrigths: Reserva de todos los derechos
Source:
Journal of Cognitive Neuroscience. (issn: 0898-929X )
DOI: 10.1162/jocn_a_00894
Publisher:
MIT Press
Publisher version: https://doi.org/10.1162/jocn_a_00894
Thanks:
This work was supported by grants from the Canadian Institutes of Health Research, the Natural Sciences and Engineering Research Council of Canada, and the Ontario Ministry of Economic Development and Innovation. We thank ...[+]
Type: Artículo

References

Ahn, W.-Y., Busemeyer, J. R., Wagenmakers, E.-J., & Stout, J. C. (2008). Comparison of Decision Learning Models Using the Generalization Criterion Method. Cognitive Science, 32(8), 1376-1402. doi:10.1080/03640210802352992

Alexander, W. H. (2007). Shifting Attention Using a Temporal Difference Prediction Error and High-Dimensional Input. Adaptive Behavior, 15(2), 121-133. doi:10.1177/1059712307078663

Anderson, B. A. (2013). A value-driven mechanism of attentional selection. Journal of Vision, 13(3), 7-7. doi:10.1167/13.3.7 [+]
Ahn, W.-Y., Busemeyer, J. R., Wagenmakers, E.-J., & Stout, J. C. (2008). Comparison of Decision Learning Models Using the Generalization Criterion Method. Cognitive Science, 32(8), 1376-1402. doi:10.1080/03640210802352992

Alexander, W. H. (2007). Shifting Attention Using a Temporal Difference Prediction Error and High-Dimensional Input. Adaptive Behavior, 15(2), 121-133. doi:10.1177/1059712307078663

Anderson, B. A. (2013). A value-driven mechanism of attentional selection. Journal of Vision, 13(3), 7-7. doi:10.1167/13.3.7

Anderson, B. A., Laurent, P. A., & Yantis, S. (2011). Value-driven attentional capture. Proceedings of the National Academy of Sciences, 108(25), 10367-10371. doi:10.1073/pnas.1104047108

Ardid, S., Balcarras, M., & Womelsdorf, T. (2014). «Adaptive learning» as a mechanistic candidate for reaching optimal task-set representations flexibly. BMC Neuroscience, 15(S1). doi:10.1186/1471-2202-15-s1-p8

Ardid, S., & Wang, X.-J. (2013). A Tweaking Principle for Executive Control: Neuronal Circuit Mechanism for Rule-Based Task Switching and Conflict Resolution. Journal of Neuroscience, 33(50), 19504-19517. doi:10.1523/jneurosci.1356-13.2013

Asaad, W. F., & Eskandar, E. N. (2008). A flexible software tool for temporally-precise behavioral control in Matlab. Journal of Neuroscience Methods, 174(2), 245-258. doi:10.1016/j.jneumeth.2008.07.014

Awh, E., Belopolsky, A. V., & Theeuwes, J. (2012). Top-down versus bottom-up attentional control: a failed theoretical dichotomy. Trends in Cognitive Sciences, 16(8), 437-443. doi:10.1016/j.tics.2012.06.010

Cai, X., & Padoa-Schioppa, C. (2014). Contributions of Orbitofrontal and Lateral Prefrontal Cortices to Economic Choice and the Good-to-Action Transformation. Neuron, 81(5), 1140-1151. doi:10.1016/j.neuron.2014.01.008

Chelazzi, L., Perlato, A., Santandrea, E., & Della Libera, C. (2013). Rewards teach visual selective attention. Vision Research, 85, 58-72. doi:10.1016/j.visres.2012.12.005

Collins, A. G. E., & Frank, M. J. (2013). Cognitive control over learning: Creating, clustering, and generalizing task-set structure. Psychological Review, 120(1), 190-229. doi:10.1037/a0030852

Dayan, P., & Berridge, K. C. (2014). Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation. Cognitive, Affective, & Behavioral Neuroscience, 14(2), 473-492. doi:10.3758/s13415-014-0277-8

Dayan, P., Kakade, S., & Montague, P. R. (2000). Learning and selective attention. Nature Neuroscience, 3(S11), 1218-1223. doi:10.1038/81504

De Wit, S., Watson, P., Harsay, H. A., Cohen, M. X., van de Vijver, I., & Ridderinkhof, K. R. (2012). Corticostriatal Connectivity Underlies Individual Differences in the Balance between Habitual and Goal-Directed Action Control. Journal of Neuroscience, 32(35), 12066-12075. doi:10.1523/jneurosci.1088-12.2012

Dehaene, S., & Changeux, J.-P. (2000). Reward-dependent learning in neuronal networks for planning and decision making. Cognition, emotion and autonomic responses: The integrative role of the prefrontal cortex and limbic structures, 217-229. doi:10.1016/s0079-6123(00)26016-0

Della Libera, C., & Chelazzi, L. (2009). Learning to Attend and to Ignore Is a Matter of Gains and Losses. Psychological Science, 20(6), 778-784. doi:10.1111/j.1467-9280.2009.02360.x

Dolan, R. J., & Dayan, P. (2013). Goals and Habits in the Brain. Neuron, 80(2), 312-325. doi:10.1016/j.neuron.2013.09.007

Donoso, M., Collins, A. G. E., & Koechlin, E. (2014). Foundations of human reasoning in the prefrontal cortex. Science, 344(6191), 1481-1486. doi:10.1126/science.1252254

Fecteau, J. H., & Munoz, D. P. (2003). Exploring the consequences of the previous trial. Nature Reviews Neuroscience, 4(6), 435-443. doi:10.1038/nrn1114

Gershman, S. J., & Niv, Y. (2010). Learning latent structure: carving nature at its joints. Current Opinion in Neurobiology, 20(2), 251-256. doi:10.1016/j.conb.2010.02.008

Glimcher, P. W. (2011). Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis. Proceedings of the National Academy of Sciences, 108(Supplement_3), 15647-15654. doi:10.1073/pnas.1014269108

Gottlieb, J. (2012). Attention, Learning, and the Value of Information. Neuron, 76(2), 281-295. doi:10.1016/j.neuron.2012.09.034

Gottlieb, J., Hayhoe, M., Hikosaka, O., & Rangel, A. (2014). Attention, Reward, and Information Seeking. Journal of Neuroscience, 34(46), 15497-15504. doi:10.1523/jneurosci.3270-14.2014

Hare, T. A., Schultz, W., Camerer, C. F., O’Doherty, J. P., & Rangel, A. (2011). Transformation of stimulus value signals into motor commands during simple choice. Proceedings of the National Academy of Sciences, 108(44), 18120-18125. doi:10.1073/pnas.1109322108

Huys, Q. J. M., Cools, R., Gölzer, M., Friedel, E., Heinz, A., Dolan, R. J., & Dayan, P. (2011). Disentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding. PLoS Computational Biology, 7(4), e1002028. doi:10.1371/journal.pcbi.1002028

Kaping, D., Vinck, M., Hutchison, R. M., Everling, S., & Womelsdorf, T. (2011). Specific Contributions of Ventromedial, Anterior Cingulate, and Lateral Prefrontal Cortex for Attentional Selection and Stimulus Valuation. PLoS Biology, 9(12), e1001224. doi:10.1371/journal.pbio.1001224

Kennerley, S. W., Behrens, T. E. J., & Wallis, J. D. (2011). Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nature Neuroscience, 14(12), 1581-1589. doi:10.1038/nn.2961

Krauzlis, R. J., Bollimunta, A., Arcizet, F., & Wang, L. (2014). Attention as an effect not a cause. Trends in Cognitive Sciences, 18(9), 457-464. doi:10.1016/j.tics.2014.05.008

Kristjánsson, Á. (2006). Rapid learning in attention shifts: A review. Visual Cognition, 13(3), 324-362. doi:10.1080/13506280544000039

Kristjánsson, Á., & Campana, G. (2010). Where perception meets memory: A review of repetition priming in visual search tasks. Attention, Perception, & Psychophysics, 72(1), 5-18. doi:10.3758/app.72.1.5

Kruschke, J. K., & Hullinger, R. A. (s. f.). Evolution of attention in learning. Computational Models of Conditioning, 10-52. doi:10.1017/cbo9780511760402.002

Lau, B., & Glimcher, P. W. (2005). DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS. Journal of the Experimental Analysis of Behavior, 84(3), 555-579. doi:10.1901/jeab.2005.110-04

Lau, B., & Glimcher, P. W. (2008). Value Representations in the Primate Striatum during Matching Behavior. Neuron, 58(3), 451-463. doi:10.1016/j.neuron.2008.02.021

Legenstein, R., Wilbert, N., & Wiskott, L. (2010). Reinforcement Learning on Slow Features of High-Dimensional Input Streams. PLoS Computational Biology, 6(8), e1000894. doi:10.1371/journal.pcbi.1000894

Luk, C.-H., & Wallis, J. D. (2013). Choice Coding in Frontal Cortex during Stimulus-Guided or Action-Guided Decision-Making. Journal of Neuroscience, 33(5), 1864-1871. doi:10.1523/jneurosci.4920-12.2013

Navalpakkam, V., Koch, C., Rangel, A., & Perona, P. (2010). Optimal reward harvesting in complex perceptual environments. Proceedings of the National Academy of Sciences, 107(11), 5232-5237. doi:10.1073/pnas.0911972107

Padoa-Schioppa, C. (2011). Neurobiology of Economic Choice: A Good-Based Model. Annual Review of Neuroscience, 34(1), 333-359. doi:10.1146/annurev-neuro-061010-113648

Passingham, R. E., & Wise, S. P. (2012). The Neurobiology of the Prefrontal Cortex. doi:10.1093/acprof:osobl/9780199552917.001.0001

Peck, C. J., Jangraw, D. C., Suzuki, M., Efem, R., & Gottlieb, J. (2009). Reward Modulates Attention Independently of Action Value in Posterior Parietal Cortex. Journal of Neuroscience, 29(36), 11182-11191. doi:10.1523/jneurosci.1929-09.2009

Peck, C. J., Lau, B., & Salzman, C. D. (2013). The primate amygdala combines information about space and value. Nature Neuroscience, 16(3), 340-348. doi:10.1038/nn.3328

Rangel, A., & Clithero, J. A. (2014). The Computation of Stimulus Values in Simple Choice. Neuroeconomics, 125-148. doi:10.1016/b978-0-12-416008-8.00008-5

Rangel, A., & Hare, T. (2010). Neural computations associated with goal-directed choice. Current Opinion in Neurobiology, 20(2), 262-270. doi:10.1016/j.conb.2010.03.001

Roelfsema, P. R., & Ooyen, A. van. (2005). Attention-Gated Reinforcement Learning of Internal Representations for Classification. Neural Computation, 17(10), 2176-2214. doi:10.1162/0899766054615699

Roelfsema, P. R., van Ooyen, A., & Watanabe, T. (2010). Perceptual learning rules based on reinforcers and attention. Trends in Cognitive Sciences, 14(2), 64-71. doi:10.1016/j.tics.2009.11.005

Rombouts, J. O., Bohte, S. M., Martinez-Trujillo, J., & Roelfsema, P. R. (2015). A learning rule that explains how rewards teach attention. Visual Cognition, 23(1-2), 179-205. doi:10.1080/13506285.2015.1010462

Rushworth, M. F. S., & Behrens, T. E. J. (2008). Choice, uncertainty and value in prefrontal and cingulate cortex. Nature Neuroscience, 11(4), 389-397. doi:10.1038/nn2066

Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011). Frontal Cortex and Reward-Guided Learning and Decision-Making. Neuron, 70(6), 1054-1069. doi:10.1016/j.neuron.2011.05.014

Seriès, P., & Seitz, A. R. (2013). Learning what to expect (in visual perception). Frontiers in Human Neuroscience, 7. doi:10.3389/fnhum.2013.00668

Seymour, B., & McClure, S. M. (2008). Anchors, scales and the relative coding of value in the brain. Current Opinion in Neurobiology, 18(2), 173-178. doi:10.1016/j.conb.2008.07.010

Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013). The Expected Value of Control: An Integrative Theory of Anterior Cingulate Cortex Function. Neuron, 79(2), 217-240. doi:10.1016/j.neuron.2013.07.007

Shteingart, H., & Loewenstein, Y. (2014). Reinforcement learning and human behavior. Current Opinion in Neurobiology, 25, 93-98. doi:10.1016/j.conb.2013.12.004

Smith, A. C., & Brown, E. N. (2003). Estimating a State-Space Model from Point Process Observations. Neural Computation, 15(5), 965-991. doi:10.1162/089976603765202622

Smith, A. C. (2004). Dynamic Analysis of Learning in Behavioral Experiments. Journal of Neuroscience, 24(2), 447-461. doi:10.1523/jneurosci.2908-03.2004

Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004). Matching Behavior and the Representation of Value in the Parietal Cortex. Science, 304(5678), 1782-1787. doi:10.1126/science.1094765

Summerfield, C., & Egner, T. (2009). Expectation (and attention) in visual cognition. Trends in Cognitive Sciences, 13(9), 403-409. doi:10.1016/j.tics.2009.06.003

Tatler, B. W., Hayhoe, M. M., Land, M. F., & Ballard, D. H. (2011). Eye guidance in natural vision: Reinterpreting salience. Journal of Vision, 11(5), 5-5. doi:10.1167/11.5.5

Tsotsos, J. K. (2011). A Computational Perspective on Visual Attention. doi:10.7551/mitpress/9780262015417.001.0001

Van der Meer, M., Kurth-Nelson, Z., & Redish, A. D. (2012). Information Processing in Decision-Making Systems. The Neuroscientist, 18(4), 342-359. doi:10.1177/1073858411435128

Wirth, S., Yanike, M., Frank, L. M., Smith, A. C., Brown, E. N., & Suzuki, W. A. (2003). Single Neurons in the Monkey Hippocampus and Learning of New Associations. Science, 300(5625), 1578-1581. doi:10.1126/science.1084324

Womelsdorf, T., & Everling, S. (2015). Long-Range Attention Networks: Circuit Motifs Underlying Endogenously Controlled Stimulus Selection. Trends in Neurosciences, 38(11), 682-700. doi:10.1016/j.tins.2015.08.009

Wunderlich, K., Rangel, A., & O’Doherty, J. P. (2010). Economic choices can be made using only stimulus values. Proceedings of the National Academy of Sciences, 107(34), 15005-15010. doi:10.1073/pnas.1002258107

[-]

recommendations

 

This item appears in the following Collection(s)

Show full item record