- -

General intelligence disentangled via a generality metric for natural and artificial intelligence

RiuNet: Repositorio Institucional de la Universidad Politécnica de Valencia

Compartir/Enviar a

Citas

Estadísticas

  • Estadisticas de Uso

General intelligence disentangled via a generality metric for natural and artificial intelligence

Mostrar el registro sencillo del ítem

Ficheros en el ítem

Thumbnail
Nombre: Hernandez-OralloL ...
Tamaño: 3.030Mb
Formato: PDF
Descripción: Versión editorial
Abrir
dc.contributor.author Hernández-Orallo, José es_ES
dc.contributor.author Loe, Bao Sheng es_ES
dc.contributor.author Cheke, Lucy es_ES
dc.contributor.author Martínez-Plumed, Fernando es_ES
dc.contributor.author Heigeartaigh, Sean O. es_ES
dc.date.accessioned 2022-03-24T19:03:35Z
dc.date.available 2022-03-24T19:03:35Z
dc.date.issued 2021-11-24 es_ES
dc.identifier.issn 2045-2322 es_ES
dc.identifier.uri http://hdl.handle.net/10251/181550
dc.description.abstract [EN] Success in all sorts of situations is the most classical interpretation of general intelligence. Under limited resources, however, the capability of an agent must necessarily be limited too, and generality needs to be understood as comprehensive performance up to a level of difficulty. The degree of generality then refers to the way an agent's capability is distributed as a function of task difficulty. This dissects the notion of general intelligence into two non-populational measures, generality and capability, which we apply to individuals and groups of humans, other animals and AI systems, on several cognitive and perceptual tests. Our results indicate that generality and capability can decouple at the individual level: very specialised agents can show high capability and vice versa. The metrics also decouple at the population level, and we rarely see diminishing returns in generality for those groups of high capability. We relate the individual measure of generality to traditional notions of general intelligence and cognitive efficiency in humans, collectives, non-human animals and machines. The choice of the difficulty function now plays a prominent role in this new conception of generality, which brings a quantitative tool for shedding light on long-standing questions about the evolution of general intelligence and the evaluation of progress in Artificial General Intelligence. es_ES
dc.description.sponsorship We are grateful to Jim DiCarlo and his lab (http://dicarlolab.mit.edu/, MIT), Judith Burkart, Laura Damerius and Carel van Schaik (Universitat Zurich), and Katherine Bruce and Mark Galizio (UNC Wilmington), for their data and the very helpful discussions. We thank David Stillwell for suggesting the relation between person-fit metrics and an early version of generality, and Heinrich Peters for pointing out Guttman's model. JHO has been partially supported by the EU (FEDER) and Spanish MINECO grant RTI2018-094403-B-C32 funded by MCIN/AEI/10.13039/501100011033 and by "ERDF A way of making Europe", Generalitat Valenciana under grant PROMETEO/2019/098 and EU's Horizon 2020 research and innovation programme under grant agreement No. 952215 (TAILOR). JHO and SOH are funded by the Future of Life Institute, FLI, under grant RFP2-152. JHO and LC are funded by US DARPA HR00112120007 (RECoG-AI). LC and SOH are funded by the Leverhulme Trust through a grant for the Leverhulme Centre for the Future of Intelligence. FMP is funded by the AI-Watch and HUMAINT projects by DG CONNECT and DG JRC of the European Commission. All authors declare no competing interests. We thank John Burden, Richard Evans and the anonymous reviewers for their insightful comments and corrections. Code and data are available for reproducibility at http://github.com/jorallo/gener ality as explained in G. es_ES
dc.language Inglés es_ES
dc.publisher Nature Publishing Group es_ES
dc.relation.ispartof Scientific Reports es_ES
dc.rights Reconocimiento (by) es_ES
dc.title General intelligence disentangled via a generality metric for natural and artificial intelligence es_ES
dc.type Artículo es_ES
dc.identifier.doi 10.1038/s41598-021-01997-7 es_ES
dc.relation.projectID info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/RTI2018-094403-B-C32/ES/RAZONAMIENTO FORMAL PARA TECNOLOGIAS FACILITADORAS Y EMERGENTES/ es_ES
dc.relation.projectID info:eu-repo/grantAgreement/DOD//HR00112120007/ es_ES
dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/952215/EU/ es_ES
dc.relation.projectID info:eu-repo/grantAgreement/FLI//RFP2-152/ es_ES
dc.relation.projectID info:eu-repo/grantAgreement/GVA//PROMETEO%2F2019%2F098//DEEPTRUST/ es_ES
dc.rights.accessRights Abierto es_ES
dc.description.bibliographicCitation Hernández-Orallo, J.; Loe, BS.; Cheke, L.; Martínez-Plumed, F.; Heigeartaigh, SO. (2021). General intelligence disentangled via a generality metric for natural and artificial intelligence. Scientific Reports. 11(1):1-16. https://doi.org/10.1038/s41598-021-01997-7 es_ES
dc.description.accrualMethod S es_ES
dc.relation.publisherversion https://doi.org/10.1038/s41598-021-01997-7 es_ES
dc.description.upvformatpinicio 1 es_ES
dc.description.upvformatpfin 16 es_ES
dc.type.version info:eu-repo/semantics/publishedVersion es_ES
dc.description.volume 11 es_ES
dc.description.issue 1 es_ES
dc.identifier.pmid 34819537 es_ES
dc.identifier.pmcid PMC8613222 es_ES
dc.relation.pasarela S\458368 es_ES
dc.contributor.funder Leverhulme Trust es_ES
dc.contributor.funder European Commission es_ES
dc.contributor.funder Generalitat Valenciana es_ES
dc.contributor.funder Future of Life Institute es_ES
dc.contributor.funder U.S. Department of Defense es_ES
dc.contributor.funder Agencia Estatal de Investigación es_ES
dc.contributor.funder European Regional Development Fund es_ES
dc.contributor.funder COMISION DE LAS COMUNIDADES EUROPEA es_ES
dc.description.references Pintner, R. Intelligence and its measurement: A symposium-v. J. Educ. Psychol. 12, 139 (1921). es_ES
dc.description.references Fogel, D. B. Evolutionary Computation: Toward a New Philosophy of Machine Intelligence (IEEE Press, 1995). es_ES
dc.description.references Legg, S. & Hutter, M. Universal intelligence: A definition of machine intelligence. Minds Mach. 17, 391–444 (2007). es_ES
dc.description.references Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015). es_ES
dc.description.references Yang, J. et al. General learning ability in perceptual learning. Proc. Natl. Acad. Sci. 117, 19092–19100 (2020). es_ES
dc.description.references Spearman, C. The Abilities of Man: Their Nature and Measurement (Macmillan, 1927). es_ES
dc.description.references Detterman, D. K. General intelligence: Cognitive and biological explanations. In The General Factor of Intelligence: How General is it? (eds Sternberg, R. J. & Grigorenko, E. L.) 223–243 (Lawrence Erlbaum Associates, 2002). es_ES
dc.description.references Burkart, J. M., Schubiger, M. N. & van Schaik, C. P. The evolution of general intelligence. Behav. Brain Sci. 40, e195 (2017). es_ES
dc.description.references Poirier, M.-A., Kozlovsky, D. Y., Morand-Ferron, J. & Careau, V. How general is cognitive ability in non-human animals? A meta-analytical and multi-level reanalysis approach. Proc. R. Soc. B 287, 20201853 (2020). es_ES
dc.description.references Newell, A., Shaw, J. C. & Simon, H. A. Report on a general problem-solving program. In IFIP Congress, 256–264 (1959). es_ES
dc.description.references McCarthy, J. Generality in artificial intelligence. Commun. ACM 30, 1030–1035 (1987). es_ES
dc.description.references Johnson, W. Correlation and explaining variance: To square or not to square?. Intelligence 39, 249–254 (2011). es_ES
dc.description.references Detterman, D. K. & Daniel, M. H. Correlations of mental tests with each other and with cognitive variables are highest for low IQ groups. Intelligence 13, 349–359 (1989). es_ES
dc.description.references Hernández-Orallo, J. The Measure of All Minds: Evaluating Natural and Artificial Intelligence (Cambridge University Press, 2017). es_ES
dc.description.references Hernández-Orallo, J. et al. A new AI evaluation cosmos: Ready to play the game? AI Magazine 38 (2017). es_ES
dc.description.references Hernández-Orallo, J. Evaluation in artificial intelligence: From task-oriented to ability-oriented measurement. Artif. Intell. Rev. 48, 397–447 (2017). es_ES
dc.description.references Lapuschkin, S. et al. Unmasking clever Hans predictors and assessing what machines really learn. Nat. Commun. 10, 1096 (2019). es_ES
dc.description.references Hernández-Orallo, J. Gazing into Clever Hans machines. Nat. Mach. Intell. 1 (2019). es_ES
dc.description.references Cave, S. & ÓhÉigeartaigh, S. S. Bridging near-and long-term concerns about AI. Nat. Mach. Intell. 1, 5–6 (2019). es_ES
dc.description.references Hernández-Orallo, J. Unbridled mental power. Nat. Phys. 15, 106 (2019). es_ES
dc.description.references Sternberg, R. J. The holey grail of general intelligence. Science 289, 399–401 (2000). es_ES
dc.description.references Carroll, J. B. The Three-Stratum Theory of Cognitive Abilities (The Guilford Press, 1997). es_ES
dc.description.references Sternberg, R. A triarchic view of giftedness: Theory and practice. Handbook of Gifted Education 2, 43–53 (1997). es_ES
dc.description.references Rust, J., Kosinski, M. & Stillwell, D. Modern Psychometrics: The Science of Psychological Assessment 4th edn. (Routledge, 2020). es_ES
dc.description.references Shaw, R. C. & Schmelz, M. Cognitive test batteries in animal cognition research: Evaluating the past, present and future of comparative psychometrics. Anim. Cogn. 20, 1003–1018 (2017). es_ES
dc.description.references Ramón, S. et al. Textura del sistema nervioso del hombre y de los vertebrados: estudios sobre el plan estructural y composición histológica de los centros nerviosos adicionados de consideraciones fisiológicas fundadas en los nuevos descubrimientos, vol. 1 (Moya, 1899). es_ES
dc.description.references Del Giudice, M. & Crespi, B. J. Basic functional trade-offs in cognition: An integrative framework. Cognition 179, 56–70 (2018). es_ES
dc.description.references Nielsen, T. S., Barros, G. A., Togelius, J. & Nelson, M. J. Towards generating arcade game rules with VGDL. In 2015 IEEE Conference on Computational Intelligence and Games (CIG), 185–192 (IEEE, 2015). es_ES
dc.description.references Insa-Cabrera, J., Dowe, D. L., España-Cubillo, S., Hernández-Lloreda, M. V. & Hernández-Orallo, J. Comparing humans and AI agents. In International Conference on Artificial General Intelligence, 122–132 (Springer, 2011). es_ES
dc.description.references Elithorn, A. A preliminary report on a perceptual maze test sensitive to brain damage. J. Neurol. Neurosurg. Psychiatry 18, 287 (1955). es_ES
dc.description.references Loe, B. S. & Rust, J. The perceptual maze test revisited: Evaluating the difficulty of automatically generated mazes. Assessment 26, 1524–1539 (2019). es_ES
dc.description.references Davies, A. D. & Davies, M. The difficulty and graded scoring of Elithorn’s perceptual maze test. Br. J. Psychol. 56, 295–302 (1965). es_ES
dc.description.references Thurstone, L. L. Primary mental abilities. Psychometric Monographs (1938). es_ES
dc.description.references Simon, H. A. & Kotovsky, K. Human acquisition of concepts for sequential patterns. Psychol. Rev. 70, 534 (1963). es_ES
dc.description.references Hernández-Orallo, J., Martínez-Plumed, F., Schmid, U., Siebers, M. & Dowe, D. L. Computer models solving intelligence test problems: Progress and implications. Artif. Intell. 230, 74–107 (2016). es_ES
dc.description.references Hernández-Orallo, J. Beyond the turing test. J. Logic Lang. Inf. 9, 447–466 (2000). es_ES
dc.description.references Katayama, S. An analytical inductive functional programming system that avoids unintended programs. In Proceedings of the ACM SIGPLAN 2012 Workshop on Partial Evaluation and Program Manipulation, 43–52 (ACM, 2012). es_ES
dc.description.references Rajalingham, R. et al. Large-scale, high-resolution comparison of the core visual object recognition behavior of humans, monkeys, and state-of-the-art deep artificial neural networks. J. Neurosci. 38, 7255–7269 (2018). es_ES
dc.description.references Young, J. W. et al. The odour span task: A novel paradigm for assessing working memory in mice. Neuropharmacology 52, 634–645 (2007). es_ES
dc.description.references April, L. B., Bruce, K. & Galizio, M. The magic number 70 (plus or minus 20): Variables determining performance in the rodent odor span task. Learn. Motiv. 44, 143–158 (2013). es_ES
dc.description.references Smith, M. R., Martinez, T. & Giraud-Carrier, C. An instance level analysis of data complexity. Mach. Learn. 95, 225–256 (2014). es_ES
dc.description.references Martínez-Plumed, F., Prudêncio, R. B., Martínez-Usó, A. & Hernández-Orallo, J. Item response theory in AI: Analysing machine learning classifiers at the instance level. Artif. Intell. 271, 18–42 (2019). es_ES
dc.description.references Morizot, J., Ainsworth, A. T. & Reise, S. P. Toward modern psychometrics. Handbook of Research Methods in Personality Psychology 407 (2009). es_ES
dc.description.references Herrmann, E., Call, J., Hernández-Lloreda, M. V., Hare, B. & Tomasello, M. Humans have evolved specialized skills of social cognition: The cultural intelligence hypothesis. Science 317, 1360–1366 (2007). es_ES
dc.description.references Koorehdavoudi, H. & Bogdan, P. A statistical physics characterization of the complex systems dynamics: Quantifying complexity from spatio-temporal interactions. Sci. Rep. 6, 1–13 (2016). es_ES
dc.description.references Balaban, V., Lim, S., Gupta, G., Boedicker, J. & Bogdan, P. Quantifying emergence and self-organisation of Enterobacter cloacae microbial communities. Sci. Rep. 8, 1–9 (2018). es_ES
dc.description.references Hazy, J. K. When the value of cooperating scales: Collective intelligence, information processing, and the leadership meta-capability. Front. Hum. Dyn. 3, 1 (2021). es_ES
dc.description.references Seed, A., Emery, N. & Clayton, N. Intelligence in corvids and apes: A case of convergent evolution?. Ethology 115, 401–420 (2009). es_ES
dc.description.references Jacobs, I. & Gärdenfors, P. The false dichotomy of domain-specific versus domain-general cognition. Behav. Brain Sci. 40, e207 (2017). es_ES
dc.description.references Brynjolfsson, E. & Mitchell, T. What can machine learning do? Workforce implications. Science 358, 1530–1534 (2017). es_ES
dc.description.references Brown, N. & Sandholm, T. Superhuman AI for multiplayer poker. Science 365, 885–890 (2019). es_ES
dc.description.references Rahwan, I. et al. Machine behaviour. Nature 568, 477–486 (2019). es_ES
dc.description.references Fleuret, F. et al. Comparing machines and humans on a visual categorization test. Proc. Natl. Acad. Sci. 108, 17621–17625 (2011). es_ES
dc.description.references Ullman, S., Assif, L., Fetaya, E. & Harari, D. Atoms of recognition in human and computer vision. Proc. Natl. Acad. Sci. 113, 2744–2749 (2016). es_ES
dc.description.references Firestone, C. Performance vs competence in human–machine comparisons. Proc. Natl. Acad. Sci. 117, 26562–26571 (2020). es_ES
dc.description.references Crosby, M. et al. The animal-AI testbed and competition. Proceedings of Machine Learning Research 164–176 (2020). es_ES
dc.description.references Shanahan, M., Crosby, M., Beyret, B. & Cheke, L. Artificial intelligence and the common sense of animals. Trends Cogn. Sci. 24, 862–872 (2020). es_ES
dc.description.references Tenenbaum, J. B. & Griffiths, T. L. Generalization, similarity, and Bayesian inference. Behav. Brain Sci. 24, 629 (2001). es_ES
dc.description.references Balla, M., Lucas, S. M. & Perez-Liebana, D. Evaluating generalisation in general video game playing. In 2020 IEEE Conference on Games (CoG), 423–430 (2020). es_ES
dc.description.references Risi, S. & Togelius, J. Increasing generality in machine learning through procedural content generation. Nat. Mach. Intell. 2, 428–436 (2020). es_ES
dc.description.references Schrittwieser, J. et al. Mastering Atari, go, chess and shogi by planning with a learned model. Nature 588, 604–609 (2020). es_ES
dc.description.references Brown, T. B. et al. Language models are few-shot learners. Advances in Neural Information Processing Systems 33 (NeurIPS 2020), arXiv preprintarXiv:2005.14165 (2020). es_ES


Este ítem aparece en la(s) siguiente(s) colección(ones)

Mostrar el registro sencillo del ítem