Hernández Orallo, José(Springer International Publishing, 2015-07-15)
We explore the aggregation of tasks by weighting them using a difficulty
function that depends on the complexity of the (acceptable) policy for the task (instead
of a universal distribution over tasks or an adaptive ...
José Hernández-Orallo(Springer Verlag (Germany), 2016-08-19)
The evaluation of artificial intelligence systems and components is crucial for the
progress of the discipline. In this paper we describe and critically assess the different ways
AI systems are evaluated, and the role ...
The evaluation of an ability or skill happens in some kind of testbed, and so does with social intelligence. Of course, not all testbeds are suitable for this matter. But, how can we be sure of their appropriateness? In ...
José Hernández-Orallo(Springer Verlag (Germany), 2015-05)
This paper presents a way to estimate the difficulty and discriminating power of
any task instance. We focus on a very general setting for tasks: interactive (possibly multiagent)
environments where an agent acts upon ...
Hernández Orallo, José(Springer International Publishing, 2015)
We establish a setting for asynchronous stochastic tasks that
account for episodes, rewards and responses, and, most especially, the
computational complexity of the algorithm behind an agent solving a
task. This is used ...