Further Details on Predicting IRT Difficulty

Abstract

This supplementary material serves as technical appendix of the paper When AI Difficulty is Easy: The Explanatory Power of Predicting IRT Difficulty (Martínez-Plumed et al. 2022), published in The Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22). The following sections give detailed information about 1) data gathering for benchmarks; 2) IRT properties and methodology followed; 3) learning models configuration and hyperparameter setting; 4) differences between difficulty prediction and class prediction; 5) the deployment and results of alternative approaches for difficulty estimation; 6) specifics and results using a generic difficulty metric in different applications and 7) extended IRT applications.