Is the Italian student survey on teaching reliable? For what purposes?

Abstract

[EN] Italian universities submit a compulsory survey to students for the evaluation of teaching activities. The questionnaire, designed by the Italian National Agency for University and Research System Evaluation (ANVUR), aims to evaluate four dimensions of teaching quality (course, instructor, personal interest and overall satisfaction) through twelve questions on a four-level scale. This paper addresses first the issue of the questionnaire’s reliability in the representation of the four evaluation dimensions. The main result is that the questionnaire do not represent properly the four dimensions of evaluation which it is intended for. Secondly, through a preliminary statistical analysis, it discusses the use of the survey for comparative purposes. A comparative analysis can be adversely affected by several contextual and subjective factors, like the size of the class and the gender of the instructor. The paper concludes by discussing the difficulty of finding proper conditioning, and raises doubts regarding an uncritical comparative use of student evaluations of teaching activities.