AHP-based software for students' self-assessment of critical thinking capacity

Abstract

[EN] Students' capacity for eliciting valid and accurate assessment is crucial for their self-assessment skill. Despite the central role that self-assessment plays in current conceptions of formative and classroom assessment, little attention has been paid for measuring the students' ability for carrying out valid and accurate assessment. This ability, connected with critical thinking, can be evaluated by comparing a students¿ assessment to performance on tests. Recently, the consistency index has been proposed as a metric for measuring students¿ critical thinking capacity, affording the observation of poor performance in the judgment capacity of a sample of 23 civil engineering students. This stresses the need of improving this aspect of students¿ education, which is required not only as a transversal competence, but also for the attainment of effective self-assessment capacity. In our study, we present an educational software that implements the Analytic Hierarchy Process (AHP) method for providing students with self-assessment of their ability to issue consistent judgment, as well as for training their critical thinking capacity. This way we aim at making students aware of eventual lacks in their skill for valid and consistent assessment, as well as at giving them the opportunity of improving this aspect through self-assessment. The educational software embodies a three-step process, in which the self-assessment takes place in the last stage: first, the student is asked to carry out a conventional teacher assessment based on a set of criteria, which are then weighted by him/her through AHP without consistency control. Second, students are asked to turn on the software¿s consistency index control, which will reveal eventual inconsistencies, and accordingly revise his/her judgments until they are acceptable. Finally, the student is invited to self-assess his/her capacity to issue consistent judgment by analysing differences between their assessment with (step 2) and without (step 1) the aid of the consistency control, to reflect upon the differences between consistent and inconsistent assessment, and to think of potential causes driving to inconsistent judgment. The whole process can be performed by the students in the presented software, which serves as a platform for effective self-feedback and training of their assessment accuracy and critical thinking capacity.