A Rasch analysis validation of a survey on the use and beliefs of machine translation

Abstract

[EN] Many English as a Foreign Language (EFL) students use Machine Translation (MT) to a varying degree and studies on the students’ use and beliefs of MT are primarily conducted through surveys and interviews. It is important for educators to understand students’ use and beliefs related to MT, however, there are few studies in the Japanese setting, and none validated the surveys prior to implementation. The goal of this study was to validate a survey on MT to be used in the Japanese context. Considerations for validity are crucial when designing surveys for quantitative analysis. In this study, a 41-item Likert scale survey designed to understand students’ opinions of MT use and beliefs in the Japanese tertiary education system was piloted with 93 first-year EFL learners in an academic writing class. Subsets of items targeting use, volume of text, and degree of acceptability were validated and optimized using the Rasch analysis. Results showed that further work is required to increase items for subsets and to reoptimize the instrument.