Exploring multimodal analysis in VR-assisted language learning: Insights and applications

[EN] Multimodality recognizes that meaning making does not happen through language alone and examines the co-existence of various communicative modes, including sensory modes such as sight, hearing and touch (the haptic mode, concerned with tactile experience), as well as gestures, body movements, images, and both spoken and written language. As a theoretical and methodological framework, multimodality has been widely explored in education (Kress, 2012; Kress & van Leeuwen, 2001; Norris, 2004a, 2004b). However, its application within High-Immersion Virtual Reality (HiVR) environments has remained underexplored (Chen & Sevilla‐Pavón, 2023; Jauregi-Ondarra et al., 2024). With its capacity for high immersion, embodiment and sensory engagement, HiVR enables contextualized, situated learning social interactions since learners can simulate authentic conditions that resemble real-life scenarios (Christoforou et al., 2019) and learn by doing. This paper explores how multimodal analysis can be methodologically applied in HiVR environments by examining a single learning episode from a larger study on immersive language learning (Thrasher et al., 2024b). Through a comparative application of three frameworks - Multimodal Discourse Analysis (MDA), Multimodal Interaction Analysis (MIA), and the Grammar of Transposition - the study aims to contribute to the underresearched field of multimodal analysis in VR-assisted language learning (VRALL) and to offer a practical entry point for future research in immersive language education.