Yue Li
School of English Studies, Beijing Language and Culture University, Beijing 100083, China
With the development of technologies and pedagogies, the multimodality of language education is becoming increasingly important, especially in the context of the challenges posed by the COVID-19 pandemic. Through advanced multimodal theories and media, teachers could integrate multiple sensory and communicative modes in language teaching. This study uses bibliometric analysis to explore the multimodality in language education from both qualitative and quantitative aspects. The author uses tools such as VOSviewer, CiteSpace, and CitNetExplorer to visualize this topic’s research evolution. The findings show that multimodal media and pedagogies evolve to boost student engagement and understanding in language learning, particularly in terms of video-mediated interaction and social media. Future research is recommended to assess the long-term influences of multimodal approaches and explore their integration with emerging educational technologies.
Adetayo, A.J., Enamudu, A.I., Lawal, F.M., et al., 2024. From text to video with AI: The rise and potential of Sora in education and libraries. Library Hi Tech News. ahead-of-print(ahead-of-print). DOI: https://doi.org/10.1108/LHTN-02-2024-0028
Ajayi, L., 2008. Meaning-making, multimodal representation, and transformative pedagogy: An exploration of meaning construction instructional practices in an ESL high school classroom. Journal of Language, Identity & Education. 7(3–4), 206–229. DOI: https://doi.org/10.1080/15348450802237822
Ajayi, L., 2009. English as a second language learners’ exploration of multimodal texts in a junior high school. Journal of Adolescent & Adult Literacy. 52(7), 585–595. DOI: https://doi.org/10.1598/JAAL.52.7.4
Azıonya, C.M., Nhedzı, A., 2021. The digital divide and higher education challenge with emergency online learning: Analysis of tweets in the wake of the COVID-19 lockdown. Turkish Online Journal of Distance Education. 22(4), 164–182. DOI: https://doi.org/10.17718/tojde.1002822
Badem-Korkmaz, F., Balaman, U., 2022. Eliciting student participation in video-mediated EFL classroom interactions: Focus on teacher response-pursuit practices. Computer Assisted Language Learning. DOI: https://doi.org/10.1080/09588221.2022.2127772
Baggett, P., 1984. Role of temporal overlap of visual and auditory material in forming dual media associations. Journal of Educational Psychology. 76(3), 408–417. DOI: https://doi.org/10.1037/0022-0663.76.3.408
Bailey, D., Rakushin Lee, A., 2020. Learning from experience in the midst of COVID-19: Benefits, challenges, and strategies in online teaching. CALL-EJ. 21(2), 178–198.
Balaman, S., 2018. Digital storytelling: A multimodal narrative writing genre. Journal of Language and Linguistic Studies. 14(3), 202–212.
Benson, V., 2008. Is the digital generation ready for Web 2.0-based learning? The open knowlege society. A computer science and information systems manifesto. Springer: Berlin Heidelberg. DOI: https://doi.org/10.1007/978-3-540-87783-7_56
Bonwell, C.C., Eison, J.A., 1991. Active learning: Creating excitement in the classroom. 1991 ASHE-ERIC Higher Education Reports. Office of Educational Research and Improvement: Washington, DC.
Chen, C., 2006. CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature. Journal of the American Society for Information Science and Technology. 57(3), 359–377. DOI: https://doi.org/10.1002/asi.20317
Çimenli, B., Sert, O., Jenks, C., 2022. Topic maintenance in video-mediated virtual exchanges: Rolling the ball back in L2 interactions. System. 108, 102834. DOI: https://doi.org/10.1016/j.system.2022.102834
Cope, B., 2006. Multiliteracies: Literacy learning and the design of social futures. Routledge: London.
Cope, B., Kalantzis, M., 2009. A grammar of multimodality. International Journal of Learning. 16(2), 361–426. DOI: https://doi.org/10.18848/1447-9494/cgp/v16i02/46137
Danielsson, K., Selander, S., 2016. Reading multimodal texts for learning—A model for cultivating multimodal literacy. Designs for Learning. 8(1), 25–36. DOI: https://doi.org/10.16993/dfl.72
Develotte, C., Guichon, N., Vincent, C., 2010. The use of the webcam for teaching a foreign language in a desktop videoconferencing environment. ReCALL. 22(3), 293–312. DOI: https://doi.org/10.1017/S0958344010000170
Erikson, E.H., 1968. Identity: Youth and crisis. W. W. Norton & Company: New York.
Escamilla-Fajardo, P., Alguacil, M., López-Carril, S., 2021. Incorporating TikTok in higher education: Pedagogical perspectives from a corporal expression sport sciences course. Journal of Hospitality, Leisure, Sport & Tourism Education. 28, 100302. DOI: https://doi.org/10.1016/j.jhlste.2021.100302
Ganapathy, M., Seetharam, S.A., 2016. The effects of using multimodal approaches in meaning-making of 21st century literacy texts among ESL students in a private school in Malaysia. Advances in Language and Literary Studies. 7(2), 143–155.
García, O., Flores, N., Spotti, M., 2017. The Oxford handbook of language and society. Oxford University Press: Oxford.
Gomes Junior, R.C., 2020. Instanarratives: Stories of foreign language learning on Instagram. System. 94, 102330. DOI: https://doi.org/10.1016/j.system.2020.102330
Gruszczynska, A., Merchant, G., Pountney, R., 2013. “Digital Futures in Teacher Education”: Exploring open approaches towards digital literacy. Electronic Journal of E-Learning. 11(3), 193–206.
Hampel, R., Stickler, U., 2005. New skills for new classrooms: Training tutors to teach languages online. Computer Assisted Language Learning. 18(4), 311–326. DOI: https://doi.org/10.1080/09588220500335455
Hampel, R., Stickler, U., 2012. The use of videoconferencing to support multimodal interaction in an online language classroom. ReCALL. 24(2), 116–137. DOI: https://doi.org/10.1017/S095834401200002X
Hegarty, M., Kriz, S., Cate, C., 2003. The roles of mental animations and external animations in understanding mechanical systems. Cognition and Instruction. 21(4), 209–249. DOI: https://doi.org/10.1207/s1532690xci2104_1
Hur, J.W., Suh, S., 2012. Making learning active with interactive whiteboards, podcasts, and digital storytelling in ELL classrooms. Computers in the Schools. 29(4), 320–338. DOI: https://doi.org/10.1080/07380569.2012.734275
Illeris, K., 2008. How we learn: Learning and non-learning in school and beyond (2nd. ed.). Routledge: London.
Jewitt, C., 2005. Multimodality, “Reading”, and “Writing” for the 21st Century. Discourse: Studies in the Cultural Politics of Education. 26(3), 315–331. DOI: https://doi.org/10.1080/01596300500200011
Jewitt, C., Kress, G.R., 2003. Multimodal literacy. Peter Lang Group: Lausanne.
Jiang, L., Gao, J., 2020. Fostering EFL learners’ digital empathy through multimodal composing. RELC Journal. 51(1), 70–85. DOI: https://doi.org/10.1177/0033688219898565
Jiang, L., Luk, J., 2016. Multimodal composing as a learning activity in English classrooms: Inquiring into the sources of its motivational capacity. System. 59, 1–11. DOI: https://doi.org/10.1016/j.system.2016.04.001
Kanat Mutluoğlu, A., Balaman, U., 2023. The use of video enhanced observation in video-mediated post-observation conversations on pre-service EFL teachers’ online practicum teaching. System, 118, 103151. DOI: https://doi.org/10.1016/j.system.2023.103151
Khlaif, Z.N., Salha, S., 2021. Using TikTok in education: A form of micro-learning or nano-learning? Interdisciplinary Journal of Virtual Learning in Medical Sciences. 12(3), 213–218. DOI: https://doi.org/10.30476/ijvlms.2021.90211.1087
Knight, J., Dooly, M., Barberà, E., 2018. Multimodal meaning making: Navigational acts in online speaking tasks. System. 78, 65–78. DOI: https://doi.org/10.1016/j.system.2018.07.007
Kress, G.R., Van Leeuwen, T., 2001. Multimodal discourse: The modes and media of contemporary communication. Oxford University Press: Oxford.
Kuning, D.S., 2020. Applications of social media to learn speaking: Applications of social media to learn speaking. Edukasi Lingua Sastra. 18(1), 77–85. DOI: https://doi.org/10.47637/elsa.v18i1.227
Lee, H., Mayer, R.E., 2015. Visual aids to learning in a second language: Adding redundant video to an audio lecture. Applied Cognitive Psychology. 29(3), 445–454. DOI: https://doi.org/10.1002/acp.3123
Lee, H.C., 2014. Using an arts-integrated multimodal approach to promote english learning: A case study of two Taiwanese junior college students. English Teaching: Practice and Critique. 13(2), 55–75.
Li, M., 2020. Multimodal pedagogy in TESOL teacher education: Students’ perspectives. System. 94, 102337. DOI: https://doi.org/10.1016/j.system.2020.102337
Lim, F.V., Toh, W., Nguyen, T.T.H., 2022. Multimodality in the English language classroom: A systematic review of literature. Linguistics and Education. 69, 101048. DOI: https://doi.org/10.1016/j.linged.2022.101048
Lim, F.V., Towndrow, P.A., Min Tan, J., 2023. Unpacking the teachers’ multimodal pedagogies in the primary English language classroom in Singapore. RELC Journal. 54(3), 729–743. DOI: https://doi.org/10.1177/00336882211011783
Lim, J.M., 2020. Language in multimodal writing processes and performance: Developing multimodal writing tasks for L2 learners. Michigan State University: Michigan.
Mayer, R.E., 1999. The promise of educational psychology: Learning in the content areas. Merrill: New York.
Mayer, R.E., 2009. Multimedia learning (2nd ed). Cambridge University Press: Cambridge.
Mayer, R.E., Moreno, R., 2002. Animation as an aid to multimedia learning. Educational Psychology Review. 14(1), 87–99. DOI: https://doi.org/10.1023/A:1013184611077
Mayer, R.E., Pilegard, C., 2014. Principles for managing essential processing in multimedia learning: Segmenting, pre-training, and modality principles. The Cambridge Handbook of Multimedia Learning (2nd ed.). Cambridge University Press: Cambridge. pp. 316–344. DOI: https://doi.org/10.1017/CBO9781139547369.016
Mondahl, M., Rasmussen, J., Razmerita, L., 2009. Web 2.0 applications, collaboration and cognitive processes in case-based foreign language learning. Visioning and engineering the knowledge society. A web science perspective. Springer: Berlin. pp. 98–107. DOI: https://doi.org/10.1007/978-3-642-04754-1_11
Mondahl, M., Razmerita, L., 2014. Social media, collaboration and social learning a case-study of foreign language learning. Electronic Journal of E-learning. 12(4), 339–352.
Naghdipour, B., Eldridge, N.H., 2016. Incorporating social networking sites into traditional pedagogy: A case of Facebook. TechTrends. 60(6), 591–597. DOI: https://doi.org/10.1007/s11528-016-0118-4
Olagbaju, O.O., Popoola, A.G., 2020. Effects of audio-visual social media resources-supported instruction on learning outcomes in reading. International Journal of Technology in Education. 3(2), 92–104.
Rapanta, C., Botturi, L., Goodyear, P., et al., 2020. Online university teaching during and after the Covid-19 crisis: Refocusing teacher presence and learning activity. Postdigital Science and Education. 2(3), 923–945. DOI: https://doi.org/10.1007/s42438-020-00155-y
Razmerita, L., Kirchner, K., Sudzina, F., 2009. Personal knowledge management: The role of Web 2.0 tools for managing knowledge at individual and organisational levels. Online Information Review. 33(6), 1021–1039. DOI: https://doi.org/10.1108/14684520911010981
Reinhardt, J., 2019. Social media in second and foreign language teaching and learning: Blogs, wikis, and social networking. Language Teaching. 52(1), 1–39. DOI: https://doi.org/10.1017/S0261444818000356
Ryan, J., Scott, A., Walsh, M., 2010. Pedagogy in the multimodal classroom: An analysis of the challenges and opportunities for teachers. Teachers and Teaching. 16(4), 477–489. DOI: https://doi.org/10.1080/13540601003754871
Satar, H.M., 2013. Multimodal language learner interactions via desktop videoconferencing within a framework of social presence: Gaze. ReCALL. 25(1), 122–142. DOI: https://doi.org/10.1017/S0958344012000286
Schär, S.G., Kaiser, J., Krueger, H. (editors), 1999. Multimedia: The effect of picture, voice & text for the learning of concepts and principles. Proceedings of HCI International (the 8th International Conference on Human-Computer Interaction) on Human-Computer Interaction: Ergonomics and User Interfaces; 1999 Aug 22–26; Munich, Germany.
Skulmowski, A., Xu, K.M., 2022. Understanding cognitive load in digital and online learning: A new perspective on extraneous cognitive load. Educational Psychology Review. 34(1), 171–196. DOI: https://doi.org/10.1007/s10648-021-09624-7
Stein, P., 2007. Multimodal pedagogies in diverse classrooms: Representation, rights and resources. Routledge: London. DOI: https://doi.org/10.4324/9780203935804
The New London Group, 1996. A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review. 66(1), 60–93. DOI: https://doi.org/10.17763/haer.66.1.17370n67v22j160u
Toohey, K., Dagenais, D., Fodor, A., et al., 2015. “That Sounds So Cooool”: Entanglements of children, digital tools, and literacy practices. TESOL Quarterly. 49(3), 461–485. DOI: https://doi.org/10.1002/tesq.236
Unsworth, L., 2010. Resourcing multimodal literacy pedagogy: Toward a description of the meaning-making resources of language–image interaction. In: Locke T. (ed.). Beyond the Grammar Wars. Routledge: London.
Van Eck, N.J., Waltman, L., 2010. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics. 84(2), 523–538. DOI: https://doi.org/10.1007/s11192-009-0146-3
Van Eck, N.J., Waltman, L., 2014. CitNetExplorer: A new software tool for analyzing and visualizing citation networks. Journal of Informetrics. 8(4), 802–823. DOI: https://doi.org/10.1016/j.joi.2014.07.006
Vygotskij, L.S., Cole, M., 1981. Mind in society: The development of higher psychological processes. Harvard University Press: Harvard.
Wang, Y., 2004. Supporting synchronous distance language learning with desktop videoconferencing. Language Leraning & Technology.
Warren-Riley, S., Hurley, E.V., 2017. Multimodal pedagogical approaches to public writing: Digital media advocacy and mundane texts. Composition Forum. 36.
Wigham, C.R., Satar, M., 2021. Multimodal (inter)action analysis of task instructions in language teaching via videoconferencing: A case study. ReCALL. 33(3), 195–213. DOI: https://doi.org/10.1017/S0958344021000070
Yang, S., Xu, W., Liu, R., et al., 2024. Influencing and moderating variables in informal digital learning of English through a structural equation model. Computer Assisted Language Learning. DOI: https://doi.org/10.1080/09588221.2023.2280645
Yu, Z., 2022a. A meta-analysis and bibliographic review of the effect of nine factors on online learning outcomes across the world. Education and Information Technologies. 27(2), 2457–2482. DOI: https://doi.org/10.1007/s10639-021-10720-y
Yu, Z., 2022b. The effect of teacher presence in videos on intrinsic cognitive loads and academic achievements. Innovations in Education and Teaching International. 59(5), 574–585. DOI: https://doi.org/10.1080/14703297.2021.1889394
Yu, Z., 2024. A systematic review of motivations, attitudes, learning outcomes, and parental involvement in social network sites in education across 15 years. Behaviour & Information Technology. DOI: https://doi.org/10.1080/0144929X.2023.2260893
Yu, Z., Yu, L., 2021. A meta-analysis of facebook-assisted learning outcomes and their gender differences. International Journal of Mobile and Blended Learning (IJMBL). 13(4), 1–20. DOI: https://doi.org/10.4018/IJMBL.2021100104
Copyright © 2024 Yue Li
This is an open access article under the Creative Commons Attribution 4.0 International License.
