The Development and Validation of the Online Reading Engagement Scale (ORES): A New Tool for Digital Academic Contexts

Authors

  • Changyan Dang

    School of Languages, Civilisation and Philosophy, Universiti Utara Malaysia, Kedah 06010, Malaysia

    School of Foreign Languages, Shandong Women's University, Jinan 250300, China

  • Rafizah Rawian

    School of Languages, Civilisation and Philosophy, Universiti Utara Malaysia, Kedah 06010, Malaysia

  • Mazura Jamali

    School of Languages, Civilisation and Philosophy, Universiti Utara Malaysia, Kedah 06010, Malaysia

DOI:

https://doi.org/10.30564/fls.v7i11.11213
Received: 23 July 2025 | Revised: 13 August 2025 | Accepted: 20 August 2025 | Published Online: 17 October 2025

Abstract

This study introduces and validates the Online Reading Engagement Scale (ORES), a multidimensional instrument designed to assess behavioural engagement, cognitive engagement, affective engagement, and social engagement in digital academic reading contexts. Grounded in the engagement model by Wigfield and Guthrie in 2000, this scale fills a research gap by providing a comprehensive tool specifically designed for university-level EFL students engaged in online reading. The instrument was developed through item generation and refinement informed by prior research and expert feedback. A sample of 668 Chinese undergraduates participated in the study, completing a 32-item questionnaire distributed on Wenjuanxing, an online survey platform. Exploratory factor analysis (EFA) revealed a four-factor structure, explaining 61.1% of the total variance. Confirmatory factor analysis (CFA) further supported the model with strong fit indices (e.g., RMSEA = 0.024, CFI = 0.986). Reliability and validity assessments confirmed internal consistency (Cronbach's α > 0.90), convergent validity, and discriminant validity across all dimensions. The findings establish ORES as a reliable and valid instrument for assessing online reading engagement in academic settings, providing a theoretical and empirical foundation for future research and pedagogical interventions aimed at enhancing digital literacy and student engagement in higher education.

Keywords:

Online Reading Engagement; Scale Development; Digital Academic Reading; English as a Foreign Language (EFL)

References

[1] Fredricks, J.A., Filsecker, M., 2016. Lawson MA. Student engagement, context, and adjustment: Addressing definitional, measurement, and methodological issues. Learning and Instruction. 43, 1–4.

[2] Dörnyei, Z., Kormos J., 2000. The role of individual and social variables in oral task performance. Language Teaching Research. 4(3), 275–300.

[3] Hiver, P., Al-Hoorie, A., Vitta, J., Wu, J., 2024. Engagement in language learning: A systematic review of 20 years of research methods and definitions. Language Teaching Research. 28, 201–230.

[4] Guthrie, J.T., Wigfield, A., Barbosa, P., et al., 2004. Increasing Reading Comprehension and Engagement Through Concept-Oriented Reading Instruction. Journal of Educational Psychology. 96(3), 403–423. DOI: http://dx.doi.org/10.1037/0022-0663.96.3.403

[5] Lee, Y., Jang, B.G., Smith, K.C., 2021. A Systematic Review of Reading Engagement Research: What Do We Mean, What Do We Know, and Where Do We Need to Go? Reading Psychology. 42(5), 540–576.

[6] Abdelhalim, S.M., 2017. Developing EFL students’ reading comprehension and reading engagement: effects of a proposed instructional strategy. Theory and Practice in Language Studies. 7(1), 37–48.

[7] Anderson, L.L., Meline, M., Harn, B., 2023. Student Engagement Within Adolescent Reading Comprehension Interventions: A Systematic Literature Review. Journal of Education. 203(2), 258–268.

[8] Barber, T.A., Buehl, M.M., Kidd, J.K., et al., 2015. Reading Engagement in Social Studies: Exploring the Role of a Social Studies Literacy Intervention on Reading Comprehension, Reading Self-Efficacy, and Engagement in Middle School Students with Different Language Backgrounds. Read Psychology. 36(1), 31–85.

[9] Chen, F., Li, S., Lin, L., et al., 2024. Identifying temporal changes in student engagement in social annotation during online collaborative reading. Education and Information Technologies. 29(13), 16101–16124.

[10] Gao, L., 2023. Contemporary American literature in Online Learning: fostering Reading Motivation and Student Engagement. Education and Information Technologies. 28(4), 4725–4740.

[11] Alrashidi, O., Phan, H.P., Ngu, B.H., 2016. Academic Engagement: An Overview of Its Definitions, Dimensions, and Major Conceptualisations. International Education Studies, 9(12), 41–52.

[12] Abbasi, M., Ghamoushi, M., Zenouzagh, Z.M., 2024. EFL learners’ engagement in online learning context: Development and validation of potential measurement inventory. Universal Access in the Information Society, 23(3), 1467–1481. DOI: https://doi.org/10.1007/s10209-023-00993-0

[13] Guthrie, J.T., Wigfield, A., 1997. Reading engagement: A rationale for theory and teaching. Reading Engagement: Motivating Readers through Integrated Instruction. 1997,1–12.

[14] Guthrie, J.T., Wigfield, A., 1999. How motivation fits into a science of reading. Scientific Studies of Reading. 3(3), 199–205.

[15] Wigfield, A., Guthrie, J.T., 2000. Engagement and motivation in reading. Handbook of Reading Research. 3(2000), 403–422.

[16] Unrau, N.J., Quirk, M., 2014. Reading Motivation and Reading Engagement: Clarifying Commingled Conceptions. Reading Psychology. 35(3), 260–284. DOI: https://doi.org/10.1080/02702711.2012.684426

[17] OECD., 2019. PISA 2018 Assessment and Analytical Framework. OECD: Paris, France. DOI: https://doi.org/10.1787/b25efab8-en

[18] Appleton, J.J., Christenson, S.L., Furlong, M.J., 2008. Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools. 45(5), 369–386. DOI: https://doi.org/10.1002/pits.20303

[19] Fredricks, J.A., Reschly, A.L., Christenson, S.L., 2019. Interventions for Student Engagement: Overview and State of the Field. Handbook of Student Engagement Interventions. 1–11. DOI: https://doi.org/10.1016/B978-0-12-813413-9.00001-2

[20] Wigfield, A., Guthrie, J.T., Perencevich, K.C., et al., 2008. Role of reading engagement in mediating effects of reading comprehension instruction on reading outcomes. Psychology in the Schools. 45(5), 432–445. DOI: https://doi.org/10.1002/pits.20307

[21] Fredricks, J.A., McColskey, W., 2012. The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. In: Christenson, S.L., Reschly, A.L., Wylie, C. (eds.). Handbook of research on student engagement. Springer Science + Business Media: Boston, MA, USA. pp. 763–782.

[22] Lin, J., Li, Q., Sun, H., et al., 2021. Chinese secondary school students' reading engagement profiles: Associations with reading comprehension. Reading and Writing. 34(9), 2257–2287. DOI: https://doi.org/10.1007/s11145-021-10139-4

[23] Guthrie, J.T., Wigfield, A., You, W., 2012. Instructional Contexts for Engagement and Achievement in Reading. In: Christenson, S., Reschly, A., Wylie, C. (eds.). Handbook of Research on Student Engagement. Springer: Boston, MA, USA. pp. 601–634. DOI: https://doi.org/10.1007/978-1-4614-2018-7_29

[24] Cook, C.R., Thayer, A.J., Fiat, A., et al., 2020. Interventions to enhance affective engagement. In: Reschly, A.L., Pohl, A.J., Christenson, S.L. (eds.). Student Engagement: Effective Academic, Behavioral, Cognitive, and Affective Interventions at School. Springer International Publishing, Cham, Switzerland. pp. 203–237.

[25] Fredricks, J.A., Blumenfeld, P.C., Paris, A.H., 2004. School engagement: Potential of the concept, state of the evidence. Review of Educational Research. 74(1), 59–109.

[26] Svalberg, A.M.L., 2018. Researching language engagement; current trends and future directions. Language Awareness. 27(1–2) 21–39.

[27] Alexander, P.A., 2018. Engagement and literacy: Reading between the lines. Journal of Research in Reading. 41(4), 732–739. DOI: https://doi.org/10.1111/1467-9817.12262

[28] Barber, A.T., Klauda, S.L., Wang, W., 2022. Reading anxiety, engagement, and achievement: A comparison of emergent bilinguals and English monolinguals in the elementary grades. Reading Research Quarterly. 57(1), 353–376.

[29] Soriano-Ferrer, M., Morte-Soriano, M., 2017. Teacher Perceptions of Reading Motivation in Children with Developmental Dyslexia and Average Readers. Procedia - Social and Behavioral Sciences, 237, 50–56. DOI: https://doi.org/10.1016/j.sbspro.2017.02.012

[30] Taboada, A., Townsend, D., Boynton, M. J., 2013. Mediating Effects of Reading Engagement on the Reading Comprehension of Early Adolescent English Language Learners. Reading & Writing Quarterly. 29(4), 309–332. DOI: https://doi.org/10.1080/10573569.2013.741959

[31] Cao, C., Zhang, T., Xin, T., 2024. The effect of reading engagement on scientific literacy–an analysis based on the XGBoost method. Frontiers in Psychology. 15, 1329724. DOI: https://doi.org/10.3389/fpsyg.2024.1329724

[32] Cheung, K., Mak, S., Sit, P., et al., 2016. A typology of student reading engagement: Preparing for response to intervention in the school curriculum. Studies in Educational Evaluation. 48, 32–42.

[33] Lin, L., King, R.B., Fu, L., et al., 2024. Information and communication technology engagement and digital reading: How meta‐cognitive strategies impact their relationship. British Journal of Educational Technology. 55(1), 277–296. DOI: https://doi.org/10.1111/bjet.13355

[34] Zhao, W., Song, Y., Zhao, Q., et al., 2019. The effect of teacher support on primary school students' reading engagement: The mediating role of reading interest and Chinese academic self-concept. Educational Psychology. 39(2), 236–253. DOI: https://doi.org/10.1080/01443410.2018.1497146

[35] McGeown, S., Smith, K.C., 2024. Reading Engagement Matters! A New Scale to Measure and Support Children’s Engagement with Books. The Reading Teacher. 77(4), 462–472. DOI: https://doi.org/10.1002/trtr.2267

[36] Cubillos, M., Zegers, M., Inciarte, H., 2025. Examining Adolescent Reading Engagement: Design and Validation of the Teacher‐Reported Reading Engagement Survey (TRRES). Reading Research Quarterly, 60(2), e611. DOI: http://dx.doi.org/10.1002/rrq.611

[37] Deng, R., Benckendorff, P., Gannaway, D., 2020. Learner engagement in MOOCs: Scale development and validation. British Journal of Educational Technology. 51(1), 245–262. DOI: https://doi.org/10.1111/bjet.12810

[38] Dixson, M.D., 2015. Measuring student engagement in the online course: The Online Student Engagement scale (OSE). Online Learning. 19(4), n4. DOI: https://doi.org/10.24059/olj.v19i4.561

[39] Gunuc, S., Kuzu, A., 2015. Student engagement scale: Development, reliability and validity. Assessment & Evaluation in Higher Education. 40(4), 587–610. DOI: https://doi.org/10.1080/02602938.2014.938019

[40] Hoi, V.N., Hang, H.L., 2021. The structure of student engagement in online learning: A bi‐factor exploratory structural equation modelling approach. Journal of Computer Assisted Learning, 37(4). 1141–1153.

[41] Lin, S.-H., Huang, Y.-C., 2018. Assessing College Student Engagement: Development and Validation of the Student Course Engagement Scale. Journal of Psychoeducational Assessment. 36(7), 694–708.

[42] Zhoc, K.C.H., Webster, B.J., King, R.B., et al., 2019. Higher Education Student Engagement Scale (HESES): Development and Psychometric Evidence. Research in Higher Education. 60(2), 219–244. DOI: https://doi.org/10.1007/s11162-018-9510-6

[43] Singer, L.M., Alexander, P.A., 2017. Reading on Paper and Digitally: What the Past Decades of Empirical Research Reveal. Review of Educational Research. 87(6), 1007–1041. DOI: https://doi.org/10.3102/0034654317722961

[44] Delgado, P., Vargas, C., Ackerman, R., et al., 2018. Don't throw away your printed books: A meta-analysis on the effects of reading media on reading comprehension. Educational Research Review. 25, 23–38. DOI: https://doi.org/10.1016/j.edurev.2018.09.003

[45] Jiang, J., 2024. “Emotions are what will draw people in”: A study of critical affective literacy through digital storytelling. Journal of Adolescent & Adult Literacy. 67(4), 253–263. DOI: https://doi.org/10.1002/jaal.1322

[46] Nash, B.L., 2024. Emotion, stimulation, habit, and criticality: Learning from teachers' multifaceted conceptions of digital reading. Journal of Research on Technology in Education. 56(1), 7–24.

[47] Hair, J.F., Risher, J.J., Sarstedt, M., et al., 2019. When to use and how to report the results of PLS-SEM. European Business Review. 31(1), 2–24.

[48] Hair Jr, J.F., Hult, G.T.M., Ringle, C.M., et al., 2021. Partial least squares structural equation modeling (PLS-SEM) using R: A workbook. Springer Nature: Cham, Switzerland. Available from: https://library.oapen.org/handle/20.500.12657/51463 (cited 6 February 2025).

[49] Hu, L., Bentler, P.M., 1999. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 6(1), 1–55.

[50] Fornell, C., Larcker, D.F., 1981. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research. 18(1), 39–50. DOI: https://doi.org/10.1177/002224378101800104

[51] Kaiser, H.F., 1974. An Index of Factorial Simplicity. Psychometrika. 39(1), 31–36. DOI: https://doi.org/10.1007/BF02291575

[52] Bartlett, M.S., 1954. A note on the multiplying factors for various χ 2 approximations. Journal of the Royal Statistical Society. Series B (Methodological), 296–298.

[53] Cattell, R.B., 1966. The Scree Test for The Number of Factors. Multivariate Behavioral Research. 1(2), 245–276.

[54] Kaiser, H.F., 1960. The Application of Electronic Computers to Factor Analysis. Educational and Psychological Measurement. 20(1), 141–151. DOI: https://doi.org/10.1177/001316446002000116

[55] Widaman, K. F., Helm, J. L., 2023. Exploratory factor analysis and confirmatory factor analysis. Available from: https://psycnet.apa.org/record/2023-80648-017 (cited 23 April 2025).

[56] Henseler, J., Ringle, C.M., Sarstedt, M., 2015. A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science. 43(1), 115–135. DOI: https://doi.org/10.1007/s11747-014-0403-8

[57] Hair, J., Alamer, A., 2022. Partial Least Squares Structural Equation Modeling (PLS-SEM) in second language and education research: Guidelines using an applied example. Research Methods in Applied Linguistics. 1(3), 100027. DOI: https://doi.org/10.1016/j.rmal.2022.100027

[58] Chen, M. R. A., Hwang, G. J., Lin, Y. H., Khalil, V. A., Li, H., Ogata, H., 2022. A reading engagement-promoting strategy to facilitate EFL students' mobile learning achievement, behaviour and engagement. International Journal of Mobile Learning and Organisation. 16(4), 489–506. https://doi.org/10.1504/IJMLO.2022.125968

[59] Afflerbach, P., Harrison, C., 2017. What Is Engagement, How Is It Different From Motivation, and How Can I Promote It? Journal of Adolescent & Adult Literacy. 61(2), 217–220. DOI: https://doi.org/10.1002/jaal.679

[60] McGeown, S., Smith, K.C., 2023. Measuring Reading Engagement. Available from: https://blogs.ed.ac.uk/literacylab/wp-ontent/uploads/sites/6404/2023/06/Reading-Engagement-Scale-FINAL.pdf (cited 6 August 2024)

[61] Cheung, G.W., Cooper-Thomas, H.D., Lau, R.S., et al., 2024. Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations. Asia Pacific Journal of Management. 41(2), 745–783. DOI: https://doi.org/10.1007/s10490-023-09871-y

[62] Xie, Y., Wang, J., Li, S., et al., 2023. Research on the influence path of metacognitive reading strategies on scientific literacy. Journal of Intelligence. 11(5), 78.

[63] Lee, K.M., Park, S., Jang, B.G., et al., 2019. Multidimensional Approaches to Examining Digital Literacies in the Contemporary Global Society. Media and Communication. 7(2), 36–46. DOI: https://doi.org/10.17645/mac.v7i2.1987

[64] Sun, Y., Lin, S.-Y., Chung, K. K. H., 2020. University Students’ Perceived Peer Support and Experienced Depressive Symptoms during the COVID-19 Pandemic: The Mediating Role of Emotional Well-Being. International Journal of Environmental Research and Public Health. 17(24), 9308. DOI: https://doi.org/10.3390/ijerph17249308

[65] Pan, M., Lai, C., Guo, K., 2025. Effects of GenAI-empowered interactive support on university EFL students' self-regulated strategy use and engagement in reading. The Internet and Higher Education. 65, 100991. DOI: https://doi.org/10.1016/j.iheduc.2024.100991

Downloads

How to Cite

Dang, C., Rawian, R., & Jamali, M. (2025). The Development and Validation of the Online Reading Engagement Scale (ORES): A New Tool for Digital Academic Contexts. Forum for Linguistic Studies, 7(11), 57–72. https://doi.org/10.30564/fls.v7i11.11213