Decrypting Complexity: A Tri-Metric Evaluation of Readability and Fidelity in AI-Simplified Scientific Texts for ESL University Learners

Authors

  • Musharraf Aziz

    School of Language, Civilisation and Philosophy, University Utara Malaysia (UUM), Sintok 06010, Malaysia
  • Omar Al-Jamili

    School of Computing, University Utara Malaysia (UUM), Sintok 06010, Malaysia
  • Nur Rasyidah Mohd Nordin

    School of Language, Civilisation and Philosophy, University Utara Malaysia (UUM), Sintok 06010, Malaysia
  • Shamim Akhter

    Faculty of Education & Liberal Arts (FELA), INTI International University, Nilai 71800, Malaysia

DOI:

https://doi.org/10.30564/fls.v7i11.11269
Received: 26 July 2025 | Revised: 5 August 2025 | Accepted: 14 August 2025 | Published Online: 17 October 2025

Abstract

Undergraduate university students in ESL contexts often need enhanced readability of complex scientific articles in research journals. This study aimed to assess the efficacy and "toolability" of AI-based Chat-GPT in readability amplification of research abstracts in language and linguistics journals, indexed in Scopus and Web of Science. Robust latent semantic analysis (LSA), with vectorial space document-embedding, was performed to evaluate co-occurrence and notional preservation. One hundred abstracts (n = 100), extracted from four journals, were prompted into an open Chat-GPT 4.o session for simplification at undergraduate level ESL users. Three metrics, Flesch-Kincaid Grade Level, Flesch Reading Ease and McAlpine EFLAW were used for readability measurement at pre-transformation and post-transformation stages. The content fidelity in the input and output models were determined by latent semantic analysis recorded from 0 to 1 of the fidelity range. To rule out bias, objective evaluation by field experts was performed on a randomly extracted subgroup (n = 50). Further, t-tests and correlation analysis were conducted for comparing estimations and accuracy evaluation. The findings showed adequate semantic similarity and fidelity, almost overruling post-simplification semantic disruption. The readability increased, with a low Flesch-Kincaid Grade, high Flesch- Kincaid Ease and representative EFLAW score. However, weak correlation of LSA and field experts' estimations warranted caution and human-AI contra-estimations. The study offers micro-, meso- and macro-implications for incorporating AI in scientific reading comprehension, given caution is practiced with unsupervised dependence. Future research may involve other metrics like BERTScore, robust mixed research designs, comparative cognitive protocols evaluation of texts and other AI models.

Keywords:

AI-Assisted Text Simplification; Scientific Text Readability; Latent Semantic Analysis; Reading Comprehension; Esl University Learners

References

[1] Cáceres-Serrano, P., Alvarado-Izquierdo, J.M., 2017. The effect of contextual and socioeconomic factors on reading comprehension levels. Modern Journal of Language Teaching Methods. 7(8), 76–85.

[2] Corso, H.V., Cromley, J.G., Sperb, T., et al., 2016. Modeling the relationship among reading comprehension, intelligence, socioeconomic status, and neuropsychological functions: The mediating role of executive functions. Psychology & Neuroscience. 9(1), 32. DOI: https://doi.org/10.1037/pne0000036

[3] Velilla Sánchez, M.Á., 2025. Recontextualizing knowledge in academic video publications: A discourse analysis of multimodal science dissemination. Pragmatics and Society. 16(5), 676–700 DOI: https://doi.org/10.1075/ps.23124.vel

[4] Wang, S., Liu, X., Zhou, J., 2022. Readability is decreasing in language and linguistics. Scientometrics. 127(8), 4697–4729. DOI: https://doi.org/10.1007/s11192-022-04427-1

[5] Plavén-Sigray, P., Matheson, G.J., Schiffler, B.C., et al., 2017. The readability of scientific texts is decreasing over time. Elife. 6, e27725. DOI: https://doi.org/10.7554/elife.27725

[6] Picton, B., Andalib, S., Spina, A., et al., 2025. Assessing AI simplification of medical texts: readability and content fidelity. International Journal of Medical Informatics. 195, 105743. DOI: https://doi.org/10.1016/j.ijmedinf.2024.105743

[7] Araújo, S., Aguiar, M., 2023. Simplifying specialized texts with AI: a ChatGPT-based learning scenario. In Proceedings of the International Conference in Information Technology and Education, Singapore, June 2023; pp. 599–609. DOI: https://doi.org/10.1007/978-981-99-5414-8_55

[8] Vygotsky, L.S., 1978. Mind in society: The development of higher psychological processes. Harvard University Press: Cambridge, MA, USA.

[9] Raisch, S., Krakowski, S., 2021. Artificial intelligence and management: The automation–augmentation paradox. Academy of Management Review. 46(1), 192–210. DOI: https://doi.org/10.5465/amr.2018.0072

[10] Siontis, K.C., Attia, Z.I., Asirvatham, S.J., et al., 2024. ChatGPT hallucinating: can it get any more humanlike? European Heart Journal. 45(5), 321–323. DOI: https://doi.org/10.1093/eurheartj/ehad766

[11] Li, S., 2024. A Cross Language Information Retrieval Model Based on Latent Semantic Analysis. In: Intelligent Computing Technology and Automation. IOS Press: Amsterdam, Netherlands. pp. 1082–1089. DOI: https://doi.org/10.3233/atde231290

[12] Egger, R., Gokce, E., 2022. Natural language processing (NLP): An introduction: making sense of textual data. In: Applied Data Science in Tourism: Interdisciplinary Approaches, Methodologies, and Applications. Springer International Publishing: Cham, Switzerland. pp. 307–334. DOI: https://doi.org/10.1007/978-3-030-88389-8_15

[13] Jeon, C.H., Shin, J.Y., Ryu, S., 2025. Analyzing Student Communication Patterns in Science Classes Using Machine Learning and Natural Language Processing: A Case Study on High School Science Education. Journal of Science Education and Technology. 1–21. DOI: https://doi.org/10.1007/s10956-025-10226-z

[14] Schicchi, D., Taibi, D., 2023. AI-driven inclusion: Exploring automatic text simplification and complexity evaluation for enhanced educational accessibility. In Proceedings of the International Conference on Higher Education Learning Methodologies and Technologies Online, Cham, 2023; pp. 359–371. DOI: https://doi.org/10.1007/978-3-031-67351-1_24

[15] Anjum, A., Lieberum, N., 2023. Automatic Simplification of Scientific Texts using Pre-trained Language Models: A Comparative Study at CLEF Symposium 2023. In Proceedings of the CLEF 2023: Conference and Labs of the Evaluation Forum, Thessaloniki, Greece, 18–21 September 2023; pp. 2899–2907. Available from: https://ceur-ws.org/Vol-3497/paper-242.pdf

[16] Shardlow, M., Sellar, S., Rousell, D., 2022. Collaborative augmentation and simplification of text (CoAST): Pedagogical applications of natural language processing in digital learning environments. Learning Environments Research. 25(2), 399–421. DOI: https://doi.org/10.1007/s10984-021-09368-9

[17] Uçar, S.Ş., Aldabe, I., Aranberri, N., et al., 2024. Exploring automatic readability assessment for science documents within a multilingual educational context. International Journal of Artificial Intelligence in Education. 34(4), 1417–1459. DOI: https://doi.org/10.1007/s40593-024-00393-2

[18] McAlpine, R., 2012. From Plain English to Global English. CC Press: Wellington, New Zealand.

[19] Pan, M., Guo, K., Lai, C., 2024. Using Artificial Intelligence Chatbots to Support English-as-a-Foreign Language Students’ Self-Regulated Reading. RELC Journal. DOI: https://doi.org/10.1177/00336882241264030

[20] Cohen, J., 1988. Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Erlbaum: Hillsdale, NJ, USA.

[21] Qiang, J., Huang, M., Zhu, Y., et al., 2025. Redefining Simplicity: Benchmarking Large Language Models from Lexical to Document Simplification. arXiv preprint. arXiv:2502.08281.

[22] Tessensohn, T.C., Yunus, M.M., Ismail, H.H., 2025. Using AI-Powered Tools in Enhancing Reading Skills in the ESL Classroom: A Systematic Review (2020–2024). International Journal of Academic Research in Progressive Education and Development. 14(2), 57–70. DOI: https://doi.org/10.6007/IJARPED/v14-i2/24959

[23] Sweller, J., 1988. Cognitive load during problem solving: Effects on learning. Cognitive S. 12, 25.

[24] Crossley, S., Choi, J.S., Scherber, Y., et al., 2023. Using Large Language Models to Develop Readability Formulas for Educational Settings. In: Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky: 24th International Conference, AIED 2023, Tokyo, Japan, 3–7 July 2023. DOI: https://doi.org/10.1007/978-3-031-36336-8_66

[25] Han, Y., Ceross, A., Bergmann, J.H., 2024. The use of readability metrics in legal text: A systematic literature review. arXiv preprint. arXiv:2411.09497.

[26] Alsulami, M.M., 2025. Evaluating ChatGPT’s semantic alignment with community answers: A topic-aware analysis using BERTScore and BERTopic. Preprints. Available from: https://www.preprints.org/manuscript/202504.2000/v1 (cited 20 July 2025).

[27] Liu, Y., Han, T., Ma, S., et al., 2023. Summary of ChatGPT-related research and perspective towards the future of large language models. Meta-Radiology. 1(1–2), 100017. DOI: https://doi.org/10.1016/j.metrad.2023.100017

[28] Nahatame, S., Yamaguchi, K., 2025. Revisiting Text Readability and Processing Effort in Second Language Reading: Bayesian Analysis of Eye-Tracking Data. OSF Preprints. DOI: https://doi.org/10.31219/osf.io/5wksq_v3

[29] Aziz, M., Rawian, R., 2022. Modeling higher order thinking skills and metacognitive awareness in English reading comprehension among university learners. In: Frontiers in Education. Frontiers Media SA: Lausanne, Switzerland. DOI: https://doi.org/10.3389/feduc.2022.991015

[30] Al-Jamili, O., Aziz, M., Mohammed, F., et al., 2024. Evaluating the efficacy of computer games-based learning intervention in enhancing English speaking proficiency. Heliyon. 10(16). DOI: https://doi.org/10.1016/j.heliyon.2024.e36440

Downloads

How to Cite

Aziz, M., Al-Jamili, O., Mohd Nordin, N. R., & Akhter, S. (2025). Decrypting Complexity: A Tri-Metric Evaluation of Readability and Fidelity in AI-Simplified Scientific Texts for ESL University Learners. Forum for Linguistic Studies, 7(11), 86–103. https://doi.org/10.30564/fls.v7i11.11269

Issue

Vol. 7 , Iss. 11 (November 2025): In progress

Article Type

Article

License

Copyright © 2025 Musharraf Aziz, Omar Al-Jamili, Nur Rasyidah Mohd Nordin, Shamim Akhter

Creative Commons License

This is an open access article under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License.