The A Systematic Review of Automated Writing Evaluation (AWE) Systems on University Students' English Writing Performance

Authors

  • Jiapeng Du

    School of Languages, Civilisation and Philosophy, Universiti Utara Malaysia, Sintok 06010, Malaysia

    School of Foreign Languages, Taishan University, Tai'an 271000, China
  • Nur Rasyidah binti Mohd Nordin

    School of Languages, Civilisation and Philosophy, Universiti Utara Malaysia, Sintok 06010, Malaysia

DOI:

https://doi.org/10.30564/fls.v7i11.11764
Received: 22 August 2025 | Revised: 10 October 2025 | Accepted: 11 October 2025 | Published Online: 21 October 2025

Abstract

Automated Writing Evaluation (AWE) systems, supported by advances in artificial intelligence and natural language processing, have become increasingly prominent in higher education as tools to enhance English writing instruction. This paper presents a systematic literature review of empirical studies published between 2020 and 2024 that examine the impact of AWE on university students' English writing. Following the PRISMA framework, 20 peer-reviewed studies were selected and analyzed across multiple dimensions, including writing performance, feedback types, and students' perceptions. Findings indicate that AWE systems generally improve surface-level writing features, particularly accuracy and fluency, while their effect on higher-order skills, such as content development, organization, and critical thinking remains limited. Comparative studies suggest that AWE feedback is efficient and consistent, yet often less effective than teacher feedback in fostering deep learning. Students' perceptions of AWE are mixed: many value the immediacy and convenience of feedback, while concerns about overreliance, limited adaptability, and occasional inaccuracies persist. Moreover, the lack of long-term, cross-cultural research highlights methodological and contextual gaps in the current literature. This review not only synthesizes the main contributions of existing studies but also identifies key challenges, emphasizing the need for more integrated, longitudinal, and learner-centered approaches. Several directions for future research are proposed, aiming to optimize the pedagogical potential of AWE systems and support their effective integration into university-level English writing instruction.

Keywords:

Automated Writing Evaluation (AWE); English Writing; Writing Performance; Feedback Types Student Perceptions; Systematic Review

References

[1] Richards, J.C., 2010. Competence and performance in language teaching. RELC Journal. 41(2), 101–122. DOI: https://doi.org/10.1177/0033688210372953

[2] Zhang, X., 2018. Developing college EFL writers' critical thinking skills through online resources: A case study. SAGE Open. 8(4), Article 2158244018820386. DOI: https://doi.org/10.1177/2158244018820386

[3] Schriver, K., 2012. What we know about expertise in professional communication. In: Berninger, V.W. (Ed.). Past, Present, and Future Contributions of Cognitive Writing Research to Cognitive Psychology, 1st ed. Psychology Press: New York, NY, USA. pp. 275–312.

[4] Salomone, R., Salomone, R.C., 2022. The Rise of English: Global Politics and the Power of Language. Oxford University Press: New York, NY, USA. pp. 45–62.

[5] Whalley, B., France, D., Park, J., et al., 2021. Towards flexible personalized learning and the future educational system in the fourth industrial revolution in the wake of COVID-19. Higher Education Pedagogies. 6(1), 79–99. DOI: https://doi.org/10.1080/23752696.2021.1883458

[6] Burstein, J., Chodorow, M., 1999. Automated essay scoring for nonnative English speakers. In Proceedings of a Symposium on Computer Mediated Language Assessment and Evaluation in Natural Language Processing—ASSESSEVALNLP'99, College Park, MD, USA, 22 June 1999; pp. 68–75.

[7] Koltovskaia, S, 2022. Automated writing evaluation for formative second language assessment: Exploring performance, teacher use, and student engagement [Doctoral dissertation]. Oklahoma State University: Stillwater, OK, US.

[8] Link, S., Mehrzad, M., Rahimi, M., 2022. Impact of automated writing evaluation on teacher feedback, student revision, and writing improvement. Computer Assisted Language Learning. 35(4), 605–634. DOI: https://doi.org/10.1080/09588221.2020.1743323

[9] Zhai, N., Ma, X., 2023. The effectiveness of automated writing evaluation on writing quality: A meta-analysis. Journal of Educational Computing Research. 61(4), 875–900. DOI: https://doi.org/10.1177/07356331221127300

[10] Stevenson, M., Phakiti, A., 2019. Automated feedback and second language writing. In: Hyland, K., Hyland, F. (Eds.). Feedback in Second Language Writing: Contexts and issues, 2nd ed. Cambridge University Press: Cambridge, UK. pp. 125–142.

[11] Li, A.W., Hebert, M., 2024. Unpacking an online peer-mediated and self-reflective revision process in second-language persuasive writing. Reading and Writing. 37(6), 1545–1573. DOI: https://doi.org/10.1007/s11145-023-10466-8

[12] Li, J., Link, S., Hegelheimer, V., 2015. Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction. Journal of Second Language Writing. 27, 1–18. DOI: https://doi.org/10.1016/j.jslw.2014.10.004

[13] Saricaoglu, A., Bilki, Z., 2021. Voluntary use of automated writing evaluation by content course students. ReCALL. 33(3), 265–277. DOI: https://doi.org/10.1017/S0958344021000021

[14] Gochitashvili, K., 2023. Creating a safe and stress-free environment when using extracurricular activities in second language teaching. International Journal of Multilingual Education. (23), 1–15.

[15] Grimes, D., Warschauer, M., 2021. Utility in a fallible tool: A multi-site case study of automated writing evaluation. The Journal of Technology, Learning and Assessment. 8(6). Available from: https://ejournals.bc.edu/index.php/jtla/article/view/1625

[16] McNamara, D.S., Kendeou, P., 2022. The early automated writing evaluation (eAWE) framework. Assessment in Education: Principles, Policy & Practice. 29(2), 150–182. DOI: https://doi.org/10.1080/0969594X.2022.2037509

[17] Chen, M., Cui, Y., 2022. The effects of AWE and peer feedback on cohesion and coherence in continuation writing. Journal of Second Language Writing. 57, 100915. DOI: https://doi.org/10.1016/j.jslw.2022.100915

[18] Cope, B., Kalantzis, M., McCarthey, S., et al., 2011. Technology-mediated writing assessments: Principles and processes. Computers and Composition. 28(2), 79–96. DOI: https://doi.org/10.1016/j.compcom.2011.04.007

[19] Zhang, Z.V., Hyland, K., 2018. Student engagement with teacher and automated feedback on L2 writing. Assessing Writing. 36, 90–102. DOI: https://doi.org/10.1016/j.asw.2018.02.004

[20] Lee, Y.J., 2020. The long-term effect of automated writing evaluation feedback on writing development. English Teaching. 75(1), 67–92. DOI: https://doi.org/10.15858/engtea.75.1.202003.67

[21] Weigle, S.C., 2013. Assessing Writing. Cambridge University Press: Cambridge, UK. pp. 58–77.

[22] Shermis, M.D., Burstein, J., 2013. Handbook of Automated Essay Evaluation, 1st ed. Routledge: New York, NY, USA. pp. 36–55.

[23] Warschauer, M., Ware, P., 2006. Automated writing evaluation: Defining the classroom research agenda. Language Teaching Research. 10(2), 157–180. DOI: https://doi.org/10.1191/1362168806lr190oa

[24] Attali, Y., Burstein, J., 2006. Automated essay scoring with e-rater® V.2. The Journal of Technology, Learning and Assessment. 4(3). Available from: https://ejournals.bc.edu/index.php/jtla/article/view/1650

[25] Messer, M., Brown, N.C., Kölling, M., et al., 2024. Automated grading and feedback tools for programming education: A systematic review. ACM Transactions on Computing Education. 24(1), 1–43. DOI: https://doi.org/10.1145/3636515

[26] Fu, Q.K., Zou, D., Xie, H., et al., 2022. A review of AWE feedback: Types, learning outcomes, and implications. Computer Assisted Language Learning. 37(1–2), 179–221. DOI: https://doi.org/10.1080/09588221.2022.2033787

[27] Xu, J., Zhang, S., 2022. Understanding AWE feedback and English writing of learners with different proficiency levels in an EFL classroom: A sociocultural perspective. The Asia-Pacific Education Researcher. 31, 357–367. DOI: https://doi.org/10.1007/s40299-021-00577-7

[28] Faigley, L., 1985. Nonacademic writing: The social perspective. In: Odell, L., Goswam, D. (Eds.). Writing in Nonacademic Settings. The Guilfordfrod Press: New York, NY, USA. pp. 231–248.

[29] Kellogg, R.T., Whiteford, A.P., Turner, C.E., et al., 2013. Working memory in written composition: An evaluation of the 1996 model. Journal of Writing Research. 5(2), 159–190. DOI: https://doi.org/10.17239/jowr-2013.05.02.1

[30] Kormos, J., 2011. Task complexity and linguistic and discourse features of narrative writing performance. Journal of Second Language Writing. 20(2), 148–161. DOI: https://doi.org/10.1016/j.jslw.2011.02.001

[31] Skehan, P., 1998. A Cognitive Approach to Language Learning. Oxford University Press: New York, NY, USA. pp. 75–88.

[32] Farsani, M.A., Stapleton, P., Jamali, H.R., 2025. Charting L2 argumentative writing: A systematic review. Journal of Second Language Writing. 68, 101208. DOI: https://doi.org/10.1016/j.jslw.2025.101208

[33] Page, M.J., McKenzie, J.E., Bossuyt, P.M., et al., 2021. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. The British Medical Journal. 372, Article n71. DOI: https://doi.org/10.1136/bmj.n71

[34] Chen, C.F., Cheng, W.Y., 2008. Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness. Language Learning & Technology. 25(2), 94–112.

[35] Hu, G., McKay, S.L., 2012. English language education in East Asia: Some recent developments. Journal of Multilingual and Multicultural Development. 33(4), 345–362. DOI: https://doi.org/10.1080/01434632.2012.661434

[36] Namaziandost, E., Sawalmeh, M.H.M., Soltanabadi, M.I., 2020. The effects of spaced versus massed distribution instruction on EFL learners’ vocabulary recall and retention. Cogent Education. 7(1), Article 1792261. DOI: https://doi.org/10.1080/2331186X.2020.1792261

[37] Koltovskaia, S., 2020. Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study. Assessing Writing. 44, 100450. DOI: https://doi.org/10.1016/j.asw.2020.100450

[38] Dizon, G., Gayed, J.M., 2024. A systematic review of Grammarly in L2 English writing contexts. Cogent Education, 11(1). DOI: https://doi.org/10.1080/2331186X.2024.2397882

[39] Nazari, N., Shabbir, M.S., Setiawan, R., 2021. Application of artificial intelligence powered digital writing assistant in higher education: Randomized controlled trial. Heliyon. 7(5), Article e07014. DOI: https://doi.org/10.1016/j.heliyon.2021.e07014

[40] Hassanzadeh, M., Fotoohnejad, S., 2021. Implementing an automated feedback program for a foreign language writing course: A learner-centric study. Journal of Computer Assisted Learning. 37(5), 1494–1507. DOI: https://doi.org/10.1111/jcal.12587

[41] Zulkornain, L.H., Mat, A.C., Rahman, N.A.A., 2023. Does automated writing evaluation (AWE) improve students' writing? Focus on technical aspects and readability. International Journal of Information and Education Technology. 13(10), 1656–1662.

[42] Algburi, E., Razali, A.B., Nimehchisalem, V., et al., 2024. Combination of AWE (Criterion) feedback with the process approach and its impact on EFL writing content/idea development and organization. International Journal of Academic Research in Progressive Education and Development. 13(1), 793–803. DOI: http://dx.doi.org/10.6007/IJARPED/v13-i1/20082

[43] Barrot, J.S., Llenares, I.I., del Rosario, L.S., 2021. Students' online learning challenges during the pandemic and how they cope with them: The case of the Philippines. Education and Information Technologies. 26, 7321–7338. DOI: https://doi.org/10.1007/s10639-021-10589-x

[44] Sun, B., Fan, T., 2022. The effects of an AWE-aided assessment approach on business English writing performance and writing anxiety: A contextual consideration. Studies in Educational Evaluation. 72, Article 101123. DOI: https://doi.org/10.1016/j.stueduc.2021.101123

[45] Li, W., Lu, Z., Liu, Q., 2020. Syntactic complexity development in college students' essay writing based on AWE. In: Frederiksen, K.-M., Larsen, S., Bradley, L., et al. (Eds.). CALL for Widening Participation: Short Papers from EUROCALL. Research-publishing.net: Paris, France. pp. 190–194. DOI: https://doi.org/10.14705/rpnet.2020.48.1187

[46] Tian, L., Zhou, Y., 2020. Learner engagement with automated feedback, peer feedback and teacher feedback in an online EFL writing context. System. 91, Article 102247. DOI: https://doi.org/10.1016/j.system.2020.102247

[47] Diyyab, E.A., Aly, E., 2021. A suggested program based on 7E instructional model and AWE systems to develop faculty of education ESP students' academic writing skills. Journal of Education — Sohag University. (88)88, 1–39.

[48] Ginting, R.S., Fithriani, R., 2022. Peer and automated writing evaluation (AWE): Indonesian EFL college students' preference for essay evaluation. LLT Journal: A Journal on Language and Language Teaching. 25(2), 461–473. DOI: https://doi.org/10.24071/llt.v25i2.4879

[49] Wan, X., 2024. Learning English writing collaboratively with AWE: Chinese EFL learners' perceptions, engagement, and scaffolding [Doctoral dissertation]. University of Macau: Macau, China.

[50] Al-Inbari, F.A.Y., Al-Wasy, B.Q.M., 2023. The impact of automated writing evaluation (AWE) on EFL learners' peer and self-editing. Education and Information Technologies. 28(6), 6645–6665. DOI: https://doi.org/10.1007/s10639-022-11458-x

[51] Sari, E., Han, T., 2024. The impact of automated writing evaluation on English as a foreign language learners' writing self-efficacy, self-regulation, anxiety, and performance. Journal of Computer Assisted Learning. 40(5), 2065–2080. DOI: https://doi.org/10.1111/jcal.13004

[52] Miranty, D., Widiati, U., 2021. Automated writing evaluation (AWE) in higher education: Indonesian EFL students' perceptions about Grammarly use across student cohorts. Pegem Journal of Education and Instruction. 11(4), 126–137. DOI: https://doi.org/10.47750/pegegog.11.04.12

[53] Xie, Y., Huang, L., Wang, Y., 2020. The impact of AWE and peer feedback on Chinese EFL learners' English writing performance. In: Lee, L.K., U, L.H., Wang, F.L., et al. (Eds.). Technology in Education: Innovations for Online Teaching and Learning. ICTE 2020: Communications in Computer and Information Science, vol 1302. Springer: Singapore. pp. 258–270. DOI: https://doi.org/10.1007/978-981-33-4594-2_22

[54] Sadler, D.R., 1989. Formative assessment and the design of instructional systems. Instructional science. 18(2), 119–144.

[55] Hyland, K, Hyland, F., 2006 Feedback on second language students' writing. Language Teaching. 39(2), 83–101. Available from: https://core.ac.uk/download/pdf/37893651.pdf

[56] Suh, E.K., Dyer, J., McGee, B., et al., 2022. To, through, and beyond higher education: A literature review of multilingual immigrant students' community college transitions. Community College Journal of Research and Practice. 46(5), 301–317. DOI: https://doi.org/10.1080/10668926.2020.1841045

[57] Sabti, A.A., Md Rashid, S., Nimehchisalem, V., et al., 2019. The Impact of Writing Anxiety, Writing Achievement Motivation, and Writing Self-Efficacy on Writing Performance: A Correlational Study of Iraqi Tertiary EFL Learners. Sage Open. 9(4). DOI: https://doi.org/10.1177/2158244019894289

Downloads

How to Cite

Du, J., & Nordin, N. R. binti M. (2025). The A Systematic Review of Automated Writing Evaluation (AWE) Systems on University Students’ English Writing Performance. Forum for Linguistic Studies, 7(11), 615 –. https://doi.org/10.30564/fls.v7i11.11764

Issue

Vol. 7 , Iss. 11 (November 2025): In progress

Article Type

Review

License

Copyright © 2025 Jiapeng Du, Nur Rasyidah binti Mohd Nordin

Creative Commons License

This is an open access article under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License.