A Cognitive Diagnostic Model of Reading Ability based on China’s Standards of English Language Ability

Authors

  • Zhe Zhang

    Faculty of Education, Universiti Kebangsaan Malaysia, Bangi 43600, Selangor, Malaysia
  • Mohd Isa Hamzah

    Faculty of Education, Universiti Kebangsaan Malaysia, Bangi 43600, Selangor, Malaysia
  • Khairul Azhar Jamaludin

    Faculty of Education, Universiti Kebangsaan Malaysia, Bangi 43600, Selangor, Malaysia

DOI:

https://doi.org/10.30564/fls.v6i6.7551
Received: 21 October 2024 | Revised: 7 November 2024 | Accepted: 8 November 2024 | Published Online: 11 December 2024

Abstract

Traditional English reading assessments in China typically provide a single summative score, without revealing the extent to which students have mastered different aspects of reading comprehension. Cognitive Diagnostic Models (CDMs), as a type of multidimensional latent trait model, offer diagnostic insights by presenting examinees' proficiency levels across a set of predefined skills. Cognitive diagnostic assessments based on these models can deliver multidimensional, fine-grained evaluation results, thus overcoming the limitations of traditional unidimensional score-based assessments. Based on the China's Standards of English Language Ability (CSE), this study constructs a cognitive diagnostic assessment model for English reading. Using college general English reading test papers and the G-DINA model, the English reading abilities of 200 students were diagnosed, and personalized performance reports were generated. A questionnaire survey was conducted to gather students' evaluations and suggestions regarding the feedback reports. The results indicate that the cognitive diagnostic model for English reading, constructed according to the CSE, exhibits good model fit and high diagnostic reliability. The personalized performance reports visually and comprehensively present the strengths and weaknesses of the test takers' reading abilities, providing valuable guidance for students' future remedial learning. This study utilizes a multidimensional assessment approach based on cognitive diagnostic models to evaluate the English reading abilities of Chinese university students, effectively addressing the limitations of traditional assessment methods that typically offer only a single score.

Keywords:

China's Standards of English Language Ability; Cognitive Diagnostic Assessment; G-DINA Model; Diagnostic Report

References

[1] Ravand, H., Robitzsch, A., 2018. Cognitive Diagnostic Model of Best Choice: A Study of Reading Comprehension. Educational Psychology. 38(10), 1255-1277. DOI: https://doi.org/10.1080/01443410.2018.1489524

[2] Javidanmehr, Z., Anani Sarab, M.R., 2019. Retrofitting Non-Diagnostic Reading Comprehension Assessment: Application of the G-DINA Model to a High Stakes Reading Comprehension Test. Language Assessment Quarterly. 16(3), 294-311. DOI: https://doi.org/10.1080/15434303.2019.1654479

[3] Yu, X., 2021. Text Complexity of Reading Comprehension Passages in the National Matriculation English Test in China: The Development from 1996 to 2020. International Journal of Language Testing. 11(2), 142-167.

[4] Gorham, A., Randall, J., 2022. Classical Test Theory. Routledge.pp.1-14. DOI: https://doi.org/10.4324/9781138609877-REE26-1

[5] Reeve, B., 2024. Item Response Theory [IRT]. In: Maggino, F. (eds.). Encyclopedia of Quality of Life and Well-Being Research. Springer International Publishing: Cham, Switzerland. pp. 3689-3697

[6] Leighton, J.P., Gierl, M.J., 2007. Verbal Reports as Data for Cognitive Diagnostic Assessment. In Cambridge University Press: Cambridge, England. pp. 146-172. DOI: https://doi.org/10.1017/CBO9780511611186.006

[7] Lee, Y.W., Sawaki, Y., 2009. Application of Three Cognitive Diagnosis Models to ESL Reading and Listening Assessments. Language Assessment Quarterly. 6(3), 239-263. DOI: https://doi.org/10.1080/15434300903079562

[8] Jang, E. E., Dunlop, M., Park, G., et al., 2015. How Do Young Students with Different Profiles of Reading Skill Mastery, Perceived Ability, and Goal Orientation Respond to Holistic Diagnostic Feedback? Language Testing. 32(3), 359-383. DOI: https://doi.org/10.1177/0265532215570924

[9] Kim, A.Y., 2015. Exploring Ways to Provide Diagnostic Feedback with an ESL Placement Test: Cognitive Diagnostic Assessment of L2 Reading Ability. Language Testing. 32(2), 227-258. DOI: https://doi.org/10.1177/0265532214558457

[10] Toprak, T.E., Cakir, A., 2021. Examining the L2 Reading Comprehension Ability of Adult ELLs: Developing a Diagnostic Test Within the Cognitive Diagnostic Assessment Framework. Language Testing. 38(1), 106-131. DOI: https://doi.org/10.1177/0265532220941470

[11] Fan, T., Yan, X., 2020. Diagnosing English Reading Ability in Chinese Senior High Schools. Studies in Educational Evaluation. 67, 100931.

[12] Du, W., Ma, X., 2022. A CSE-Based Cognitive Diagnostic Modelling of EFL Reading Inferential Ability and Its Diagnostic Report. Journal of China Examinations. 2022(12), 1-9. DOI: https://doi.org/10.19360/j.cnki.11-3303/g4.2022.12.001.

[13] Luo, J., 2019. Cognitive diagnostic assessment for reading comprehension based on college English placement test [PhD Dissertation]. Changsha, HN: Hunan University. pp. 35-60.

[14] Wu, Y., 2020. Towards a framework of diagnostic score reporting of college students’ English reading ability: linking cognitive diagnostic assessment with score reporting [PhD Dissertation].Shanghai, SH: Shanghai International Studies University.pp.27-55.

[15] Zhang, H., Shi, Y., 2020. The Group and Individualized Diagnosis on EFL Reading in University-Based Final Test. Technology Enhanced Foreign Language Education. 2020(5), 79-85+12. DOI: https://doi.org/10.16538/j.cnki.cn11-3303/g4.2020.05.011

[16] Du, W., Ma, X., 2018. CDA-Based Diagnostic Model Construction for EFL Reading Tests. Foreign Language Teaching and Research. 2018(1), 74-88+160-161. DOI: https://doi.org/10.16538/j.cnki.cn11-3303/g4.2018.01.007

[17] Irkinovich, N.R., Izatullaevna, I.I., 2022. Methods for Teaching Reading. Journal of Pedagogical Inventions and Practices. 15, 47-50.

[18] Dhanapala, K.V., Yamada, J., 2015. Reading Processing Skills Among EFL Learners in Different Proficiency Levels. Reading Matrix: An International Online Journal. 15(1), 25-40.

[19] Kim, H., Rah, Y., 2019. Constructional Processing in a Second Language: The Role of Constructional Knowledge in Verb‐Construction Integration. Language Learning. 69(4), 1022-1056. DOI: https://doi.org/10.1111/lang.12366

[20] Smith, R., Snow, P., Serry, T., et al., 2021. The Role of Background Knowledge in Reading Comprehension: A Critical Review. Reading Psychology. 42(3), 214-240. DOI: https://doi.org/10.1080/02702711.2021.1888348

[21] Ferstl, E.C., 2019. The Construction-Integration Model: A Framework for Studying Context Effects in Sentence Processing. Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society; August 13-16, 1994. Georgia Institute of Technology. Atlanta, GA. pp. 289-294.

[22] Kintsch, W., Dijk, T A., 1978. Towards a Model of Discourse Comprehension and Production. Psychological Review. 85(4), 363-394.

[23] Kintsch, W., 1998. Comprehension: A Paradigm for Cognition. Cambridge University Press: Cambridge, England.pp.75-92.

[24] Zhang, S., Wijekumar, K., Han, B., 2021. An Analysis of Grade 4 Reading Textbooks used in Mainland China: Do the Texts and Activities Support Higher Order Reading Comprehension Skills? Technology, Knowledge and Learning. 26, 251-291.

[25] Bachman, L.F., 1990. Fundamental Considerations in Language Testing. Oxford University Press: Oxford, England. pp. 67-80.

[26] Khan, A.B., Mansoor, H.S., 2020. Integrated Collaborative Learning Approach (ICLA): Conceptual Framework of Pedagogical Approach for the Integration of Language Skills. Competitive Social Science Research Journal. 1(1), 14-28.

[27] Liao, L., Ye, S.X., Yang, J., 2023. A mini review of communicative language testing. Frontiers in Psychology. 14, 1058411. DOI: https://doi.org/10.3389/fpsyg.2023.1058411

[28] Yan, W.J., 2019. Action Research as a Model for the Enhancement of Communicative Competence among Learners of English as a Foreign Language in Mainland China. 2019 3rd International Conference on Education Technology and Economic Management (ICETEM 2019); May 12, 2019; China, Xi’an. pp. 35-41.

[29] Liu, J.D., 2015. Exploring Standard-Based Foreign Language Assessment. Foreign Language Teaching and Research. 47(3), 417-425. DOI: https://doi.org/10.16538/j.cnki.cn11-3303/g4.2015.03.009

[30] Fan, T.T, Zeng, Y.Q., 2019. A Review of Cognitive Diagnostic Testing and Its Application in Reading Comprehension Ability. Chinese Foreign Language. 2016(2). 82-89.

[31] Liu, J.D., Han, B.C., 2018. Theoretical Considerations for Developing Use-Oriented Chin’s Standards of English. Modern Foreign Languages. 41(1), 78-90+146.

[32] Zeng, Y., Fan, T., 2017. Developing Reading Proficiency Scales for EFL Learners in China. Language Testing in Asia. 7(1), 1-15.

[33] Alderson. J. C., 2007. The challenge of (diagnostic) testing: Do we know what we are measuring? In J. Fox. et.al. (eds.). Language Testing Reconsidered. University of Ottawa Press: Ottawa, Canada. pp. 21-39.

[34] Buck, G., Tatsuoka, K., 1998. Application of the Rule-Space Procedure to Language Testing: Examining Attributes of a Free Response Listening Test. Language Testing. 15(2), 119–157. DOI: https://doi.org/10.1177/026553229801500201

[35] Tatsuoka, K.K., 1990. Diagnostic Monitoring of Skill and Knowledge Acquisition. 1st ed. Routledge: London, England. pp. 12-20.

[36] Tatsuoka, K.K., 2009. Cognitive Assessment: An Introduction to the Rule Space Method. Routledge: New York, USA. pp. 15-27.

[37] Wang, D., Cai, Y., Tu, D., 2020. Q-Matrix Estimation Methods for Cognitive Diagnosis Models: Based on Partial Known Q-Matrix. Multivariate Behavioral Research. April, 1-13. DOI: 10.1080/00273171.2020.1746901.

[38] Rupp, A.A., Templin, J., Henson, R., 2010. Diagnostic Measurement: Theory, Methods, and Applications. The Guilford Press: New York, USA. pp. 50-89.

[39] Chen, Q.M., Zhang, M.Q., 2010. Development of Cognitive Diagnosis Models and Their Application Methods. Advances in Psychological Science. 18(3), 522. DOI: https://journal.psych.ac.cn/adps/EN/Y2010/V18/I03/522

[40] Sun, H., Kang, Y., 2022. A Review of Cognitive Diagnosis Assessment in Language from 2000 to 2021: A Visualized Analysis with CiteSpace II. Journal of Humanities and Social Sciences Studies. 4(3), 182-191. DOI: https://doi.org/10.32996/jhsss.2022.4.3.17

[41] De la Torre, J., 2011. The generalized DINA model framework. Psychometrical. 76(2), 179-199.

[42] Hemati, S. J., Baghaei, P., 2020. A Cognitive Diagnostic Modeling Analysis of the English Reading Comprehension Section of the Iranian National University Entrance Examination. International Journal of Language Testing, 10(1), 11-32.

[43] Li, H., Suen, H. K., 2013. Constructing and validating a Q-matrix for cognitive diagnostic analyses of a reading test. Educational Assessment. 18(1), 1-25. DOI: https://doi.org/10.1080/10627197.2013.761522

[44] Roberts, M. R., Gierl, M. J., 2010. Developing score reports for cognitive diagnostic assessments. Educational Measurement: Issues and Practice, 29(3), 25-38. DOI: https://doi.org/10.1111/j.1745-3992.2010.00181.x

[45] Urquhart, A. H.; Weir, C. J., 2014. Reading in a second language Process, product and practice,1st ed. Routledge: London, England.pp.120-155.

[46] Alderson, J. C., 2015. The Diagnosis of Reading in a Second or Foreign Language. Routledge: New York, USA. pp. 85-90.

[47] Landis, J. R., Koch, G. G.,1977. The measurement of observer agreement for categorical data. Biometrics. Vol. 33, No. 1, 159-174.DOI: https://doi.org/10.2307/2529310

[48] Jang, E. E., 2009. Cognitive diagnostic assessment of L2 reading comprehension ability: Validity arguments for Fusion Model application to LanguEdge assessment. Language Testing. 26(1), 031-073. DOI: https://doi.org/10.1177/0265532208097336

[49] Zhou, S., 2019. A review of the current situation of English education in China from the perspective of social environment. Overseas English.14 (1), 46-52.

[50] Mirzaei, A., Vincheh, M. H., Hashemian, M., 2020. Retrofitting the IELTS reading section with a general cognitive diagnostic model in an Iranian EAP context. Studies in Educational Evaluation. 64, 100817.DOI: https://doi.org/10.1016/j.stueduc.2019.100817

Downloads

How to Cite

zhang, zhe, Hamzah, M. I., & Jamaludin , K. A. (2024). A Cognitive Diagnostic Model of Reading Ability based on China’s Standards of English Language Ability. Forum for Linguistic Studies, 6(6), 800–815. https://doi.org/10.30564/fls.v6i6.7551

Issue

Vol. 6 , Iss. 6 (December 2024)

Article Type

Article

License

Copyright © 2024 Moussa Diagne Faye, Vini Yves Bernadin Loyara, Amadou Keita, Mamadou Diop, Angelbert Chabi Biaou, Mahamadou Koita, Hamma Yacouba

Creative Commons License

This is an open access article under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License.