Validity and Reliability of an Evaluation Instrument for English Curriculum System in Chinese Vocational Higher Education Institutions

Authors

  • Qu Yuping

    Faculty of Social Sciences and Liberal Arts, UCSI University, Kuala Lumpur, KL, Malaysia

  • Arasinah Kamis

    Faculty of Technical & Vocational, Universiti Pendidikan Sultan Idris, 35900 Tanjong Malim Perak, Malaysia

  • Liping Jiang

    School of Foreign Languages & International Business, Guangdong Mechanical & Electrical Polytechnic, Guangzhou, China

DOI:

https://doi.org/10.30564/fls.v6i4.6599
Received: 3 April 2024 | Revised: 28 April 2024 | Accepted: 1 May 2024 | Published: 4 September 2024

Abstract

The English curriculum is the basic part of the curriculum system of vocational higher education institutions (VHEIs) in China, and it is the public compulsory curriculum that students of all majors must study. It not only cultivates students' ability to learn and apply English but also lays an English foundation for students' future further learning and lifelong development. The purpose of this study is to validate the evaluation instrument for English curriculum system. This is a quantitative research and the questionnaire has adapted from previous scholars. A stratified random sampling technique has been employed to select 350 English lecturers from a total of 864 English lecturers at 20 vocational HEIs in Shandong Province, China. The results showed: ⅰ) Each dimension Clonbach alpha coefficient and McDonald’s omega (Composite reliability) are more than 0.70. ⅱ) For the convergent validity, each item factor loading > 0.5, each dimension AVE >0.5. ⅲ) For the discriminant validity, the square root of the AVE of the five dimensions is greater than the corresponding correlation coefficients between the two dimensions. ⅳ) Measurement model meet the fit indices, that CMIN χ2=411.228, with degrees of freedom (df) 314, CMIN/df=1.310 (1-3), CFI=0.979 (>0.9), GFI=0.916 (>0.9), NFI=0.916 (>0.9), and RMSEA=0.031(<0.08). The results prove that this English curriculum implementation instrument has good validity and reliability. The instrument can measure the level of English curriculum systems in VHEIs in Shandong Province, China. The research findings showed there were five dimensions in the evaluation of English curriculum system. The research can be used as the basis for evaluation curriculum system for other researchers and education administrators.

Keywords:

English curriculum system; CIPPO Evaluation Model; Reliability; Confirmatory factor analysis; Vocational Higher Education Institutions

References

Bagozzi, R.P., Yi, Y., 1988. On the evaluation of structural equation models. Journal of the Academy of Marketing Science. 16(1), 74--94. DOI: https://doi.org/10.1007/bf02723327

Bai, H., 2021. Dilemma and path selection of English curriculum standards for Higher Vocational Education. Vocational Education Research. (12), 60--64.

Bentler, P.M., 1990. Comparative fit indexes in structural models. Psychological Bulletin. 107(2), 238--246. DOI: https://doi.org/10.1037/0033-2909.107.2.238

Brown, T.A., 2015. Confirmatory Factor Analysis for Applied Research, Second Edition. Guilford Publications.

Byrne, B.M., 1998. Structural equation modeling with LISREL, PRELIS and SIMPLIS: Basic concepts, applications and programming. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Byrne, B.M., Van De Vijver, F.J.R., 2010. Testing for Measurement and Structural Equivalence in Large-Scale Cross-Cultural Studies: Addressing the issue of Nonequivalence. International Journal of Testing, 10(2), 107--132. DOI: https://doi.org/10.1080/15305051003637306

Chen, Y., 2022. A study on the current problems of evaluation in public English teaching in higher vocational colleges. Journal of Hubei Open Vocational College. (13), 186--188.

Creswell, J.W., Creswell, J.D., 2017. Research design: Qualitative, Quantitative, and Mixed Methods Approaches. SAGE Publications.

Fornell, C., Larcker, D.F., 1981. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research. 18(1), 39--50. DOI: https://doi.org/10.1177/002224378101800104

General Office of the Ministry of Education. (2021, March 23). English curriculum standards for vocational higher education institutions (2021). Available from: https://www.gov.cn/zhengce/zhengceku/2021-04/10/content_5598801.htm

Göbel, J. (2019, November 11). "Discover your talents." deutschland.de. Available from: https://www.deutschland.de/zh-hans/topic/jingji/lianheguojiaokewenzuzhicujinzhiyejiaoyu

Goodlad, J.I., 1979. Curriculum Inquiry: The study of Curriculum practice. Available from: https://ci.nii.ac.jp/ncid/BA05076058

Hair, J.F., Black, W.C., Babin, B.J., et al., 2006. Multivariate data analysis (6th ed.). Pearson Prentice Hall Upper Saddle River, NJ.

Hair, J., Black, B., Babin, B., et al., 2010. Multivariate Data Analysis: A Global Perspective (7th ed.). New Jersey, USA: Pearson Education Inc.

Hamid, M.R., Sutaman, S., Natsir, M., et al., 2022. The development of an evaluation instrument for the implementation of the Arabic language curriculum in Islamic High School. Jurnal Al-Bayan, 14(1), 242--257. DOI: https://doi.org/10.24042/albayan.v14i1.10303

Hayes, A.F., Coutts, J.J., 2020. Use Omega Rather than Cronbach’s Alpha for Estimating Reliability. But... Communication Methods and Measures. 14(1), 1--24. DOI: https://doi.org/10.1080/19312458.2020.1718629

Imansari, N., Sutadji, E., 2017. A conceptual framework curriculum evaluation Electrical engineering education. International Journal of Evaluation and Research in Education. 6(4), 265. DOI: https://doi.org/10.11591/ijere.v6i3.7624

Jiang, L., 2022. Effective Utilization of Computer-Mediated Communication Technology in Network-based Foreign Language Teaching. Wireless communications and mobile computing. 1--7. DOI: https://doi.org/10.1155/2022/1048311

Jiang, L., 2024. Factors influencing EFL teachers' implementation of SPOC-based blended learning in higher vocational colleges in China: A study based on grounded theory. Interactive Learning Environments. 32(3), 859--878. DOI: https://doi.org/10.1080/10494820.2022.2100428

Jiang, L., Al-Shaibani, G.K.S., 2024. Influencing factors of students' small private online course-based learning adaptability in a higher vocational college in China. Interactive Learning Environments. 32(3), 972--993. DOI: https://doi.org/10.1080/10494820.2022.2105901

Jiang, L., Liang, X., 2023. Influencing factors of Chinese EFL students' continuance learning intention in SPOC-based blended learning environment. Education and Information Technologies. 28(10), 13379--13404. DOI: https://doi.org/10.1007/s10639-023-11734-4

Kline, R.B., 2023. Principles and practice of structural equation modeling. Guilford publications.

Krejcie, R.V., Morgan, D.W., 1970. Determining sample size for research activities. Educational and Psychological Measurement. 30(3), 607--610. DOI: https://doi.org/10.1177/001316447003000308

Kusmiyati, N., Hamidah, K., 2023. CIPPO model Evaluation on the English Language training program at the Indonesian Navy Education Services. IJHCM (International Journal of Human Capital Management) (Online). 7(1), 104--114. DOI: https://doi.org/10.21009/ijhcm.07.01.8

Marsh, H.W., Hocevar, D., 1985. Application of confirmatory factor analysis to the study of self-concept: First- and higher order factor models and their invariance across groups. Psychological Bulletin. 97(3), 562--582. DOI: https://doi.org/10.1037/0033-2909.97.3.562

Purnawirawan, O., Chintya, P.P., Sholihah, M., 2020. The application of CIPPO Evaluation Model in evaluating the performance of school for Producing Entrepreneurs programs in Vocational High School. Proceedings of the International Conference on Science and Education and Technology (ISET 2019). DOI: https://doi.org/10.2991/assehr.k.200620.075

Robiah, R., Hendarman, H., Hidayat, R., 2023. Evaluasi Program Literasi Anak dengan Pendekatan Model CIPPO. Murhum. 4(1), 528--539. DOI: https://doi.org/10.37985/murhum.v4i1.262

Rubio, D.M., Berg-Weger, M., Tebb, S., et al., 2003. Objectifying content validity: Conducting a content validity study in social work research. Social Work Research. 27(2), 94--104. DOI: https://doi.org/10.1093/swr/27.2.94

Schumacker, R.E., Lomax, R.G., 2012. A Beginner’s Guide to Structural equation modeling: Third Edition. Routledge.

Scriven, M., 1996. Types of evaluation and types of evaluator. Evaluation Practice. 17(2), 151--161. DOI: https://doi.org/10.1016/s0886-1633(96)90020-3

Stein, C.M., Morris, N.J., Nock, N.L., 2012. Structural equation modeling. Statistical human genetics: Methods and protocols. 495--512.

Sürücü, L., Maşlakçı, A., 2020. VALIDITY AND RELIABILITY IN QUANTITATIVE RESEARCH. Business & Management Studies: An International Journal (Online). 8(3), 2694--2726. DOI: https://doi.org/10.15295/bmij.v8i3.1540

Widayanto, L.D., Soeharto, S., Sudira, P., et al., 2021. Implementation of the Education and Training Program seen from the CIPPO Perspective. Journal of Education Research and Evaluation (Online). 5(4), 614. DOI: https://doi.org/10.23887/jere.v5i4.36826

Downloads

How to Cite

Qu Yuping, Arasinah Kamis, & Jiang, L. (2024). Validity and Reliability of an Evaluation Instrument for English Curriculum System in Chinese Vocational Higher Education Institutions. Forum for Linguistic Studies, 6(4), 9–17. https://doi.org/10.30564/fls.v6i4.6599