Bibliometric analysis and visualization of translation assessment: Research theme, evolution and hotspots

Authors

  • Dai Tao

    Education Department, Faculty of Social Sciences & Liberal Arts, UCSI University;
    School of Foreign Languages and Cultures, Southwest University of Science and Techonology

  • Lin Siew Eng

    Education Department, Faculty of Social Sciences & Liberal Arts, UCSI University

DOI:

https://doi.org/10.59400/FLS.v6i2.2135

Abstract

Translation assessment, referring to evaluating various aspects related to translation, is crucial to the improvement of translation competence and quality. This study, taking valid papers on translation assessment published in WoS core collection from 2000 to 2022 as research samples, visualizes and reviews the research theme, research evolutions and emerging hot topics of translation assessment through bibliometric analysis. The number of annual publications shows that despite the general growth, the number of articles on translation assessment fluctuates from year to year. By Cite Space-based analysis of keywords co-occurrence, keyword clustering, time zone and burst detection, it finds that translation assessment mainly covers five themes, namely translation competence, translation quality, machine translation, translation teaching and training, and others. The overall evolution has been a gradual shift from concepts, metrics and evaluations of topics related to translation assessment into a topic of in-depth and interdisciplinary study. The hotspots in recent years include translation competence acquisition, neural machine translation and translation quality estimation. The findings reveal research evolutions and hot spots, propose implications for further translation assessment research and provide references for scholars who are interested in this topic.

Keywords:

translation assessment; translation competence; translation quality; bibliometrics analysis

References

Adom, D., Adu-Mensah, J., & Dake, D. A. (2020). Test, measurement, and evaluation: Understanding and use of the concepts in education. International Journal of Evaluation and Research in Education (IJERE), 9(1), 109. https://doi.org/10.11591/ijere.v9i1.20457

Airasian, P.W. (1997). Classroom assessment, 3rd ed. McGraw-Hill.

Akulina, K. V., & Tikhonova, E. V. (2021). Application of the translation quality assessment system in the training of linguist-translators. Yazyk i Kul’tura, 56, 129–145. https://doi.org/10.17223/19996195/56/8

Albir, A. H. (2007). Competence-based Curriculum Design for Training Translators. The Interpreter and Translator Trainer, 1(2), 163–195. https://doi.org/10.1080/1750399x.2007.10798757

Albir, P. Group. A. H., Galán-Mañas, A., Kuznik, A., Olalla-Soler, C., Rodríguez-Inés, P., & Romero, L. (2020). Translation competence acquisition. Design and results of the PACTE group’s experimental research. The Interpreter and Translator Trainer, 14(2), 95–233. https://doi.org/10.1080/1750399x.2020.1732601

Alos, J. (2015). Explicating the implicit: an empirical investigation into pragmatic competence in translator training. The Interpreter and Translator Trainer, 9(3), 287–305. https://doi.org/10.1080/1750399x.2015.1100398

al-Qinai, J. (2002). Translation Quality Assessment. Strategies, Parametres and Procedures. Meta, 45(3), 497–519. https://doi.org/10.7202/001878ar

Angelone, E. (2020). The impact of screen recording as a diagnostic process protocol on inter-rater consistency in translation assessment Erik Angelone, Kent State University. Journal of Specialised Translation, 34, 32-50.

Angelone, E. (2021). Broadening the scope of error categories in translation assessment through screen recording. Across Languages and Cultures, 22(2), 143–157. https://doi.org/10.1556/084.2021.00021

Arango-Keeth, F., & Koby, G. S. (2003). Assessing assessment. Beyond the Ivory Tower, 117–134. https://doi.org/10.1075/ata.xii.09ara

Bai, L., Feng, L., & Yan, M. (2018). Constructs and principles of the Chinese English Translation Proficiency Rating Scale. Modem Foreign Language, 41(1), 101-110.

Baniata, L. H., Park, S., & Park, S.-B. (2018). A Multitask-Based Neural Machine Translation Model with Part-of-Speech Tags Integration for Arabic Dialects. Applied Sciences, 8(12), 2502. https://doi.org/10.3390/app8122502

Beeby, A., Castillo, L., Fox, O., Galán Mañas, A., Hurtado Albir, A., Kuznik, A., Massana, G., Neunzig, W., Olalla, Ch., Rodríguez Inés, P., & Romero, L. (2015). Results of PACTE’s Experimental Research on the Acquisition of Translation Competence. Translation Spaces, 4(1), 29–53. Portico. https://doi.org/10.1075/ts.4.1.02bee

Bowker, L. (2002). Towards a Methodology for a Corpus-Based Approach to Translation Evaluation. Meta, 46(2), 345–364. https://doi.org/10.7202/002135ar

Brunette, L. (2000). Towards a Terminology for Translation Quality Assessment. The Translator, 6(2), 169–182. https://doi.org/10.1080/13556509.2000.10799064

Brunette, L. (2000). Towards a Terminology for Translation Quality Assessment. The Translator, 6(2), 169–182. https://doi.org/10.1080/13556509.2000.10799064

Burchardt, A., Lommel, A., Bywood, L., Harris, K., & Popović, M. (2016). Machine translation quality in an audiovisual context. Target, 28(2), 206–221. Portico. https://doi.org/10.1075/target.28.2.03bur

Cao, D. (2014). Teaching and learning legal translation. Semiotica, 2014(201). https://doi.org/10.1515/sem-2014-0022

Carl, M., & Báez, M. C. T. (2019). Machine translation errors and the translation process: A study across different languages. Journal of Specialised Translation, 31, 107-132.

Castilho, S., & O’Brien, S. (2018). Acceptability of machine-translated content: A multi-language evaluation by translators and end-users. Linguistica Antverpiensia, New Series – Themes in Translation Studies, 16. https://doi.org/10.52034/lanstts.v16i0.430

Chakhachiro, R. (2005). Revision for quality. Perspectives, 13(3), 255–238. https://doi.org/10.1080/09076760508668994

Chapelle, C. A., & Brindley, G. (2010). What is language assessment. 247-267.

Chatzikoumi, E. (2019). How to evaluate machine translation: A review of automated and human metrics. Natural Language Engineering, 26(2), 137–161. https://doi.org/10.1017/s1351324919000469

Che, S., Kamphuis, P., Zhang, S., Zhao, X., & Kim, J. H. (2022). A Visualization Analysis of Crisis and Risk Communication Research Using CiteSpace. International Journal of Environmental Research and Public Health, 19(5), 2923. https://doi.org/10.3390/ijerph19052923

Colina, S. (2008). Translation Quality Evaluation. The Translator, 14(1), 97–134. https://doi.org/10.1080/13556509.2008.10799251

Comelles, E., & Atserias, J. (2018). VERTa: a linguistic approach to automatic machine translation evaluation. Language Resources and Evaluation, 53(1), 57–86. https://doi.org/10.1007/s10579-018-9430-2

Cortez-Godínez, J. (2019). Influencia del nivel socioeconómico en la adquisición de la Competencia Traductora. Sendebar, 30(0). https://doi.org/10.30827/sendebar.v30i0.6594

Daems, J., Vandepitte, S., Hartsuiker, R. J., & Macken, L. (2017). Identifying the Machine Translation Error Types with the Greatest Impact on Post-editing Effort. Frontiers in Psychology, 8. https://doi.org/10.3389/fpsyg.2017.01282

De Sutter, G., Cappelle, B., De Clercq, O., Loock, R., & Plevoets, K. (2018). Towards a Corpus-based, Statistical Approach to Translation Quality: Measuring and Visualizing Linguistic Deviance in Student Translations. Linguistica Antverpiensia, New Series – Themes in Translation Studies, 16. https://doi.org/10.52034/lanstts.v16i0.440

Diao, B.C., Adnan, N., Harji, M. B., Ravindran, L. (2022). Evolution and hotspots of peer instruction: a visualized analysis using CiteSpace. Education and Information Technologies. 1-18.

Ding, Z.Y., Zang, H.F., Zhang, Q., Miao, G.M. & Chi, Y.H. Automatic machine translation evaluation based on sentence structure information. Paper presented at the 2009 International Conference on Asian Language Processing.

Du, W., & Saeheaw, T. (2020). Errors in translation: A case study of Chinese into English translation. Language Learning in Higher Education, 10(1), 129–142. https://doi.org/10.1515/cercles-2020-2009

Dugonik, J., Bošković, B., Brest, J., & Sepesy Maučec, M. (2019). Improving Statistical Machine Translation Quality Using Differential Evolution. Informatica, 30(4), 629–645. https://doi.org/10.15388/informatica.2019.222

Dussart, A. (2005). Faux sens, contresens, non-sens… un faux débat ? Meta, 50(1), 107–119. https://doi.org/10.7202/010661ar

Ellegaard, O., & Wallin, J. A. (2015). The bibliometric analysis of scholarly production: How great is the impact? Scientometrics, 105(3), 1809–1831. https://doi.org/10.1007/s11192-015-1645-z

Erten, C., Harding, P. J., Kobourov, S. G., Wampler, K., & Yee, G. (2004). Exploring the computing literature using temporal graph visualization SPIE Proceedings. https://doi.org/10.1117/12.539245

Finch, A., Yasuda, K., Okuma, H., Sumita, E., Nakamura, S. (2011). A Bayesian Model of Transliteration and Its Human Evaluation When Integrated into a Machine Translation System. IEICE Transactions on Information and Systems, E94-D(10), 1889–1900. https://doi.org/10.1587/transinf.e94.d.1889

Fomicheva, M., & Specia, L. (2019). Taking MT Evaluation Metrics to Extremes: Beyond Correlation with Human Judgments. Computational Linguistics, 45(3), 515–558. https://doi.org/10.1162/coli_a_00356

Gambier, Y., & van Doorslaer, L. (Eds.). (2007). The Metalanguage of Translation. Target, 19(2). Portico. https://doi.org/10.1075/target.19.2

Garcia, I. (2010). Is machine translation ready yet? Target, 22(1), 7–21. Portico. https://doi.org/10.1075/target.22.1.02gar

Gile, D. (2000). The History of Research into Conference Interpreting. Target, 12(2), 297–321. Portico. https://doi.org/10.1075/target.12.2.07gil

Göpferich, S. (2013). Translation competence. Target, 25(1), 61–76. Portico. https://doi.org/10.1075/target.25.1.06goe

Guo, Y.-M., Huang, Z.-L., Guo, J., Li, H., Guo, X.-R., & Nkeli, M. J. (2019). Bibliometric Analysis on Smart Cities Research. Sustainability, 11(13), 3606. https://doi.org/10.3390/su11133606

Guzmán, F., Joty, S., Màrquez, L., & Nakov, P. (2017). Machine translation evaluation with neural networks. Computer Speech & Language, 45, 180–200. https://doi.org/10.1016/j.csl.2016.12.005

Hague, D. R. (2015). Empathic accuracy and translator training: helping students imagine other minds. The Interpreter and Translator Trainer, 9(1), 20–29. https://doi.org/10.1080/1750399x.2015.1010359

Hague, D., Melby, A., & Zheng, W. (2011). Surveying Translation Quality Assessment. The Interpreter and Translator Trainer, 5(2), 243–267. https://doi.org/10.1080/13556509.2011.10798820

Hajmohammadi, A. (2005). TRANSLATION EVALUATION IN A NEWS AGENCY. Perspectives, 13(3), 215–224. https://doi.org/10.1080/09076760508668993

Han, C. (2020). Translation quality assessment: a critical methodological review. The Translator, 26(3), 257–273. https://doi.org/10.1080/13556509.2020.1834751

Han, C., Hu, B., Fan, Q., Duan, J., & Li, X. (2022). Using computerised comparative judgement to assess translation. Across Languages and Cultures, 23(1), 56–74. https://doi.org/10.1556/084.2022.00001

Han, Y., & Meng, S. (2022). Machine English Translation Evaluation System Based on BP Neural Network Algorithm. Computational Intelligence and Neuroscience, 2022, 1–10. https://doi.org/10.1155/2022/4974579

Hassan, H., Hearne, M., Way, A., & Sima’an, K. (2006). Syntactic phrase-based statistical machine translation. Paper presented at the 2006 IEEE Spoken Language Technology Workshop. https://doi.org/10.1109/slt.2006.326799

House, J. (2002). Translation Quality Assessment: Linguistic Description versus Social Evaluation. Meta, 46(2), 243–257. https://doi.org/10.7202/003141ar

Huang, Q., & Xin, X. (2020). A bibliometric analysis of translation criticism studies and its implications. Perspectives, 28(5), 737–755. https://doi.org/10.1080/0907676x.2020.1740750

Janmaijaya, M., Shukla, A., Abraham, A., & Muhuri, P. (2018). A Scientometric Study of Neurocomputing Publications (1992–2018): An Aerial Overview of Intrinsic Structure. Publications, 6(3), 32. https://doi.org/10.3390/publications6030032

Jiang, C. (2010). Quality assessment for the translation of museum texts: application of a systemic functional model. Perspectives, 18(2), 109–126. https://doi.org/10.1080/09076761003678734

Károly, A. (2014). Translation in foreign language teaching: A case study from a functional perspective. Linguistics and Education, 25, 90–107. https://doi.org/10.1016/j.linged.2013.09.009

Karoubi, B. (2016). Translation quality assessment demystified. Babel, 62(2), 253–277. Portico. https://doi.org/10.1075/babel.62.2.05kar

Kasperavičienė, R., & Horbačauskienė, J. (2020). Self-revision and other-revision as part of translation competence in translator training. Journal of Language and Cultural Education, 8(1), 117–133. https://doi.org/10.2478/jolace-2020-0007

Ke, X.H. (2009). An Automatic Translation Evaluation System Based on Semantic Similarities and Fuzzy Neartude. Paper presented at the 2009 International Conference on Environmental Science and Information Application Technology.

Kiraly, D. (2000). A Social Constructivist Approach to Translator Education: Empowerment from Theory to Practice. St Jerome Publishing.

Kit, C., & Wong, T. M. (2008). Comparative evaluation of online machine translation systems with legal texts. Law Library Journal, 100(2), 299-321.

Korol, T. (2020). Translation assessment: Is there anything to be tested objectively? Advanced Education, 15, 32-40. https://doi.org/10.20535/2410-8286.200720

Korol, T. (2021). Digital teacher feedback as a translation assessment tool. Journal of Teaching English for Specific and Academic Purposes, 575. https://doi.org/10.22190/jtesap2104575k

Kudo, T. (2018). Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.18653/v1/p18-1007

Lee, Y. O., & Ronowicz, E. (2014). The development of an error typology to assess translation from English into Korean in class. Babel, 60(1), 35–51. Portico. https://doi.org/10.1075/babel.60.1.03lee

Li, D. (2006). Making Translation Testing More Teaching-oriented: A Case Study of Translation Testing in China. Meta, 51(1), 72–88. https://doi.org/10.7202/012994ar

Liu, L., Fujita, A., Utiyama, M., Finch, A., & Sumita, E. (2017). Translation Quality Estimation Using Only Bilingual Corpora. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 25(9), 1762–1772. https://doi.org/10.1109/taslp.2017.2716195

Lommel, A., Uszkoreit, H., & Burchardt, A. (2014). Multidimensional Quality Metrics (MQM): A Framework for Declaring and Describing Translation Quality Metrics. Tradumàtica Tecnologies de La Traducció, 12, 455–463. https://doi.org/10.5565/rev/tradumatica.77

Maier, C. (2000). Introduction. The Translator, 6(2), 137–148. https://doi.org/10.1080/13556509.2000.10799062

Maier, C. (2014). Evaluation and Translation. Routledge. https://doi.org/10.4324/9781315759999

Martínez Melis, N., & Hurtado Albir, A. (2002). Assessment In Translation Studies: Research Needs. Meta, 46(2), 272–287. https://doi.org/10.7202/003624ar

McAlester, G. (1999). “The Source Text in Translation Assessment”. In Words, Words, Words. The Translator and the Language Learner. In: Anderman G, Rogers, M. (editors). pp. 169-178. Clevedon, Philadelphia and Adelaide: Multilingual Matters.

Mei, H., & Chen, H. (2022). Assessing Students’ Translation Competence: Integrating China’s Standards of English With Cognitive Diagnostic Assessment Approaches. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.872025

Mellinger, C. D. (2018). Re-thinking translation quality. Target, 310–331. https://doi.org/10.1075/target.16104.mel

Moorkens, J. (2018). What to expect from Neural Machine Translation: a practical in-class translation evaluation exercise. The Interpreter and Translator Trainer, 12(4), 375–387. https://doi.org/10.1080/1750399x.2018.1501639

Munday, J. (2016). Introducing Translation Studies. Routledge. https://doi.org/10.4324/9781315691862

Nida, E., & Taber, C. (2003). The Theory and Practice of Translation. https://doi.org/10.1163/9789004496330

Nikolaeva, S., & Korol, T. (2021). Prospective Philologists’ translation assessment triangulation: screen video recording and think aloud protocol combination. Advanced Education, 8(18), 30–41. LOCKSS. https://doi.org/10.20535/2410-8286.228550

Olawumi, T. O., & Chan, D. W. M. (2018). A scientometric review of global research on sustainability and sustainable development. Journal of Cleaner Production, 183, 231–250. https://doi.org/10.1016/j.jclepro.2018.02.162

Orozco, M., & Hurtado Albir, A. (2004). Measuring Translation Competence Acquisition. Meta, 47(3), 375–402. https://doi.org/10.7202/008022ar

PACET (2019). Evolution of the Efficacy of the Translation Process in Translation Competence Acquisition. Meta: Journal Des Traducteurs, 64(1), 242. https://doi.org/10.7202/1065336ar

PACTE (2005). Investigating translation competence: conceptual and methodological issues. Volet Traduction, 50(2), 609–619. https://doi.org/10.7202/011004ar

Palumbo, G. (2009). Key terms in translation studies: Bloomsbury Publishing.

Pan, W., Jian, L., & Liu, T. (2019). Grey system theory trends from 1991 to 2018: a bibliometric analysis and visualization. Scientometrics, 121(3), 1407–1434. https://doi.org/10.1007/s11192-019-03256-z

Pollack, J., & Adler, D. (2015). Emergent trends and passing fads in project management research: A scientometric analysis of changes in the field. International Journal of Project Management, 33(1), 236–248. https://doi.org/10.1016/j.ijproman.2014.04.011

Popescu, T. (2013). A Corpus-based Approach to Translation Error Analysis. a Case-study of Romanian EFL Learners. Procedia - Social and Behavioral Sciences, 83, 242–247. https://doi.org/10.1016/j.sbspro.2013.06.048

Pranckutė, R. (2021). Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World. Publications, 9(1), 12. https://doi.org/10.3390/publications9010012

Pym, A. (2004). Redefining Translation Competence in an Electronic Age. In Defence of a Minimalist Approach. Meta, 48(4), 481–497. https://doi.org/10.7202/008533ar

Pym, A. (2013). Research skills in translation studies: What we need training in. Across Languages and Cultures, 14(1), 1–14. https://doi.org/10.1556/acr.14.2013.1.1

Sadat, F., & Habash, N. (2006). Combination of Arabic preprocessing schemes for statistical machine translation. Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the ACL - ACL ’06. https://doi.org/10.3115/1220175.1220176

Schwenk, H., & Esteve, Y. (2008). Data selection and smoothing in an open-source system for the 2008 NIST machine translation evaluation. Interspeech 2008. https://doi.org/10.21437/interspeech.2008-676

Şerban, A., & Mureşan, V. (2019). Register and Translation Competence Acquisition. Investigating Student Attitudes. Studia Universitatis Babeș-Bolyai Philologia, 64(4), 213–230. https://doi.org/10.24193/subbphilo.2019.4.13

Si, Y.H., Shi, J.G., Chen, J.D., Wu, G.D., & Wang, H.M., (2019). Reviewing sustainable development research in China from the perspective of China Population, Resources and Environment. China population, resources and environment, 29(7), 166-176.

Su, W. (2021). Understanding rubric use in peer assessment of translation. Perspectives, 30(1), 71–85. https://doi.org/10.1080/0907676x.2020.1862260

Tingting, L., & Mengyu, X. (2019). Analysis and evaluation on the quality of news text machine translation based on neural network. Multimedia Tools and Applications, 79(23–24), 17015–17026. https://doi.org/10.1007/s11042-019-7532-5

Tymoczko, M. (2006). Trajectories of Research in Translation Studies. Séances Plénières, 50(4), 1082–1097. https://doi.org/10.7202/012062ar

Waddington, C. (2002). Different Methods of Evaluating Student Translations: The Question of Validity. Meta, 46(2), 311–325. https://doi.org/10.7202/004583ar

Wang, H., Wu, H., He, Z., Huang, L., & Church, K. W. (2022). Progress in Machine Translation. Engineering, 18, 143–153. https://doi.org/10.1016/j.eng.2021.03.023

Wang, L., & Wang, X. (2020). How to evaluate literary translations in the classroom context: through error analysis or scale-based method? Current Trends in Translation Teaching and Learning E, 7, 276-313.

Williams, M. (2002). The Application of Argumentation Theory to Translation Quality Assessment. Meta, 46(2), 326–344. https://doi.org/10.7202/004605ar

Xiao, W.Q. (2012). Study on the Undergraduate Translation Major Testing. People’s Publishing House.

Yang, M., Zhu, J., Li, J., Wang, L., Qi, H., Li, S., & Daxin, L. (2008). Extending BLEU Evaluation Method with Linguistic Weight. 2008 The 9th International Conference for Young Computer Scientists. https://doi.org/10.1109/icycs.2008.362

Yang, M.-Y., Sun, S.-Q., Zhu, J.-G., Li, S., Zhao, T.-J., & Zhu, X.-N. (2011). Improvement of Machine Translation Evaluation by Simple Linguistically Motivated Features. Journal of Computer Science and Technology, 26(1), 57–67. https://doi.org/10.1007/s11390-011-9415-8

Yang, Z. H. (2019). Studies on Translation Testing and Assessment. Foreign Language Teaching and Research Press.

Yao, J.M., Qu, Y.Q., Zhu, Q.M., & Zhang, J. (2006). A Visualization method for machine translation evaluation results. Proceedings (Stymne et al., 2012) of the 20th Pacific Asia Conference on Language, Information and Computation.

Yilmaz, I. (2019). Bibliometric Analysis of Bibliometric Studies on Tourism Published in Turkey. Anais Brasileiros de Estudos Turísticos - ABET, 9(1, 2 e 3). https://doi.org/10.34019/2238-2925.2019.v9.27111

Yunxing, L. (2001). Translation evaluation in a systemic perspective. Perspectives, 9(1), 15–21. https://doi.org/10.1080/0907676x.2001.9961403

Zhu, J., & Wang, H. (2006). The effect of translation quality in MT-based cross-language information retrieval. Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the ACL - ACL ’06. https://doi.org/10.3115/1220175.1220250

Zlatnar Moe, M., Mikolic Juznic, T., & Žigon, T. (2015). I know languages, therefore, I can translate? Translation and Interpreting Studies, 10(1), 87–107. Portico. https://doi.org/10.1075/tis.10.1.05moe

Downloads

How to Cite

Tao, D., & Eng, L. S. (2024). Bibliometric analysis and visualization of translation assessment: Research theme, evolution and hotspots. Forum for Linguistic Studies, 6(2). https://doi.org/10.59400/FLS.v6i2.2135

Issue

Article Type

Article