AI-translated poetry: Ivan Franko's poems in GPT-3.5-driven machine and human-produced translations

Authors

  • Viacheslav Karaban

    Department of the Theory and Practice of Translation from English, Taras Shevchenko National University of Kyiv
  • Anna Karaban

    Department of English Philology and Intercultural Communication, Taras Shevchenko National University of Kyiv

DOI:

https://doi.org/10.59400/fls.v6i1.1994

Abstract

The article presents a detailed comparative analysis of translations of twelve great Ukrainian poet Ivan Franko's poems done by translator Percival Cundy and the GPT-3.5 AI language model. Using various manual and automatic analytical research methods and techniques, we analyzed the translations' merits, demerits, and eight essential qualitative and quantitative linguistic and poetic characteristics to verify a hypothesis that human and GPT-3.5-driven machine translations can be quite comparable in terms of their quality and poetic features. The results obtained sufficiently prove the hypothesis and suggest that developing AI translation potential for poetry translation can help build more capable, diversified, and nuanced large language models. The AI revolutionary breakthrough in translation makes it quite possible to acquaint satisfactorily the wider public with the poetic heritage of the world's nations, especially those using minor languages, whose poetry is evidently under-translated. A follow-up study is desirable to assess the progress made by GPT4.0 and its possible later versions in poetry translation, as compared with GPT-3.5.

Keywords:

AI translation, Ivan Franco's poems, human translation, comparative analysis, poetry translation, translation from Ukrainian into English

References

[1] Agirrezabal M, Astigarraga A, Arrieta B, et al. (2016). ZeuScansion: A tool for scansion of English poetry. Journal of Language Modelling. 2016, 4(1). doi: 10.15398/jlm.v4i1.102

[2] Alowedi N, Al-Ahdal A (2023). Artificial Intelligence based Arabic-to-English machine versus human translation of poetry: An analytical study of outcomes. Journal of Namibian Studies : History Politics Culture. 2023, 33. doi: 10.59670/jns.v33i.800

[3] Badura M, Lampert M, Dreżewski R (2022). System Supporting Poetry Generation Using Text Generation and Style Transfer Methods. Procedia Computer Science. 2022, 207: 3310-3319. doi: 10.1016/j.procs.2022.09.389

[4] Bird S, Klein E, Loper E (2009). Natural language processing with Python: analyzing text with the natural language toolkit. O’Reilly Media, Inc.

[5] Boase-Beier J (2013). Poetry Translation. In: Baker M, Saldanha G (editors). The Routledge handbook of translation studies. Routledge. pp. 475-487

[6] Boulenouar M (2022). A Comparative Study of Machine and Human Translation: The Case of English-Arabic Literary Translations. Journal of Language and Linguistic Studies, 18 (Special Issue 1): 176-191.

[7] Bunchuk B (2009). The innovation of Ivan Franko’s verse form as a sign of the modernist character of his poetry. In: The Bible and Culture 11: 112-117.

[8] Chakrabarty T, Saakyan A, Muresan S (2021). Don’t Go Far Off: An Empirical Study on Neural Poetry Translation. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Published online 2021. doi: 10.18653/v1/2021.emnlp-main.577

[9] Dai J, Shen H, et al. (2022). The Differences Between Machine Translation and Human Translation from the Perspective of Literary Texts. International Journal of Arts and Social Science. 5(10): 112-133.

[10] Dastjerdi HV, Khosravani Y, Shokrollahi M, et al. (2011). Translation Quality Assessment (TQA): A Semiotic Model for Poetry Translation. Lebende Sprachen. 2011, 56(2). doi: 10.1515/les.2011.021

[11] Interactive BLEU score evaluator (n. d.) Tilde MT. Available from: https://www.letsmt.eu/Bleu.aspx

[12] Franko I (2021). Faded leaves. Litopys.

[13] Franko I, Cundy P, Manning CA (1948). Ivan Franko, the Poet of Western Ukraine. Philosophical Library.

[14] Genzel D, Uszkoreit J, Och F (2010). “Poetic” statistical machine translation: rhyme and meter. EMNLP. Available from: https://research.google/pubs/pub36745/

[15] Ghazvininejad M, Choi Y, Knight K (2018). Neural Poetry Translation. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). Published online 2018. doi: 10.18653/v1/n18-2011

[16] Grace K, Salvatier J, Dafoe A, et al. (2018). Viewpoint: When Will AI Exceed Human Performance? Evidence from AI Experts. Journal of Artificial Intelligence Research. 2018, 62: 729-754. doi: 10.1613/jair.1.11222

[17] Hadley J (2020). Literary Machine Translation. Counterpoint. (4): 15-18. Available from: https://www.ceatl.eu/wp-content/uploads/2020/12/Counterpoint_2020_04_article_04.pdf

[18] Humblé P (2019). Machine translation and poetry. The case of English and Portuguese. Ilha do Desterro, 72: 41-56. doi: 10.5007/2175-8026.2019v72n2p41

[19] Hromyak RT, Kovaliv YI, Teremko VI (1997). Dictionary of Literary Studies. Akademia. p. 331.

[20] Jacobs AM (2018). The Gutenberg English Poetry Corpus: Exemplary Quantitative Narrative Analyses. Frontiers in Digital Humanities. 2018, 5. doi: 10.3389/fdigh.2018.00005

[21] Jacobs AM, Kinder A (2022). Computational analyses of the topics, sentiments, literariness, creativity and beauty of texts in a large Corpus of English Literature. arXiv:2201.04356.

[22] Johansen M (1922). Elementary laws of versification. Vseukrlitkom.

[23] Kenny D, Winters M (2020). Machine translation, ethics and the literary translator’s voice. Translation Spaces. 2020, 9(1): 123-149. doi: 10.1075/ts.00024.ken

[24] Kovaliv Y (2007). Literary encyclopedia. Kyiv: Academia Publishing House.

[25] Kulchytskyi I, Tsiokh L, Malaniuk M (2018). Quantitative Equivalence Level in Poetry Translation. 2018 IEEE 13th International Scientific and Technical Conference on Computer Sciences and Information Technologies (CSIT). Published online September 2018. doi: 10.1109/stc-csit.2018.8526715

[26] Kuzman T, Vintar Š, Arcan M (2019). Neural machine translation of literary texts from English to Slovene. In Proceedings of the qualities of literary machine translation (pp. 1-9). Available from: https://aclanthology.org/W19-7301.pdf

[27] Ma Y, Wang B. Description and Quality Assessment of Poetry Translation: Application of a Linguistic Model. Contrastive Pragmatics. 2020, 3(1): 89-111. doi: 10.1163/26660393-bja10015

[28] Matusov E (2019). The challenges of using neural machine translation for literature. In Proceedings of the qualities of literary machine translation (pp. 10-19). Available From: https://aclanthology.org/W19-7302.pdf

[29] OpenAI (2023). ChatGPT (Mar 14 version) [Large language model]. Available From: https://chat.openai.com/chat

[30] Poibeau T (2022). On “Human Parity” and “Super Human Performance” in Machine Translation Evaluation. In Language Resource and Evaluation Conference. June 2022, Marseille, France.

[31] Seljan S, Dunder I, Pavlovski M (2020). Human Quality Evaluation of Machine-Translated Poetry. 2020 43rd International Convention on Information, Communication and Electronic Technology (MIPRO). Published online September 28, 2020. doi: 10.23919/mipro48935.2020.9245436

[32] Simonton DK (1990). Lexical choices and aesthetic success: A computer content analysis of 154 Shakespeare sonnets. Computers and the Humanities. 1990, 24(4): 251-264. doi: 10.1007/bf00123412

[33] Studzińska J (2020). Turing test for (automatic) translation of poetry (Polish). Porównania. 2020, 26: 299-313. doi: 10.14746/por.2020.1.17

[34] Toral A, Way A (2018). What Level of Quality Can Neural Machine Translation Attain on Literary Text? Translation Quality Assessment. Published online 2018: 263-287. doi: 10.1007/978-3-319-91241-7_12

[35] Vincent R (2019). Multilingual Poetry Generation [Master’s thesis]. NTNU (Norwegian University of Science and Technology).

[36] Visby M (2020). The future relationship of literary translation and AI: Reflections from CEATL president. Counterpoint. 4: 28-31.

[37] Zong Z (2018). Research on the Relations Between Machine Translation and Human Translation. Journal of Physics: Conference Series. 2018, 1087: 062046. doi: 10.1088/1742-6596/1087/6/062046

Downloads

Issue

Vol. 6 , Iss. 1 (March 2024)

Article Type

Article

License

Copyright © 2024 Viacheslav Karaban, Anna Karaban

Creative Commons License

This is an open access article under the Creative Commons Attribution 4.0 International License.