AI-translated poetry: Ivan Franko's poems in GPT-3.5-driven machine and human-produced translations

Authors

  • Viacheslav Karaban

    Department of the Theory and Practice of Translation from English, Taras Shevchenko National University of Kyiv
  • Anna Karaban

    Department of English Philology and Intercultural Communication, Taras Shevchenko National University of Kyiv

DOI:

https://doi.org/10.59400/fls.v6i1.1994

Abstract

The article presents a detailed comparative analysis of translations of twelve great Ukrainian poet Ivan Franko's poems done by translator Percival Cundy and the GPT-3.5 AI language model. Using various manual and automatic analytical research methods and techniques, we analyzed the translations' merits, demerits, and eight essential qualitative and quantitative linguistic and poetic characteristics to verify a hypothesis that human and GPT-3.5-driven machine translations can be quite comparable in terms of their quality and poetic features. The results obtained sufficiently prove the hypothesis and suggest that developing AI translation potential for poetry translation can help build more capable, diversified, and nuanced large language models. The AI revolutionary breakthrough in translation makes it quite possible to acquaint satisfactorily the wider public with the poetic heritage of the world's nations, especially those using minor languages, whose poetry is evidently under-translated. A follow-up study is desirable to assess the progress made by GPT4.0 and its possible later versions in poetry translation, as compared with GPT-3.5.

Keywords:

AI translation; Ivan Franco's poems, human translation; comparative analysis; poetry translation; translation from Ukrainian into English

References

Agirrezabal M, Astigarraga A, Arrieta B, et al. (2016). ZeuScansion: A tool for scansion of English poetry. Journal of Language Modelling. 2016, 4(1). doi: 10.15398/jlm.v4i1.102

Alowedi N, Al-Ahdal A (2023). Artificial Intelligence based Arabic-to-English machine versus human translation of poetry: An analytical study of outcomes. Journal of Namibian Studies : History Politics Culture. 2023, 33. doi: 10.59670/jns.v33i.800

Badura M, Lampert M, Dreżewski R (2022). System Supporting Poetry Generation Using Text Generation and Style Transfer Methods. Procedia Computer Science. 2022, 207: 3310-3319. doi: 10.1016/j.procs.2022.09.389

Bird S, Klein E, Loper E (2009). Natural language processing with Python: analyzing text with the natural language toolkit. O’Reilly Media, Inc.

Boase-Beier J (2013). Poetry Translation. In: Baker M, Saldanha G (editors). The Routledge handbook of translation studies. Routledge. pp. 475-487

Boulenouar M (2022). A Comparative Study of Machine and Human Translation: The Case of English-Arabic Literary Translations. Journal of Language and Linguistic Studies, 18 (Special Issue 1): 176-191.

Bunchuk B (2009). The innovation of Ivan Franko’s verse form as a sign of the modernist character of his poetry. In: The Bible and Culture 11: 112-117.

Chakrabarty T, Saakyan A, Muresan S (2021). Don’t Go Far Off: An Empirical Study on Neural Poetry Translation. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Published online 2021. doi: 10.18653/v1/2021.emnlp-main.577

Dai J, Shen H, et al. (2022). The Differences Between Machine Translation and Human Translation from the Perspective of Literary Texts. International Journal of Arts and Social Science. 5(10): 112-133.

Dastjerdi HV, Khosravani Y, Shokrollahi M, et al. (2011). Translation Quality Assessment (TQA): A Semiotic Model for Poetry Translation. Lebende Sprachen. 2011, 56(2). doi: 10.1515/les.2011.021

Interactive BLEU score evaluator (n. d.) Tilde MT. Available from: https://www.letsmt.eu/Bleu.aspx

Franko I (2021). Faded leaves. Litopys.

Franko I, Cundy P, Manning CA (1948). Ivan Franko, the Poet of Western Ukraine. Philosophical Library.

Genzel D, Uszkoreit J, Och F (2010). “Poetic” statistical machine translation: rhyme and meter. EMNLP. Available from: https://research.google/pubs/pub36745/

Ghazvininejad M, Choi Y, Knight K (2018). Neural Poetry Translation. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). Published online 2018. doi: 10.18653/v1/n18-2011

Grace K, Salvatier J, Dafoe A, et al. (2018). Viewpoint: When Will AI Exceed Human Performance? Evidence from AI Experts. Journal of Artificial Intelligence Research. 2018, 62: 729-754. doi: 10.1613/jair.1.11222

Hadley J (2020). Literary Machine Translation. Counterpoint. (4): 15-18. Available from: https://www.ceatl.eu/wp-content/uploads/2020/12/Counterpoint_2020_04_article_04.pdf

Humblé P (2019). Machine translation and poetry. The case of English and Portuguese. Ilha do Desterro, 72: 41-56. doi: 10.5007/2175-8026.2019v72n2p41

Hromyak RT, Kovaliv YI, Teremko VI (1997). Dictionary of Literary Studies. Akademia. p. 331.

Jacobs AM (2018). The Gutenberg English Poetry Corpus: Exemplary Quantitative Narrative Analyses. Frontiers in Digital Humanities. 2018, 5. doi: 10.3389/fdigh.2018.00005

Jacobs AM, Kinder A (2022). Computational analyses of the topics, sentiments, literariness, creativity and beauty of texts in a large Corpus of English Literature. arXiv:2201.04356.

Johansen M (1922). Elementary laws of versification. Vseukrlitkom.

Kenny D, Winters M (2020). Machine translation, ethics and the literary translator’s voice. Translation Spaces. 2020, 9(1): 123-149. doi: 10.1075/ts.00024.ken

Kovaliv Y (2007). Literary encyclopedia. Kyiv: Academia Publishing House.

Kulchytskyi I, Tsiokh L, Malaniuk M (2018). Quantitative Equivalence Level in Poetry Translation. 2018 IEEE 13th International Scientific and Technical Conference on Computer Sciences and Information Technologies (CSIT). Published online September 2018. doi: 10.1109/stc-csit.2018.8526715

Kuzman T, Vintar Š, Arcan M (2019). Neural machine translation of literary texts from English to Slovene. In Proceedings of the qualities of literary machine translation (pp. 1-9). Available from: https://aclanthology.org/W19-7301.pdf

Ma Y, Wang B. Description and Quality Assessment of Poetry Translation: Application of a Linguistic Model. Contrastive Pragmatics. 2020, 3(1): 89-111. doi: 10.1163/26660393-bja10015

Matusov E (2019). The challenges of using neural machine translation for literature. In Proceedings of the qualities of literary machine translation (pp. 10-19). Available From: https://aclanthology.org/W19-7302.pdf

OpenAI (2023). ChatGPT (Mar 14 version) [Large language model]. Available From: https://chat.openai.com/chat

Poibeau T (2022). On “Human Parity” and “Super Human Performance” in Machine Translation Evaluation. In Language Resource and Evaluation Conference. June 2022, Marseille, France.

Seljan S, Dunder I, Pavlovski M (2020). Human Quality Evaluation of Machine-Translated Poetry. 2020 43rd International Convention on Information, Communication and Electronic Technology (MIPRO). Published online September 28, 2020. doi: 10.23919/mipro48935.2020.9245436

Simonton DK (1990). Lexical choices and aesthetic success: A computer content analysis of 154 Shakespeare sonnets. Computers and the Humanities. 1990, 24(4): 251-264. doi: 10.1007/bf00123412

Studzińska J (2020). Turing test for (automatic) translation of poetry (Polish). Porównania. 2020, 26: 299-313. doi: 10.14746/por.2020.1.17

Toral A, Way A (2018). What Level of Quality Can Neural Machine Translation Attain on Literary Text? Translation Quality Assessment. Published online 2018: 263-287. doi: 10.1007/978-3-319-91241-7_12

Vincent R (2019). Multilingual Poetry Generation [Master’s thesis]. NTNU (Norwegian University of Science and Technology).

Visby M (2020). The future relationship of literary translation and AI: Reflections from CEATL president. Counterpoint. 4: 28-31.

Zong Z (2018). Research on the Relations Between Machine Translation and Human Translation. Journal of Physics: Conference Series. 2018, 1087: 062046. doi: 10.1088/1742-6596/1087/6/062046

Downloads

How to Cite

Karaban, V., & Karaban, A. (2024). AI-translated poetry: Ivan Franko’s poems in GPT-3.5-driven machine and human-produced translations. Forum for Linguistic Studies, 6(1). https://doi.org/10.59400/fls.v6i1.1994

Issue

Vol. 6 , Iss. 1 (March 2024)

Article Type

Article

License

Copyright © 2024 Trinh Thi Ha, Nguyen Thi Thu Hien, Nguyen Thi An, Nguyen Thi Mai Huong

Creative Commons License

This is an open access article under the Creative Commons Attribution 4.0 International License.