A Comparative Analysis of Koreans’ English Writings and Google Translations Using Coh-Metrix 3.0

다국어 초록 (Multilingual Abstract)

The purpose of the present study is to identify the corpus-based differences between Koreans’ English writings and their corresponding Google translations. For this purpose, the present study utilized Coh-Metrix 3.0 and conducted comparative analyses on two types of writings in terms of 12 benchmarks of text analysis. Coh-Metrix 3.0 provided numeric values for the following selected categories of text analysis: (a) basic counts (i.e., DESSC, DESWC, and DESSL), (b) lexical aspects (i.e., WRDFRQc and LDTTRc), (c) readability (i.e., RDFRE and RDFKGL) (d) syntactic complexity (i.e., SYNLE, SYNNP, and SYNSTRUTa), and (e) cohesion (i.e., CRAFAOa and LSASS1). Each output for 5 categories computed by Coh-Metrix 3.0 was then statistically processed in order to find statistically significant differences. The quantitative findings, given the small sample size associated with lower statistical power and non-normality of some data sets, were interpreted together with results from a robust technique of bootstrapped independent t-tests since the employment of bootstrapping has been empirically justified in the field of applied linguistics (Plonsky 2013, 2014). The overall findings indicated that Google translations tend to produce significantly more words before main verbs and longer sentences compared to human writings. Furthermore, it was also found that Google translations were significantly less readable, but more cohesive. However, there were no significant differences observed in lexical aspects.

참고문헌 (Reference)

1 김정렬, "초중등 영어교육 연계성 문제: 영어교과서의 계열성을 중심으로" 언어교육원 50 (50): 161-184, 2014

2 Ferris, D. R., "Written corrective feedback for individual L2 writers" 22 (22): 307-329, 2013

3 Enkin, E., "Using online translators in the second language classroom : Ideas for advanced-level Spanish" 9 (9): 138-158, 2016

4 Tsai, S-C., "Using Google Translate in EFL drafts : A preliminary investigation" 32 (32): 510-526, 2019

5 Ramati, I., "Uniform multilingualism : A media genealogy of Google translate" 20 (20): 2550-2565, 2018

6 Aryadoust, V., "Understanding the growth of ESL paragraph writing skills and its relationships with linguistic features" 36 (36): 1742-1770, 2016

7 Kaltenbacher, M., "Translation in Context" John Benjamins 221-230, 2000

8 Hampshire, S., "Translation and the internet : Evaluating the quality of free online machine translators" 17 : 197-209, 2010

9 Rodreguez-Castro, M., "To Google translate™ or not? Newcomer Latino communities in the middle" 49 (49): 3-9, 2018

10 Polio, C., "The reliability and validity of automated tools for examining variation in syntactic complexity across genres" 28 (28): 165-188, 2018

11 Lewis-Kraus, G., "The great A.I. awakening"

12 Turovsky, B., "Ten years of Google translate [Blog post]"

13 Plonsky, L., "Study quality in quantitative L2 research(1990–2010) : A methodological synthesis and call for reform" 98 (98): 450-470, 2014

14 Plonsky, L., "Study quality in SLA : An assessment of designs, analyses, and reporting practices in quantitative L2 research" 35 (35): 655-687, 2013

15 Sheppard, F., "Medical writing in English : The problem with Google translate" 40 (40): 565-566, 2011

16 Garcia, I., "Machine translation-assisted language learning : Writing for beginners" 24 (24): 471-487, 2011

17 Costa-jussá, M., "Machine translation in medicine: A quality analysis of statistical machine translation in the medical domain" Advanced Research in Scientific Areas 2012

18 Ducar, C., "Machine translation and the L2 classroom : Pedagogical solutions for making peace with Google Translate" 51 (51): 779-795, 2018

19 Zhuqing Wang, "Machine Translation and Parallel Corpora: The Case of English-to-Chinese Translation of Relative Clauses" Korean Society for Language and Information 22 (22): 175-201, 2018

20 CORREA, M., "Leaving the ‘peer’ out of peer-editing : Online translators as a pedagogical tool in the Spanish as a second language classroom" 7 (7): 1-20, 2014

21 Garcia, I., "Is machine translation ready yet?" 22 (22): 7-21, 2010

22 Bellos, D., "Is That a Fish in Your Ear? Translation and the Meaning of Everything" Faber and Faber 2011

23 Colina, S., "Further evidence for a functionalist approach to translation quality evaluation" 21 (21): 235-264, 2009

24 Groves, M., "Friend or foe? Google translate in language for academic purposes" 37 : 112-121, 2015

25 Castelvecchi, D., "Deep learning boosts Google translate tool"

26 Graesser, A. C., "Computational analyses of multilevel discourse comprehension" 3 (3): 371-398, 2011

27 Ahrenberg, L., "Comparing machine translation and human translation: A case study" 21-28, 2017

28 I. Solnyshkina, M. V., "Comparative Coh-metrix analysis of reading comprehension texts: Unified (Russian) state exam in English vs Cambridge first certificate in English" 7 (7): 65-, 2014

29 김준구, "Coh-Metrix를 활용한 중학교 학업성취도 평가 영어 지문의 코퍼스 언어학적 분석" 한국중등영어교육학회 11 (11): 27-51, 2018

30 McNamara, D. S., "Coh-Metrix: Automated Evaluation of Text and Discourse" Cambridge University Press 2014

31 Graesser, A. C., "Coh-Metrix: Analysis of text on cohesion and language" 36 (36): 193-202, 2004

32 Kapadya, A., "Bringing hope to a refugee family, using Google Translate [Blog post]"

33 Google, "Breaking down the language barrier - six years in"

34 Kirchhoff, K., "Application of statistical machine translation to public health information : A feasibility study" 18 (18): 473-478, 2011

35 McCarthy, P. M., "Analyzing journal abstracts written by Japanese, American, and British scientists using Coh-metrix, and the Gramulator" 30 (30): 141-173, 2011

36 Aiken, M., "An analysis of Google Translate accuracy" 16 (16): 12-34, 2011

37 LaFlair, G. T., "Advancing Quantitative Methods in Second Language Research" Routledge 135-150, 2015

38 Krishnamoorthy, K., "A parametric bootstrap approach for ANOVA with unequal variances : Fixed and random models" 51 (51): 5731-5742, 2007

39 Turner, A. M., "A comparison of human and machine translation of health promotion materials for public health practice : Time, costs, and quality" 20 (20): 523-529, 2014

40 Kroulek, A., "A Translation Showdown: Man vs Machine Translation"

41 전문기, "A Corpus-based analysis of middle school English 1 textbooks with Coh-Metrix" Pan-Korea English Teachers Association 21 (21): 265-292, 2009

42 Zhang, R., "A Coh-metrix study of writings by majors of mechanic engineering in the vocational college" 5 (5): 1929-1934, 2015

43 Ye, D., "A Coh-metrix analysis of language varieties between the journal articles of Chinese and American scientists" 3 (3): 63-70, 2013

44 Jiyoung Lee, "A Coh-Metrix Analysis of Lexical, Syntactic and Discourse Aspects in the Newspaper Articles of Korean and British University Students" Modern English Society 19 (19): 17-26, 2018

45 안병규, "A Coh-Metrix Analysis of Elementary School English Textbooks" 21세기영어영문학회 28 (28): 435-460, 2015