언어적 특성과 서비스를 고려한 딥러닝 기반 한국어 방언 기계번역 연구 = Deep Learning based Korean Dialect Machine Translation Research Considering Linguistics Features and Service

다국어 초록 (Multilingual Abstract)

Based on the importance of dialect research, preservation, and communication, this paper conducted a study on machine translation of Korean dialects for dialect users who may be marginalized. For the dialect data used, AIHUB dialect data distributed based on the highest administrative district was used. We propose a many-to-one dialect machine translation that promotes the efficiency of model distribution and modeling research to improve the performance of the dialect machine translation by applying Copy mechanism. This paper evaluates the performance of the one-to-one model and the many-to-one model as a BLEU score, and analyzes the performance of the many-to-one model in the Korean dialect from a linguistic perspective. The performance improvement of the one-to-one machine translation by applying the methodology proposed in this paper and the significant high performance of the many-to-one machine translation were derived.

국문 초록 (Abstract)

본 논문은 방언 연구, 보존, 의사소통의 중요성을 바탕으로 소외될 수 있는 방언 사용자들을 위한 한국어 방언 기계번역 연구를 진행하였다. 사용한 방언 데이터는 최상위 행정구역을 기반으로 배포된 AIHUB 방언 데이터를 사용하였다. 방언 데이터를 바탕으로 Transformer 기반의 copy mechanism을 적용하여 방언 기계번역기의 성능 향상을 도모하는 모델링 연구와 모델 배포의 효율성을 도모하는 Many-to-one 기반의 방언 기계 번역기를 제안한다. 본 논문은 one-to-one 모델과 many-to-one 모델의 성능을 비교 분석하고 이를 다양한 언어학적 시각으로 분석하였다. 실험 결과 BLEU점수를 기준으로 본 논문이 제안하는 방법론을 적용한 one-to-one 기계번역기의 성능 향상과 many-to-one 기계번역기의 유의미한 성능을 도출하였다.

참고문헌 (Reference)

1 김순임, "독일어·영어·프랑스어의 명사 비교 연구" 인문학연구소 (49) : 41-80, 2016

2 W. Farhan, "Unsupervised dialectal neural machine translation" 57 (57): 102181-, 2020

3 Y. Wan, "Unsupervised Neural Dialect Translation with Commonality and Diversity Modeling" 34 (34): 9130-9137, 2020

4 E. Benmamoun, "The feature structure of functional categories: A comparative study of Arabic dialects" Oxford University Press 2000

5 T. Kudo, "Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing"

6 I. Guellil, "Neural vs statistical translation of algerian arabic dialect written with arabizi and arabic letter" 2017

7 C. Park, "Neural spelling correction: translating incorrect sentences to correct sentences for multimedia" 80 (80): 34591-34608, 2021

8 R. Aharoni, "Massively multilingual neural machine translation"

9 K. P. Scannell, "Machine translation for closely related language pairs" 103-109, 2006

10 K. Park, "Jejueo Datasets for Machine Translation and Speech Synthesis"

11 S. R. Kudugunta, "Investigating multilingual NMT representations at scale"

12 J. Gu, "Incorporating copying mechanism in sequence-to-sequence learning"

13 M. Johnson, "Google’s multilingual neural machine translation system: Enabling zero-shot translation" 5 : 339-351, 2017

14 Korean Language Research Institute, "Establishment of standard language regulation and Korean spelling" Ministry of Education, Republic of Korea 1988

15 W. Salloum, "Elissa: A dialectal to standard Arabic machine translation system" 385-392, 2012

16 S. Lim, "Deep Learning based Korean Dialect Machine Translation Research" 2021

17 J. Devlin, "Bert: Pre-training of Deep Bidirectional Transformers for Language Understanding"

18 A. Vaswani, "Attention is all you need" 5998-6008, 2017

19 C. Park, "Ancient Korean neural machine translation" 8 : 116617-116625, 2020