다중 스케일 기반 Self-Attention 생성적 적대 신경망을 이용한 비지도 학습 애니메이션 얼굴 변환

다국어 초록 (Multilingual Abstract)

In this paper, we propose a novel generator architecture to translate a selfie photo into an anime face image in an unsupervised way. The purpose is to translate selfie images to anime face images that preserves the traits and shapes of facial parts (e.g., eyes and noses) of an input source image with reference anime style. Recent face translation works often fails to preserve the characteristics of facial parts of selfie photos while transferring to anime images. To cope with this, the proposed method develop new adversarial generative network (GAN) architecture composed of (1) simple cycle contents loss, (2) multi-scale assisted self-attention, and (3) adaptive feature fusion. The goal of using cycle contents loss is to make the GAN preserves a wide range of selfie image contents, which includes hair shape, facial expression and shape of face. By comparing feature maps in the process of image translation, cycle contents loss prevents the feature maps being overly summarized by encoder and decoder. This prevents hair styles and facial expressions of the selfie image being oversimplified into a plain expression of anime images such as straight hair, tiny nose, and mouth. In addition, multi-scale assisted self-attention complements the existing attention by using its various scales of self-attentions. This multi-scale assistance provides additional spatial relationship of feature maps that cannot be perceived by single-scale attention. It allows model to gain additional facial characteristics that can be useful to generate an anime image containing hair style and facial expression from a selfie photo. Adaptive feature fusion helps model to understand which self-attention map is important among the multiple self-attention maps generated by the multi-scaled self-attention module. It makes GAN model to understand which scales of self-attention map is critical for image translation and allow the optimal elements without being disturbed from too much information. Our extensive and comparative experiments on selfie2anime and photo2anime datasets have been performed to demonstrate the effectiveness of our method over other state-of-the-art methods.

목차 (Table of Contents)

• 제1장 서 론 1
• 1.1 연구 배경 및 목적 1
• 1.2 논문 구성 7
• 제2장 관련연구 8
• 2.1 생성적 적대 신경망 8
• 2.2 생성적 적대 신경망 기반 이미지간 변환 연구 동향 10
• 2.3 어텐션 기반 생성적 적대 신경망 연구 동향 12
• 2.3.1 CycleGAN 12
• 2.3.2 UGATIT 13
• 2.3.3 Self-Attention의 이미지 처리 14
• 2.4 사람-애니메이션 얼굴 이미지 변환 16
• 2.5 제안 방법의 독창성 17
• 제3장 제안 방법 18
• 3.1 다중 스케일 Attention 모듈 19
• 3.2 적응형 특징 맵 합성 20
• 3.3 손실 함수 22
• 3.3.1 적대적 손실 함수 22
• 3.3.2 Cycle Consistency 손실 함수 23
• 3.3.3 주기 특성 손실 함수 24
• 3.3.4 보조 식별자 손실 함수 25
• 3.3.5 최종 손실 함수 26
• 제4장 실험 결과 및 분석 28
• 4.1 실험환경 28
• 4.2 데이터베이스 29
• 4.2.1 Selfie2anime 데이터베이스 30
• 4.2.2 Photo2anime 데이터베이스 31
• 4.3 평가 지표 32
• 4.4 실험 결과 33
• 제5장 결론 43
• 참고문헌 45
• ABSTRACT 50