딥러닝 기반 손상된 흑백 얼굴 사진 컬러 복원 = Deep Learning Based Color Restoration of Corrupted Black and White Facial Photos

다국어 초록 (Multilingual Abstract)

In this paper, we propose a method to restore corrupted black and white facial images to color. Previous studies have shown that when coloring damaged black and white photographs, such as old ID photographs, the area around the damaged area is often incorrectly colored. To solve this problem, this paper proposes a method of restoring the damaged area of input photo first and then performing colorization based on the result. The proposed method consists of two steps: BEGAN (Boundary Equivalent Generative Adversarial Networks) model based restoration and CNN (Convolutional Neural Network) based coloring. Our method uses the BEGAN model, which enables a clearer and higher resolution image restoration than the existing methods using the DCGAN (Deep Convolutional Generative Adversarial Networks) model for image restoration, and performs colorization based on the restored black and white image. Finally, we confirmed that the experimental results of various types of facial images and masks can show realistic color restoration results in many cases compared with the previous studies.

국문 초록 (Abstract)

본 논문에서는 손상된 흑백 얼굴 이미지를 컬러로 복원하는 방법을 제안한다. 기존 연구에서는 오래된 증명사진처럼 손상된 흑백 사진에 컬러화 작업을 하면 손상된 영역 주변이 잘못 색칠되는 경우가 있었다. 이와 같은 문제를 해결하기 위해 본 논문에서는 입력받은 사진의 손상된 영역을 먼저 복원한 후 그 결과를 바탕으로 컬러화를 수행하는 방법을 제안한다. 본 논문의 제안 방법은 BEGAN(Boundary Equilibrium Generative Adversarial Networks) 모델 기반 복원과 CNN(Convolutional Neural Network) 기반 컬러화의 두 단계로 구성된다. 제안하는 방법은 이미지 복원을 위해 DCGAN(Deep Convolutional Generative Adversarial Networks) 모델을 사용한 기존 방법들과 달리 좀 더 선명하고 고해상도의 이미지 복원이 가능한 BEGAN 모델을 사용하고, 그 복원된 흑백 이미지를 바탕으로 컬러화 작업을 수행한다. 최종적으로 다양한 유형의 얼굴 이미지와 마스크에 대한 실험 결과를 통해 기존 연구에 비해 많은 경우에 사실적인 컬러 복원 결과를 보여줄 수 있음을 확인하였다.

참고문헌 (Reference)

1 Arjovsky, M., "Wasserstein gan"

2 Radford, A., "Unsupervised representation learning with deep convolutional generative adversarial networks"

3 Welsh, T., "Transferring color to greyscale image" 21 (21): 277-280, 2002

4 Yeh, R. A., "Semantic image inpainting with deep generative models" 5485-5493, 2017

5 Hays, J., "Scene completion using millions of photographs" 26 (26): 4-, 2007

6 Viola, P ., "Rapid object detection using a boosted cascade of simple features" 1 : 1-1, 2001

7 Perez, P., "Poisson linage editing" 22 (22): 313-318, 2003

8 Iizuka, S., "Let there be color!: joint end-to-end learning of global and local image priors for automatic linage colorization with simultaneous classification" 35 (35): 110-, 2016

9 Bengio, Y., "Learning deep architectures for AI" 2 (2): 1-127, 2009

10 Noh, H., "Learning deconvolution network for semantic segmentation" 1520-1528, 2015

11 Huang, G. B., "Labeled faces in the wild: A database for studying face recognition in unconstrained environments" University of Massachusetts 3-, 2007

12 Bertalmio, M., "Image inpainting" 417-424, 2000

13 Goodfellow, I., "Generative adversarial nets" 2672-2680, 2014

14 Sun, T., "Deep learning face representation by joint identification-verification" 1988-1996, 2014

15 Pathak, D., "Context encoders : Feature learning by inpainting" 2536-2544, 2016

16 Levin, A., "Colorization using optimization" 23 (23): 689-694, 2004

17 Berthelot, D., "Began: Boundary equilibrium generative adversarial networks"

18 LeCun, Y., "Backpropagation applied to handwritten zip code recognition" 1 (1): 541-551, 1989

19 Efros, A. A., "'Texture synthesis by non-parametric sampling" 2 : 1033-1038, 1999