시점 변경을 위한 깊이 정보를 이용한 홀채움 기법 = Depth-guided Hole-filling Algorithm for View Synthesis

다국어 초록 (Multilingual Abstract)

The significance of precise image inpainting for image view synthesis has grown in recent years due to the rapid development of autonomous driving or XR (eXtended Reality) devices or robotics. Numerous studies have been conducted on image inpainting during this time, but they have involved intricate procedures and neglected depth information. This paper introduces a novel and straightforward hole-filling algorithm that leverages local depth information. We generate direction vector map based on the local depth information for each hole pixel and utilize the map for the image inpainting. We perform experiments using two distinct simple inpainting methods and compare them with the method augmented by depth information, evaluating both qualitative and quantitative metrics. The results of these experiments demonstrate improvements in numerical accuracy and plausibility.

참고문헌 (Reference)

1 문희정, "증강현실 기반의 인포메이션 실감체험을 위한 앱 콘텐츠 개발" 한국멀티미디어학회 23 (23): 1428-1434, 2020

2 S. M. Muddala, "Virtual View Synthesis Using Layered Depth Image Generation and Depth-Based Inpainting for Filling Disocclusions and Translucent Disocclusions" 38 : 351-366, 2016

3 S. M. Muddala, "Spatio-Temporal Consistent Depth-Image-Based Rendering Using Layered Depth Image and Inpainting" 1-19, 2016

4 Y. Chen, "Research on Image Inpainting Algorithm of Improved GAN based on Two-Discriminations Networks" 51 : 3460-3474, 2021

5 A. Lugmayr, "Repaint: In-Painting Using Denoising Diffusion Probabilistic Models" 11461-11471, 2022

6 A. Criminisi, "Region Filling and Object Removal by Exemplar-bas ed Image Inpainting" 13 (13): 1200-1212, 2004

7 X. He, "Non-Local and Multi-Scale Mechanisms for Image Inpainting" 21 (21): 3281-, 2021

8 G. Liu, "Image Inpainting for Irregular Holes using Partial Convolutions" 85-100, 2018

9 M. Bertalmio, "Image Inpainting" 417-424, 2000

10 J. Habigt, "Hole-Filling Algorithms for Depth-Image-Based Rendering" Technische Universität München 2020

11 D. Scharstein, "High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth" 31-42, 2014

12 J. Yu, "Generative Image Inpainting with Contextual Attention" 5505-5514, 2018

13 H.Y. Huang, "Fast Hole Filling for View Synthesis in Free Viewpoint Video" 9 (9): 906-, 2020

14 Y. Chen, "FFTI:Image Inpainting Algorithm Via Features Fusion and Two-steps Inpainting" 91 (91): 2023

15 A. Telea, "An Image Inpainting Technique Based on the Fast Marching Method" 9 (9): 23-34, 2004

16 Z. Zhang, "A Flexible New Technique for Camera Calibration" 22 (22): 1330-1334, 2000

17 M. L. Shih, "3D Photography Using Context-Aware Layered Depth Inpainting" 8028-8038, 2020