Virtual View Generation by a New Hole Filling Algorithm

다국어 초록 (Multilingual Abstract)

In this paper, performance improved hole-filling algorithm which includes the boundary noise removing pre-process that can be used for an arbitrary virtual view synthesis has been proposed. Boundary noise occurs due to the boundary mismatch between depth and texture images during the 3D warping process and it usually causes unusual defects in a generated virtual view. Common-hole is impossible to recover by using only a given original view as a reference and most of the conventional algorithms generate unnatural views that include constrained parts of the texture. To remove the boundary noise, we first find occlusion regions and expand these regions to the common-hole region in the synthesized view. Then, we fill the common-hole using the spiral weighted average algorithm and the gradient searching algorithm. The spiral weighted average algorithm keeps the boundary of each object well by using depth information and the gradient searching algorithm preserves the details. We tried to combine strong points of both the spiral weighted average algorithm and the gradient searching algorithm. We also tried to reduce the flickering defect that exists around the filled common-hole region by using a probability mask. The experimental results show that the proposed algorithm performs much better than the conventional algorithms.

목차 (Table of Contents)

• Abstract
• 1. Introduction
• 2. 3D Warping and Problems
• 3. Proposed Algorithm
• 4. Experimental Results
• 5. Conclusion
• References

참고문헌 (Reference)

1 김용진, "영상 인페인팅을 이용한 고품질의 가려짐 영역 보간 방법" 방송공학회 논문지 15 (1) : 3 ~ 13, 2010

2 Y. Mori, "View generation with 3D warping using depth information for FTV" ELSEVIER, Signal Processing: Image Communication 24 (1) : 65 ~ 72, 2009

3 Ohm, J. R., "Stereo/Multi-view video encoding using the MPEG family of standards" T Proc. of Electronic Imaging, Invited Paper, 1999

4 G. M. Um, "Standardization trends of 3D video coding in MPEG" ETRI, Electronics and Telecommunications Trends 24 (3) : 61 ~ 68, 2009

5 A. Criminisi, "Region Filling and Object Removal by Exemplar-Based Image Inpainting" IEEE Trans. Image Processing 13 (9) : 1200 ~ 1212, 2004

6 "Introduction to 3D video" ISO/IEC JTC1/SC29/WG11, 2008

7 "Implementation of Hole Filling Methods for VSRS 3.5.alpha" ISO/IEC JTC1/SC29/WG11, 2011

8 Ballester, C.,Bertalmio, M.,Caselles, V.,Sapiro, G., "Image Inpainting" Addison-Wesley Proc. of 27th Conference Computer Graphics and Interactive Techniques (ACM SIGGRAPH 2000) : 417 ~ 424, 2000

9 S. Zinger, "Free-viewpoint depth image based rendering" J. Vis. Commun. Image Representation 21 (5) : 533 ~ 541, 2010

10 C. Fehn, "Depth-image-based Rendering (DIBR), Compression and Transmission for a New Approach on 3D TV" Proc. SPIE 5291 : 93 ~ 104, 2004

11 L. Azzari, "Comparative analysis of occlusion-filling techniques in depth image-based rendering for 3D videos" Proc. ACM MoViD : 57 ~ 62, 2010

12 "Boundary noise removal and hole filling for VSRS 3.5 alpha" ISO/IEC JTC1/SC29/WG11, 2011

13 A. Telea, "An Image Inpainting Technique Based on the Fast Marching Method" J. Graphics Tools 9 (1) : 25 ~ 36, 2003

14 I. Daribo, "A novel inpainting-based layered depth video for 3DTV" IEEE Trans. Broadcasting 57 (2) : 533 ~ 541, 2011

15 M. S. Ko, "A new commonhole filling algorithm for virtual view synthesis with a probability mask" 11th IEEE IVMSP Workshop: 3D Image/Video Technologies and Applications, 2013