Objective Evaluation of Image Decomposition Algorithms for Depth Map Upsampling

다국어 초록 (Multilingual Abstract)

Depth map upsampling plays an essential role in various three-dimensional (3D) image and video applications such as multiview rendering and 3D scene modeling. Most of existing depth map upsampling methods have suggested to use a color image as a guide. Recently, in our previous work, the use of a structure component obtained from image decomposition rather than the color image has proven to be very powerful in the task of depth map upsampling. In this paper, to determine how image decomposition algorithms can be best used for depth map upsampling, we conducted a comprehensive comparative study.
More precisely, the purpose of this study is to present an “objective” evaluation of recent promising image decomposition methods in terms of the performance of depth map upsampling. This is, to the best of our knowledge, the frst experimental comparative demonstration on the performance of depth map upsampling enhanced with several diferent image decomposition models. We investigated eight diferent promising recent image decomposition approaches under the same experimental setup. From our quantitative comparison, we can obtain novel and valuable insights into the image decomposition-based depth map upsampling: (1) the best image decomposition solution for depth map upsampling depends on the scaling factor of upsampling, (2) the guided flter-based image decomposition method gives rise to the best performance for lower scaling factors, whereas the tree flter-based image decomposition method leads to the best upsampling performance for higher scaling factors, and (3) the performance of image decomposition-based depth upsampling is not sensitive to image features.
We believe that this comprehensive comparative study serves as a reference point and guide for developers and practitioners in choosing an appropriate image decomposition technique adopted for building depth map upsampling systems.

참고문헌 (Reference)

1 Bao L, "Tree filtering : efficient structure-preserving smoothing with a minimum spanning tree" 23 (23): 555-569, 2014

2 Karacan L, "Structure-preserving image smoothing via region covariances" 32 (32): 176-, 2013

3 Xu L, "Structure extraction from texture via relative total variation" 31 (31): 139-, 2012

4 Zhang Q, "Rolling guidance filter" Springer 8691 : 815-830, 2014

5 Jung SW, "Learning-based filter selection scheme for depth image super resolution" 24 (24): 1641-1650, 2014

6 Kopf J, "Joint bilateral upsampling" 26 (26): 96-, 2007

7 Xu L, "Image smoothing via L0 gradient minimization" 30 (30): 174-, 2011

8 He K, "Guided image filtering" 35 (35): 1397-1409, 2013

9 Buades A, "Fast cartoon +texture image filters" 19 (19): 1978-1986, 2010

10 Subr K, "Edge-preserving multiscale image decomposition based on local extrema" 28 (28): 147-, 2009

11 Min D, "Depth video enhancement based on weighted mode filtering" 21 (21): 1176-1190, 2012

12 Jung C, "Depth map upsampling with image decomposition" 51 (51): 1782-1784, 2015

13 Wu X, "Depth imagebased hand tracking in complex scene" 126 (126): 2757-2763, 2015

14 Zhao J, "An efficient depth modeling mode decision algorithm for 3D-HEVC depth map coding" 127 (127): 12048-12055, 2016

15 Scharstein D, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms" 47 (47): 7-42, 2002

16 Muller K, "3-D video representation using depth maps" 99 (99): 643-656, 2011