WE propose a novel depth upsampling method by residual interpolation (RI) that uses both a low-resolution depth map and a high-resolution intensity image. Our method is an application of the RI to depth upsampling, where the upsampling is performed in a residual domain following its success in the field of image demosaicking. Experimental results demonstrate that our method preserves the sharpness of depth discontinuities and outperforms existing well-known methods such as guided filtering.
Fig. 1 The overall processing pipeline of Residual Interpolation.
This method is motivated from the fact that the residual map becomes flatly smooth when there is a correlation between the intensity guide image and the depth map, and that the smooth condition of the residual map generally improves the performance of the interpolation. Therefore, It performs upsampling operation again on the residual domain, after the tentative upsampling.
Fig. 2 The visual comparison of x8 upsampling on the Middlebury dataset 
Table 1 Qualitative Middlebury evaluation measured as Root Mean Squared Error (RMSE).