Accurate depth map estimation is widely used in many computer vision applications including object detection, human pose analysis, and semantic scene recognition. Many depth acquisition systems have been developed. A common issue among those system is the limited resolution of the acquired depth map. In this project, we propose a simple and effective technique to enhance the resolution using the idea of  "residual interpolation".



Intensity Guided Depth Upsampling by Residual Interpolation

Yosuke Konno, Yusuke Monno, Daisuke Kiku, Masayuki Tanaka, and Masatoshi Okutomi,

JSME/RMD International Conference on Advanced Mechatronics (ICAM), pp.1-2, Dec. 2015.

The MATLAB code of this method is available for download.

Depth Map Upsampling by Self-Guided Residual Interpolation

Yosuke Konno, Masayuki Tanaka, Masatoshi Okutomi, Yukiko Yanagawa, Koichi Kinoshita, and Masato Kawade,

IEEE the 23rd International Conference on Pattern Recognition (ICPR), pp. 1395-1400, Dec. 2016.

The MATLAB code of the proposed method is available for download.

      This code is executable only on the 32-bit MATLAB environment.

Residual interpolation

Residual interpolation performs upsampling in the residual domain, where the residual is defined as the difference between the tentatively estimated high-resolution image and the input low-resolution image intensity. Residual interpolation adds the interpolated residual to the tentative estimate to enhance the upsampling result.



Pixel index

(a) input low resolution depth map

(b) tentatively upsampled depth map

(c) 1D sectional view of (a) and (b)

Pixel index

(d) gradient map of input depth map

(e) gradient map of residual map

(f) 1D sectional view of (d) and (e)

Pixel index

(g) residual between (a) and (b)

(h) 1Dsectional view of (g)

In general, the interpolation accuracy depends heavily on the input image smoothness. This is a common property for any kind of interpolation algorithms. In other words, the low gradient energy of the input image makes interpolation easy. That observation implies that one can improve the interpolation performance further if the input image could be transformed into the domain with the lower gradient energy. Residual interpolation has been proposed based on this idea.

Copyright (C) Okutomi-Tanaka Lab. All Rights Reserved.