Our Prototype RGB-NIR Imaging System Overview
Abstract
In recent years, many applications using a set of RGB and near-infrared (NIR) images, also called an RGB-NIR image, have been proposed. However, RGB-NIR imaging, i.e., simultaneous acquisition of RGB and NIR images, is still a laborious task because existing acquisition systems typically require two sensors or shots. In contrast, single-sensor RGB-NIR imaging using an RGB-NIR sensor, which is composed of a mosaic of RGB and NIR pixels, provides a practical and low-cost way of one-shot RGB-NIR image acquisition. In this paper, we investigate high-quality system designs for single-sensor RGBNIR imaging. We first present a system evaluation framework using a new hyperspectral image dataset we constructed. Different from existing work, our framework takes both the RGB-NIR sensor characteristics and the RGB-NIR imaging pipeline into account. Based on the evaluation framework, we then design each imaging factor that affects the RGB-NIR imaging quality and propose the best-performed system design. We finally present the configuration of our developed prototype RGB-NIR camera, which was implemented based on the best system design, and demonstrate several potential applications using the prototype.
Dataset and Source Code
Example RGB-NIR Images
Publications
Journal Paper
- Single-Sensor RGB-NIR Imaging: High-Quality System Design and Prototype Implementation [PDF]
Yusuke Monno, Hayato Teranaka, Kazunori Yoshizaki, Masayuki Tanaka and Masatoshi Okutomi
IEEE Sensors Journal, vol.19, no.2, pp.497-507, January, 2019.
Conference Papers
- Single-Sensor RGB and NIR Image Acquisition: Toward Optimal Performance by Taking Account of CFA Pattern, Demosaicking, and Color Correction [PDF]
Hayato Teranaka, Yusuke Monno, Masayuki Tanaka and Masatoshi Okutomi
IS&T Electronic Imaging (EI2016), February, 2016. - N-to-sRGB Mapping for Single-Sensor Multispectral Imaging [Project]
Yusuke Monno, Masayuki Tanaka and Masatoshi Okutomi
IEEE ICCV Workshop: 5th Color and Photometry in Computer Vision Workshop (CPCV2015), pp.66-73, December, 2015. - Simultaneous Capturing of RGB and Additional Band Images Using Hybrid Color Filter Array [Project]
Daisuke Kiku, Yusuke Monno, Masayuki Tanaka and Masatoshi Okutomi
IS&T/SPIE Electronic Imaging (EI2014), Digital Photography X (Proc. SPIE 9023), pp.90230V-1-9, February, 2014.
Demonstration
- A Real-Time RGB-NIR Imaging System Using a Single Image Sensor [Abstract], [Poster]
Kazunori Yoshizaki, Munenori Fukunishi, Yasuhiro Komiya, Yusuke Monno, Masayuki Tanaka, Masatoshi Okutomi and Steven Lansel
IEEE Conference on Computer Vision and Pattern Recognition (CVPR2016) (Demonstration), June, 2016.
     |