Our Prototype RGB-NIR Imaging System Overview



     In recent years, many applications using a set of RGB and near-infrared (NIR) images, also called an RGB-NIR image, have been proposed. However, RGB-NIR imaging, i.e., simultaneous acquisition of RGB and NIR images, is still a laborious task because existing acquisition systems typically require two sensors or shots. In contrast, single-sensor RGB-NIR imaging using an RGB-NIR sensor, which is composed of a mosaic of RGB and NIR pixels, provides a practical and low-cost way of one-shot RGB-NIR image acquisition. In this paper, we investigate high-quality system designs for single-sensor RGBNIR imaging. We first present a system evaluation framework using a new hyperspectral image dataset we constructed. Different from existing work, our framework takes both the RGB-NIR sensor characteristics and the RGB-NIR imaging pipeline into account. Based on the evaluation framework, we then design each imaging factor that affects the RGB-NIR imaging quality and propose the best-performed system design. We finally present the configuration of our developed prototype RGB-NIR camera, which was implemented based on the best system design, and demonstrate several potential applications using the prototype.

Dataset and Source Code

Example RGB-NIR Images





Journal Paper

  • Single-Sensor RGB-NIR Imaging: High-Quality System Design and Prototype Implementation   [PDF]
    Yusuke Monno, Hayato Teranaka, Kazunori Yoshizaki, Masayuki Tanaka and Masatoshi Okutomi
    IEEE Sensors Journal, vol.19, no.2, pp.497-507, January, 2019.

Conference Papers

  • Single-Sensor RGB and NIR Image Acquisition: Toward Optimal Performance by Taking Account of CFA Pattern, Demosaicking, and Color Correction   [PDF]
    Hayato Teranaka, Yusuke Monno, Masayuki Tanaka and Masatoshi Okutomi
    IS&T Electronic Imaging (EI2016), February, 2016.

  • N-to-sRGB Mapping for Single-Sensor Multispectral Imaging   [Project]
    Yusuke Monno, Masayuki Tanaka and Masatoshi Okutomi
    IEEE ICCV Workshop: 5th Color and Photometry in Computer Vision Workshop (CPCV2015), pp.66-73, December, 2015.

  • Simultaneous Capturing of RGB and Additional Band Images Using Hybrid Color Filter Array   [Project]
    Daisuke Kiku, Yusuke Monno, Masayuki Tanaka and Masatoshi Okutomi
    IS&T/SPIE Electronic Imaging (EI2014), Digital Photography X (Proc. SPIE 9023), pp.90230V-1-9, February, 2014.


  • A Real-Time RGB-NIR Imaging System Using a Single Image Sensor   [Abstract], [Poster]
    Kazunori Yoshizaki, Munenori Fukunishi, Yasuhiro Komiya, Yusuke Monno, Masayuki Tanaka, Masatoshi Okutomi and Steven Lansel
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR2016) (Demonstration), June, 2016.