Monochrome and Color Polarization Demosaicking Using Edge-Aware Residual Interpolation

Project Summary

One-shot monochrome and color polarization cameras based on microgrid image polarimeter has reveived increasing attention in recent years because of many potential applications using polarization information. In this project, we aim at developing a high-quality demosaicking algorithm for monochrome and color polarization cameras.

Paper Abstract

A division-of-focal-plane or microgrid image polarimeter enables us to acquire a set of polarization images in one shot. Since the polarimeter consists of an image sensor equipped with a monochrome or color polarization filter array (MPFA or CPFA), the demosaicking process to interpolate missing pixel values plays a crucial role in obtaining high-quality polarization images. In this paper, we propose a novel MPFA demosaicking method based on edge-aware residual interpolation (EARI) and also extend it to CPFA demosaicking. The key of EARI is a new edge detector for generating an effective guide image used to interpolate the missing pixel values. We also present a newly constructed full color-polarization image dataset captured using a 3-CCD camera and a rotating polarizer. Using the dataset, we experimentally demonstrate that our EARI-based method outperforms existing methods in MPFA and CPFA demosaicking.

Proposed Method Overview

Fig. 1. The outline of our proposed MPFA demosaicking method based on EARI

Fig. 2. The outline of our proposed CPFA demosaicking framework

Prsentation Video

Download Materials

Paper [PDF]

Supplementary Material [PDF]

Presentation Slides [PDF]

Matlab Code [Link]

Dataset (See below for details) [Download (3.15GB)]
(If you have any trouble to download the dataset, please contact ymonno(at)ok.sc.e.titech.ac.jp)

TokyoTech Color-Polarization Dataset

This dataset contains full 12-channel color-polarization images (four RGB images captured with polarizer angles of 0, 45, 90, and 135 degrees) with 40 scenes. Each 12-channel data was captured by rotating the linear polarizer (SIGMAKOKI SPF-50C-32 linear polarizer attached to PH-50-ARS rotating polarizer mount) placed in front of a 3-CCD camera (JAI CV-M9GE 3-CCD camera). For each polarizer orientation, we captured 1,000 images and averaged them to make the ground-truth image with reduced noise. We used the green-channel images of our color-polarization dataset for the evaluation of monochrome polarization demosaicking.

Fig. 3. Our full 12-channel color-polarization image dataset (40 scenes)

Publication

Monochrome and Color Polarization Demosaicking Using Edge-Aware Residual Interpolation [PDF]: Miki Morimatsu, Yusuke Monno, Masayuki Tanaka, and Masatoshi Okutomi; IEEE International Conference on Image Processing (ICIP), pp.2571-2575, October, 2020.