1. Abstract
2. Proposed method overview
3. Download materials
4. Publication

Polarization information of light in a scene is valuable for various image processing and computer vision tasks. A division-of-focal-plane polarimeter is a promising approach to capture the polarization images of different orientations in one shot, while it requires color-polarization demosaicking. In this paper, we propose a two-step color-polarization demosaicking network~(TCPDNet), which consists of two sub-tasks of color demosaicking and polarization demosaicking. We also introduce a reconstruction loss in the YCbCr color space to improve the performance of TCPDNet. Experimental comparisons demonstrate that TCPDNet outperforms existing methods in terms of the image quality of polarization images and the accuracy of Stokes parameters.

In this work, we propose a two-step color-polarization demosaicking network (TCPDNet). The network architecture of TCPDNet is inspired by Morimatsu's interpolation-based CPFA demosaicking method. Our method consists of two well-studied sub-tasks: color demosaicking and polarization demosaicking. In each sub-task, demosaicking network parameters are shared across different kinds of input mosaicked data. We consider that even if the input mosaicked data differ in polarization orientations (for the color demosaicking task) or color channels (for the polarization demosaicking task), their inter-channel correlations should be the same. For example, the color demosaicking networks for 0° of Bayer CFA mosaicked data and 90° of Bayer CFA mosaicked data share the same parameters because their RGB correlations should be the same regardless of the polarization orientations. We further improve the performance of TCPDNet by introducing a reconstruction loss in the YCbCr color space. Using the loss function in the YCbCr color space, we expect the demosaicking networks to learn the inter-channel correlations effectively. Experimental results show that our TCPDNet outperforms existing methods by a large margin both quantitatively and qualitatively on Tokyo Tech dataset and CPDNet dataset.

• Paper
• Supplementary material
• Code