Pro-Cam SSfM: Projector-Camera System
for Structure and Spectral Reflectance from Motion

Chunyu Li, Yusuke Monno, Hironori Hidaka, and Masatoshi Okutomi
Department of Systems and Control Engineering, School of Engineering, Tokyo Institute of Technology
International Conference on Computer Vision (ICCV2019)

Abstract

In this work, we propose a novel projector-camera system for practical and low-cost acquisition of a dense object 3D model with the spectral reflectance property. In our system, we use a standard RGB camera and leverage an off-the-shelf projector as active illumination for both the 3D reconstruction and the spectral reflectance estimation. We first reconstruct the 3D points while estimating the poses of the camera and the projector, which are alternately moved around the object, by combining multi-view structured light and structure-from-motion (SfM) techniques. We then exploit the projector for multispectral imaging and estimate the spectral reflectance of each 3D point based on a novel spectral reflectance estimation model considering the geometric relationship between the reconstructed 3D points and the estimated projector positions. Experimental results on several real objects demonstrate that our system can precisely acquire a dense 3D model with the full spectral reflectance property using off-the-shelf devices.

ICCV 2019 Overview
Overview of Pro-Cam SSfM.

Results

The following 3D models show the sRGB results converted from the estimated spectral reflectances.

A clay sculpture
A stuffed toy
A figurine

Publication

Pro-Cam SSfM: Projector-Camera System for Structure and Spectral Reflectance from Motion

Authers: Chunyu Li, Yusuke Monno, Hironori Hidaka, and Masatoshi Okutomi
Proceedings of IEEE International Conference on Computer Vision (ICCV2019), October, 2019

@article{li2019pro,
  title={{Pro-Cam SSfM}: Projector-Camera System for Structure and Spectral Reflectance from Motion},
  author={Li, Chunyu and Monno, Yusuke and Hidaka, Hironori and Okutomi, Masatoshi},
  journal={Proc. of IEEE Int. Conf. on Computer Vision (ICCV)},
  pages={2414--2423},
  year={2019}
}
Pro-Cam SSfM:汎用プロジェクタとカメラを用いた分光3D計測 (Japanese)

Authers: 李 淳雨, 紋野 雄介, 日高 宏紀, and 奥富 正敏
第26回画像センシングシンポジウム(SSII2020), June, 2020


This paper recieved SSII2020最優秀学術賞
Pro-Cam SSfM: Projector-Camera System for Structure and Spectral Reflectance from Motion (Journal)

Authers: Chunyu Li, Yusuke Monno, and Masatoshi Okutomi
The Visula Computer, Feb., 2022

@article{li2022pro,
  title={{Pro-Cam SSfM}: Projector-Camera System for Structure and Spectral Reflectance from Motion},
  author={Li, Chunyu and Monno, Yusuke and Okutomi, Masatoshi},
  journal={The Visula Computer (TVCJ)},
  year={2022}
}