Stereoscopic Image Generation

Introduction

Problem Statement

Stereoscopic panoramas[1] can give 3D illusion only for horizontal view direction but not for vertical view direction.

Bad tilted stereo image — Fig 1. Disparities in tilt stereo image from stereo panorama are not correct. We can hardly feel the depth from it.

Good tilted stereo image — Fig 2. Expected result: Disparities remain correct for any view directions.

Why does the general method fail?

The capture image of Peleg-based methods are light rays which are othogonal to a viewing circles. Light rays from a left and right points on the viewing circle appears as parallel slits on the left and right camera. This results in horizontal disparities which create depth perception from stereo image. See Fig 3.

If one try to create stereo image with vertical view direction, the slits are no longer parallel to neighbor slits (see Fig 4). This makes non-horizontal disparities. Obesers may perceive 3D illusion with dificulty or may not perceive it at all.

Parallel slits — Fig 3. Horizontal view direction results in parallel slits.

Non-parallel slits — Fig 4. Vertial view direction results in non-parallel slits.

Goal

Capture real image data and generate stereo images for any arbitrary pan-tilt view direction.

Methodology

Image Capturing

From the analysis, the data needed are the light rays which pass through points on the viewing circle as in Fig 5. These rays can be captured using an omnidirectional camera with a rotation rig. Fig 6 show the tools we used to manually capture the data.

Required light rays — Fig 5. Light rays needed for generate stereo images for arbitrary pan-tilt view direction

Manually image capturing tool — Fig 6. Tool for manually capture image data.

We also capture data using a computer controlled rotation rig. Note that using tool in Fig 6 need extra calibration since the accurate camera poses are vitual for the image generation step. The rotation rig in Fig 7 (ADVAS-PT1) gives more accurate camera pose since its pan and tilt angle can be controlled by a computer. Moreover, the omnidirectional camera (Theta S) can be remotely controlled via wireless network. We create a software to automatically capture the image data fore without human intervention as shown in Fig 8.

Automatic image capturing tool — Fig 7. Tool for manually capture image data.

Automatic image capturing software — Fig 8. Tool for manually capture image data.

Image Generation

When all required image data is capture, generating stereo image for any arbitrary view direction is simply choosing the nearest light ray for each pixel in the stereo image from the capture data. For more details, please refer to out publications.

Experimental Results

Captured Scene

Currently we have 23 scenes both indoor and out door. Note that most of scene are captured manually but the data is good enough for the project.

Stereo Image Generation

Conclusion

We proposed a novel method to capture real images and generate stereoscopic image for any arbitrary viwe direction. Our method can generate stereo image for any arbitrary view direction.

Publications

Sakharin Buachan, Shigeki Sugimoto and Masatoshi Okutomi, “Pan-Tilt Stereoscopic Image Generation from Omnidirectional Image Data Collection,” 第18回画像の認識・理解シンポジウム(MIRU2015), 2015, pp.SS1-21-1-2

BUACHAN Sakharin, 杉本茂樹, 奥富正敏, “ユーザインタラクティブな360×180 度の実環境立体視画像生成システム,” 第22回画像センシングシンポジウム（SSII2016, 2016, pp.SO1-DS2-02-1-1.

Sakharin Buachan, Shigeki Sugimoto and Masatoshi Okutomi, “Interactive Stereoscopic Image Generation from Image Data Collection for Experiencing 360×180 Degree Real Environments,” IIEEJ Transactions on Image Electronics and Visual Computing, Vol.4, No.2, 2016, pp.156-1606.

References

[1] S. Peleg, Y. Pritch, and M. Ben-Ezra, “Cameras for stereo panoramic imaging,” in Computer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on, vol. 1, 2000, pp. 208–214.