Stereoscopic panoramas can give 3D illusion only for horizontal view direction but not for vertical view direction.
Why does the general method fail?
The capture image of Peleg-based methods are light rays which are othogonal to a viewing circles. Light rays from a left and right points on the viewing circle appears as parallel slits on the left and right camera. This results in horizontal disparities which create depth perception from stereo image. See Fig 3.
If one try to create stereo image with vertical view direction, the slits are no longer parallel to neighbor slits (see Fig 4). This makes non-horizontal disparities. Obesers may perceive 3D illusion with dificulty or may not perceive it at all.
Capture real image data and generate stereo images for any arbitrary pan-tilt view direction.
From the analysis, the data needed are the light rays which pass through points on the viewing circle as in Fig 5. These rays can be captured using an omnidirectional camera with a rotation rig. Fig 6 show the tools we used to manually capture the data.
We also capture data using a computer controlled rotation rig. Note that using tool in Fig 6 need extra calibration since the accurate camera poses are vitual for the image generation step. The rotation rig in Fig 7 (ADVAS-PT1) gives more accurate camera pose since its pan and tilt angle can be controlled by a computer. Moreover, the omnidirectional camera (Theta S) can be remotely controlled via wireless network. We create a software to automatically capture the image data fore without human intervention as shown in Fig 8.
When all required image data is capture, generating stereo image for any arbitrary view direction is simply choosing the nearest light ray for each pixel in the stereo image from the capture data. For more details, please refer to out publications.
Currently we have 23 scenes both indoor and out door. Note that most of scene are captured manually but the data is good enough for the project.
Stereo Image Generation
We proposed a novel method to capture real images and generate stereoscopic image for any arbitrary viwe direction. Our method can generate stereo image for any arbitrary view direction.
Sakharin Buachan, Shigeki Sugimoto and Masatoshi Okutomi, “Pan-Tilt Stereoscopic Image Generation from Omnidirectional Image Data Collection,” 第18回画像の認識・理解シンポジウム(MIRU2015), 2015, pp.SS1-21-1-2
BUACHAN Sakharin, 杉本茂樹, 奥富正敏, “ユーザインタラクティブな360×180 度の実環境立体視画像生成システム,” 第22回画像センシングシンポジウム（SSII2016, 2016, pp.SO1-DS2-02-1-1.
Sakharin Buachan, Shigeki Sugimoto and Masatoshi Okutomi, “Interactive Stereoscopic Image Generation from Image Data Collection for Experiencing 360×180 Degree Real Environments,” IIEEJ Transactions on Image Electronics and Visual Computing, Vol.4, No.2, 2016, pp.156-1606.
 S. Peleg, Y. Pritch, and M. Ben-Ezra, “Cameras for stereo panoramic imaging,” in Computer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on, vol. 1, 2000, pp. 208–214.