Spherical Stereoscopic Panoramas

This week I visited the Blender Institute and decided to wrap up the multiview project. But since I had an Oculus DK2 with me I decided to patch multiview to support Virtual Reality gadgets.

Gooseberry Benchmark viewed with an Oculus DK2

Gooseberry Benchmark viewed with an Oculus DK2

There is something tricky about them. You can’t just render a pair of panoramas and expect them to work. The image would work great for the virtual objects in front of you, but it would have the stereo eyes swapped when you look at behind you.

How to solve that? The technique is the same one as presented in the 3D Fulldome Teaser. We start by determining an interocular distance and a convergence distance based on the stereo depth we want to convey. From there Cycles will rotate a ‘virtual’ stereo camera pair for each pixel to be rendered, so that both cameras’ rays converge at the specified distance. The zero parallax will be experienced at the convergence distance.

Oculus barrel correction screen shader applied to a view inside the panorama

Oculus barrel correction screen shader applied to a view inside the panorama

This may sound complicated, but it’s all done under the hood. If you want to read more about this technique I recommend this paper from Paul Bourke on Synthetic stereoscopic panoramic images. The paper is from 2006 so there is nothing new under the Sun.

If you have an Oculus DK2 or similar device, you can grab the final image below to play with. I used Whirligig to visualize the stereo panorama, but there are other alternatives out there.

Gooseberry Benchmark Panorama

Top-Bottom Spherical Stereo Equirectangular Panorama – click to save the original image

This image was generated with a spin off branch of multiview named Multiview Spherical Stereo. I’m still looking for a industry standard name for this method – “Omnidirectional Stereo” is a strong contender.

I would also like to remark the relevance of Open projects such as Gooseberry. The always warm-welcoming Gooseberry team just released their benchmark file, which I ended up using for those tests. To be able to get a production quality shot and run whatever multi-vr-pano-full-thing you may think of is priceless.

Builds

If you want to try to render your own Spherical Stereo Panoramas, I built the patch for the three main platforms.

* Don’t get frustrated if the links are dead. As soon as this feature is officially supported by Blender I will remove them. So if that’s the case, get a new Blender.

How to render in three steps

  1. Enable ‘Views’ in the Render Layer panel
  2. Change camera to panorama
  3. Panorama type to Equirectangular

And leave ‘Spherical Stereo’ marked (it’s on by default at the moment).

Last and perhaps least is the small demo video above. The experience of seeing a 3D set doesn’t translate well for the video. But the overall impression from the Gooseberry team was super positive.

Also, this particular feature was the exact reason I was moved towards implementing multiview in Blender. All I wanted was to be able to render stereo content for fulldomes with Blender. In order to do that, I had to design a proper 3D stereoscopic pipeline for it.

What started as a personal project in 2013 ended up being embraced by the Blender Foundation in 2014, which supported me for a 2-month work period at the Blender Institute via the Development Fund. And now in 2015, so close to the Multiview completion, we finally get the icing on the cake.

No, wait … the cake is a lie!

Links

  • Multiview Spherical Stereo branch [link] *
  • Gooseberry Production Benchmark File [link]
  • Support the Gooseberry project by signing up in the Blender Cloud [link]
  • Support further Blender Development by joining the Development Fund [link]

* If the branch doesn’t exist anymore, it means that the work was merged into master.

What is next?

Multiview is planned to be merged in master very soon, in time for Blender 2.75. The Spherical Panorama was not intended as one of the original features, but if we can review it in time it will go there as well.

I would like to investigate if we may need other methods for this techniques. For instance, this convergence technique is the equivalent of ‘Toe-In’ for perspective panorama. We could support ‘Parallel’ convergence as well, but ‘Off-Axis’ seems to not fit here. It would be interesting to test the final image in different devices.

If you manage to test it yourself, do post your impressions in the comment section!

19 comments 24,941 Views
  1. I’m definitely very interested in this. I’m planning to start working on a VR short film very soon, do you have an estimation of time for this getting into master? it would be awesome if we could use this feature officially.

    Thanks again for your amazing work.

  2. thanx,help keep me updated…thanx

  3. Awesome! I will try this out when I have a free moment.

  4. i guess that only specific scenes can be done well. the more objects you put in difrent depth areas the more stereoscopic errors will occure. Yhats why i think the shown picture 3d 360 has only a few objects. when comes to3d 360 movies… the main problem would be the resolution and bandwith needed . Consumer techology is just not yet there maybe in 5-7 years

  5. How do I render a top/bottom? I have changed the Views Format settings in the render tab, but it is still outputting anaglyph. I want to see it on my oculus!

    This is an amazing feature, and one I am glad to see! Go Blender!

  6. Dalai, thank you for your perseverance on getting MultiView to the trunk!
    Its HUGE!
    I now want to render environments for Google cardboard (if only because its a generic format, and will follow this closely!
    Craig Luce

  7. this is awesome! i’m kind of sad that discussion on this has gone silent through. @zach, setting the stereo mode in the render output panel seems to work for me, however for some reason it has L and R swapped (as in what blender things is the left camera is in in fact the right, and vice versa). I was correcting for it by scaling the camera by -1, but that also flips your image. i think i’m going to try using the ‘individual’ output option and just making my over-under in after effects.

  8. @steve, can you please report this in Blender’s bug tracker?
    http://developer.blender.org

    As for further discussions in this particular topic, check the patch:
    https://developer.blender.org/D1223

    For generic documentation on multiview (not spherical stereo) check:
    http://www.blender.org/manual/render/workflows/multiview.html

  9. For updated builds (on top of 2.75a) please visit the official patch page (https://developer.blender.org/D1223).

  10. @DALAI, thank you for the updates!! i’m super happy to see this is still progressing! we’ve got a number of vr projects coming through our shop, and this is an awesome tool! i’ll look into reporting the left/right swap on the bug tracker (looks like its still happening in the latest build) thanks again for all this!!

  11. Downloaded your build and used it in a scene I made in 2.75a.
    It works perfectly with the Oculus. Full 360 without any ‘oddness’ when not looking straight.

    I hope this becomes a permanent feature in the next official Blender build.

    THANK YOU THANK YOUI!

  12. @STEVE: > “I’ll look into reporting the left/right swap on the bug tracker (looks like its still happening in the latest build) thanks again for all this!!”

    This was fixed in the latest bug (as well as other bump map related issues).

    You can get the latest build links in my webpage: http://www.dalaifelinto.com/?p=1009

    @NOVAK: > “I hope this becomes a permanent feature in the next official Blender build.”

    Cycles will soon get a Lens Shader Node system, which is supposed to make spherical-stereo renders possible without any changes in the code. So spherical-stereo is official on hold at the moment. That said I will try to keep it up to date to Blender releases, so it can be used as comparison to the upcoming Lens Shader Node solution.

    > “THANK YOU THANK YOUI!”
    You’re welcome, you’re welcome, you’re welcome 😉

    @ALL: I’m glad you are using it, share some of your content with you can, I would love to see it.

  13. Could you do the same thing with 3ds Max?

  14. @DALAI, great to hear about the resolved left/right bug!

    Also, we used this solution to render the stereo visuals for a VR project for the Los Angeles Philharmonic:

    http://www.laphil.com/vanbeethoven

    It wasn’t overly complex in that it is mostly particles with a simple emission shader (and a bit of volumetrics), but being able to get the stereoscopic depth comped into a 360 space was really rewarding. So THANK YOU!!!

  15. Hello!

    I’m back.
    DALAI….please restore your link!

    Latest build (2.7b) isnt working. When you look behind, the flipping is again an issue:(

    Maybe my settings are wrong, but I dont think so. Please help

  16. Hello Dalai, great addition to Blender, thanks!

    I’ve noticed that the “convergence plane” setting is only considered when using the “Toe-in” method (but not when using “Off axis”).
    Is this a known limitation, or a bug?

  17. The image in the article is Bottom-Top.

  18. I’ve been thinking of finding a way to create 1px wide stripes for stereo 360 using a script and you’ve already done it? THANK YOU!!!

  19. I assume this is on 2.78 now using lenses as your builds are no longer available.
    Any way you could make them available again?

  1. Leave a Reply

    Your email address will not be published. Required fields are marked *

     

share this story on