Future viewport, the design

As outlined in the previous post there are some technical and feature targets we want to achieve. Recapping here:

1) Performance boost for drawing code. Make sure we use the best drawing method always to pass data to the GPU/Support features that are only available on new OpenGL that will enable better performance and code.

2) Node based material definition for viewport – and definition of a new real – time material system used for rendering (GLSL renderer).

3) Compositing. Includes things such as outlines, depth of field, ambient occlusion, HDR, bloom, flares.

4) Support mobile devices (OpenGL ES).

What is the state so far:

* Limited compositing (in viewport_experiments branch). When we say limited we mean that the compositor is not tied up to the interface properly, rather it just applies effects to the whole contents of the framebuffer. What we would want ideally, is to not allow UI indicators, such as wires or bones from affecting compositing. This is not too hard to enforce though and can be done similarly to how the current transparency/Xray system works, by tagging wire objects and adding them to be rendered on top of compositing.

* Some parts of our mesh drawing code use Vertex Buffer Objects in an optimal way, others do but still suffer from performance issues by not doing it right, while others do not use it at all.

How will the soc_2014_viewport_fx branch help achieving the targets?

Soc-2014_viewport_fx is providing a layer that can be used to migrate to newer or mobile versions of OpenGL with less hastle, but also tries to enforce some good rendering practices along the way, such as the requirement in modern versions of OpenGL that everything is rendered through Vertex Buffer Objects. Also it removes GLU from the dependencies (since it uses deprecated OpenGL functionality).

Also it sets in place some initial functionality so things can be drawn using shaders exclusively. This is essential if we move to modern or mobile OpenGL versions at some point.

So it mostly helps with targets 1 and 4, but more work will need to be done after merging to realize those targets fully.

At some point, if we want to support modern or mobile OpenGL, we can’t avoid rewriting a big part of our realtime rendering code. The branch already takes some care of that so the branch should be merged and worked on (merging is the first step really), unless we do not really care about supporting those platforms and features.

My estimation, from personal experiments with manual merging, is that it would take about 2-3 weeks of full time work to bring the branch to master-readiness.

Can we focus on some targets immediately?

Yes we can. Some targets such as node materials or compositing, just assume GLSL support in mesh drawing which is yet to be realized in the branch fully so it’s not really blocking their progress. However, getting the branch in as soon as possible will mean less headaches during the merge.

Viewport usability design

Draw modes

Draw modes are getting a little bit unpredictable as to what they enable and are more tied to a real time material definition limited to specular/diffuse/textured. They are also bound to the texture face data structure which is becoming less relevant since we are slowly moving to a material based approach. Often artists have to tweak a number of material and object options to get the visual feedback they need, which can also be frustrating and it is not apparent to new users either. We need a design which allows artists to easily work on a particular workflow while being able to visualize what they want without extensive guesswork of how to visualize this best. Ideally we want to drop draw modes in favour of…

Workflow modes (model, sculpt, paint, animation, game shader design)

Different workflows require different data, and different visualizations. So we can define ‘workflow modes’, which includes a set of shaders and visualization options authored specifically for the current workflow. For instance, a ‘workbench’ mode in edit mode will have a basic diffuse and specular shader with wireframe display options. For retopology, it would make sense to use more minimal, transparent mesh display, like hidden wire, with depth offsetting to avoid intersection artifacts.

Example image of edit mode display options. Some options exist to aid in specific workflows, but this is not so readily apparent

For material definition or texture painting, users might want the full final result or an unshaded version of it for detail tweaking.

Debugging (logic, rigging, etc)

Drawing can offer visual feedback to make it easier for users to examine problematic areas in their scenes. Examples include order of dependency calculation or color-encoded vertex and face counts, or even debug options available to developers.

Easy to switch from one to another, easy to config or script

Using the workflow system, users should be able to get their display to be more predictable. Each workflow mode can expose settings for the shaders or passes used but we can allow more customization than this. A node interface will allow users to request data from blender and write their own shaders to process and visualize these data in their own way. We will follow the OSL paradigm with a dedicated node that will request data from blender in the form of data attribute inputs connected to the node. The data request system is at the heart of the new data streaming design and this means that materials and custom shaders should be able to request such data. Probably even access to real time compositing will be included, though memory consumption is a concern here, and we need to better define how data will be requested in that case.

Modernize! Assume that users will always want the best, most realistic, etc.

With the capabilities modern real time shading offers, we aim to add a third render engine using OpenGL, (next to internal and cycles) which can leverage the capabilities of modern GPUs and tailored to make real time rendering a real alternative for final rendering in blender. A lot of the components are already there, but we can push it further, with shader implementations optimized especially for real time rendering instead of trying to mimic an off-line renderer.

We want to make sure that our material display is pleasing, so we are exploring more modern rendering methods such as physically based shading (a patch by Clement Foucault using notes from Unreal Engine 4 is already considered for inclusion) and deferred rendering.

Needless to say this will also mean improved preview of materials for blender internal and cycles.