Geometry Nodes Workshop: November 2023

One week after this year’s Blender Conference, the geometry nodes team came together in Amsterdam to discuss many design topics that affect the future of geometry nodes. This post gives a general overview of the topics that were discussed. You can also read all the notes we took during the meetings.

Volumes

One of the main priorities for this workshop was to come to a conclusion for how we want to better integrate volumes into geometry nodes. Working with volumetric data opens up many new opportunities, and while there are already a few volume nodes available, they only offer limited functionality.

We already discussed this topic many times over the past few years, but more recently the topic became more important because we want to use volumes for physics simulations. The main design difficulty comes from the fact that individual volume grids in a volume object are a lot like attributes, but they are also quite different in that they are not attached to the geometry like attributes on a mesh. So a lot of the discussion in the workshop went into the trade-off between extending what a field is so that it can be used for volume grids, compared to introducing a different workflow that is more tailored to volumetric data.

In the end, we decided to introduce a workflow more specific to volume grids. In the future, the same workflow can work with other data types like lists and images. The proposal is described in more detail on devtalk. For reference, this is a proposal that attempts to fit grids into fields.

Gizmos

The goal with gizmos is modifying geometry nodes inputs directly in the viewport. We want node groups to be able to specify gizmos for their inputs in a simple but powerful way.

The large design challenge is the inter-dependence between the transform of a gizmo and the value it controls. Changing the gizmo’s position affects the controlled value, while changing the value affects the gizmo position. Furthermore, we want to support so-called “crazyspace” with gizmos– if the gizmo controls a geometry which is transformed later in the node tree, we want the gizmo to be transformed in the same way so that it stays attached to the geometry in the viewport.

Design work for this started in another workshop earlier this year. It then progressed in a follow up proposal, and the latest version is available in a work-in-progress patch.

During the workshop, we went over the general design implemented in the prototype and tried to use it for some simple projects. it worked well, and we concluded that the general design in the prototype is the right way forward and discussed more UI details.

Baking

Baking is currently only supported in simulation zones. The original plan was to introduce a new “Bake” node in Blender 4.0, but that didn’t make it, mainly because the design needed more work. Instead we added support for baking individual simulations to 4.0, which is preliminary work for adding the new “Bake” node which can also be baked individually.

The general design of the Bake node has been clear for a while. In the workshop, we mainly discussed the possibility of a scene-wide overview over everything that can be baked. That would be important in a production environment where there may be many bakes that depend on each other.

A baking overview and batch editing should be integrated into the outliner editor, and we have a good idea for what data we want to see there and what operations are needed. However, we couldn’t conclusively decide whether it should be a new view mode in the outliner or whether it can be integrated into an existing mode. While the “View Layer” mode has many elements that would also be needed by the baking overview, we also need some new things that don’t necessarily fit into the existing mode. This decision needs some help from the UI team because it depends on the longer term plan for the outliner and maybe on the “editor tabs” proposal (for example, see an initial attempt, but that needs much more discussion).

Subframe Baking

For baking and simulation, it should be possible to calculate and store results multiple times per scene-frame. In earlier discussions we already concluded that we want the ability to control the number of subframes on a per-simulation basis. We also generally want subframes of separate simulations to align. For example, one simulation with 10 steps per frame combined with another with 9 is problematic because objects used by both simulations (e.g. a collider object) have to be evaluated 19 times. When both would use 10 steps, only 10 evaluations are necessary. We want to guide the user to use aligned subframes by having a dropdown that contains 1, 2, 4, 8, ..., Custom instead of just having a subframes integer property.

Additional problems arise when a simulation depends on another simulation with fewer subframes. In this case it can be necessary to evaluate some frames even more than once. While technically possible, it’s unfortunate that this could make performance worse. Unintuitively, the fix would be to increase the number of subframes for the first simulation.

We couldn’t think of a way to avoid the duplicate evaluation without breaking the principle that playback or baking should behave identically when everything is evaluated at once and when all simulations are baked one after another from start to end. We ended up discussing how we could tell the user when frames are evaluated more often than intended.

Menu Switch Node

The general design for this has been worked on previously already in a proposal and later in a workshop last year. Now, there is already a working implementation in a WIP patch.

In this workshop, we confirmed the design and discussed details like the socket shape and how the node interacts with attributes. The conclusion was that menu sockets are enough like other data types that they can just use the existing circle shape for single values and the diamond shape for fields. The color still has to be defined though. We won’t support attributes of this type for the time being, since the design outside of geometry nodes has many implications, especially for I/O. If users need this, they just have to map menu values to integers manually.

The original design for the menu switch node assumed that we would have have dynamic socket types before the node existed, which would allow switching different types with the same menu. We found that the node is already useful enough without dynamic types though, so we removed that dependency. Once we have proper dynamic socket types, the Menu Switch node can be extended.

Dynamic Socket Visibility

Some built-in nodes dynamically hide and show sockets based on node properties. Node groups should be able to do the same, so they can behave more like built-in nodes. Furthermore, the usage status of sockets should also depend on the value of other input sockets and not only on node properties that are not exposed as sockets.

Our discussion started out by going over the different options in this document on the topic. We concluded that the automatic approach to detecting socket usage is the way to go. This way users can’t construct “incorrect” usages that say an input has no effect even when it does. The approach should just work based on existing Switch nodes.

We also found that by default, it would be better to just gray out unused inputs instead of hiding them. Though there are still cases where hiding is the better behavior, for example in cases where built-in nodes do this already.

Asset Menu Path

Currently, the catalog path of an asset determines where it is shown in menus across Blender, like the add node editor’s add menu. This behavior was useful originally because it avoided the need to set up separate hierarchies for the whole asset library just to integrate with menus. However, catalog path just does not work well as menu path in many cases.

We discussed different approaches, and though we didn’t decide on a long-term solution yet, we did find that a single “Menu Level” checkbox per catalog would solve most common cases. For example, the checkbox could be turned off for a Nodes catalog that contains an asset like MyAssetLib/Nodes/Radial Array. Then the Radial Array node group would show directly in the MyAssetLib menu instead of a nested Nodes menu.

Replacement-Based Procedural Modelling

The goal of this topic is to make procedural modelling in geometry nodes significantly more powerful by roughly following the idea in the original proposal. The core idea with replacement-based procedural modelling is different from traditional modeling where users perform many small edits on a potentally large mesh. Instead, users would work on smaller changes locally that are then inserted into the large geometry.

A replacement-based approach makes it easier to work on mesh edits on a small scale first, before applying the edit to a large mesh. It would make building blocks easier to reuse for common operations and can also have significantly better performance, because many edits on a large mesh can be batched together and calculated in parallel.

The proposed Replace Faces node can also be used to build nodes for other common operations like inserting edge loops. It also brings many use cases that currently use the Tissue addon into geometry nodes.

We went over a few examples for how this can work and how the deformation works more specifically. Overall, the feedback was positive.

Modal Node Tools

Blender 4.0 introduces Node Tools to Blender, which are a way to build custom operators for e.g. mesh edit mode with geometry nodes. In later versions, we intend to extend this concept to support modal operators– operators that are interactive and take additional user input after their first invocation. In this workshop we discussed how represent these kinds of operators with geometry nodes.

The main problems to be solved are how the node group remembers data from the initial invocation and the previous events, how user-generated-events (e.g. key presses) are passed to geometry nodes, and how the operator finishes.

After considering using a new kind of “zone” for modal operators, we concluded that we can just use the existing simulation zones to remember data from previous modal updates. The Delta Time in the simulation zone would just be the actual real-world time delta. For some kinds of modal operators, it would be useful to runs continuously without additional user input, which can be achieved with a simple “Is Interactive” checkbox on the node group. If it’s turned off, geometry nodes only executes when there are new input events.

Control information is passed into node groups through normal (boolean) group inputs. The node group can then map some of its inputs to specific keymap events. The node group defines some default keymap, which could be overwritten by the user with a custom keymap.

The modal operator can always be cancelled with the escape key, so this key can not be used by the operator. For normal termination, we rely on a “Finished” boolean group output. Once the node groups returns true for that output, it finishes.

Grease Pencil Integration

One of the main open topics for the grease pencil integration into geometry nodes was the handling of grease pencil layers in nodes that convert to different geometry types. This is a tricky question because other geometry types do not have the idea of layers.

There are two main approaches for dealing with layers. We could join the geometries generated from each layer together, or can keep them separate by outputting them as instances.

It’s nice to keep layers separate since they were separate before, but turning them into instances is unclear because the word “instances” suggests that the goal is just memory usage reduction. That isn’t the case when every instance references a different geometry. This problem also exists with the new Split to Instances which outputs instances that are all different.

We considered changing the name “instances” but couldn’t really come up with a better fitting alternative. In the end we concluded that just keeping the existing name is fine, but that we can improve the tooling to make the data more obvious.

For example, the spreadsheet and socket inspection should show more details for how many unique geometries are contained in the instances. Naming geometries and their instance references would also be helpful. With those changes, the instance created from each layer could have the same name, making it much more obvious what the data is.

We also talked about a new “Grease Pencil to Curves” node, how curve sampling nodes should work, and different things “Apply Modifier” could do for grease pencil.

Dynamic Sockets

For existing nodes, but also especially for the volume integration, we need new socket designs that capture the ideas of “dynamic socket category” (single value, field, or grid) and “dynamic socket type” (float, int, etc.).

For the dynamic socket category we confirmed that using a wildcard/asterisk socket shape works quite well, since it is semantically meaningful and does not look as busy as originally feared.

For dynamic socket types, the situation is a bit more difficult. Some kind of “rainbow socket” could work, where instead of a single color, a socket could contain a color wheel. We don’t know yet if that looks too busy or not. The more difficult question is the link color.

Interactivity Option

In the past, something like an “Is Edit Mode” was requested that is similar in spirit to the “Is Viewport” node. The intended use-case is to skip some heavy work while in edit mode so that the experience is more interactive. However, the proposal is quite specific to edit mode, and the same thing could make sense for many kinds of edits, even just transforming objects in object mode. Instead, it could be better to change the Is Viewport node into a more general “Interactivity Level” node.

Furthermore, input nodes like “Is Viewport” or “Is Edit Mode” have the problem that they can’t be controlled by node groups. For example, it isn’t intuitive to use a specific node group in “render” mode even if the scene isn’t currently being rendered. This problem also applies to input nodes for node tools. A potential solution is that automatically adding inputs to a group node for these special nodes. All of these inputs could be in a subpanel. This would make it easy to use the functionality, but it can also allow overrides from the outside.

Realtime Mode

The real-time mode has been in the design phase for quite a while already. One goal is to put Blender into an “interactive mode” where simulations just run in real-time and the user can interact with the scene naturally.

To achieve that, the scene time will likely have to be decoupled from a “real-time clock” that just keeps running independently of the scene time.

One difficulty with the design is finding the right set of use-cases that need the interactive mode but can not be handled by the simpler and less intrusive design for node-based modal operators.

Asset Deduplication

The “Append and Reuse” functionality for assets works fine within a single file, but it becomes problematic when working in a production setting that also uses linking. The issue is that many .blend files have appended copies of the same data-blocks which currently can’t be automatically deduplicated when they are linked into a separate file. This leads to a large number of duplicate data blocks.

A solution to that is not local to geometry nodes but affects core functionality in Blender related to linking and appending. Fixing this might require automatically detecting if two data blocks are the same and/or version numbers on all data blocks. Both approaches are fairly involved and require their own project.

While there is a proposal for a solution, it is not generally agreed on and requires more discussion.

Next Steps

In this workshop we managed to make quite a few design decisions, so the main next step is to implement them. For example, the volume, gizmo, and grease pencil integration don’t have immediate blocking design questions anymore. Other topics like the real-time mode and asset deduplication still require more discussion.