Geometry Nodes Workshop: May 2024

In May 2024, various Geometry Nodes contributors came together in Amsterdam to discuss many design topics. This post gives a general overview of the topics that were discussed. You can also read all the notes we took during the meetings.

Gizmos

The overall design for gizmos had already been agreed upon during the last workshop. However, there were still a few open questions regarding visualizing the backward flow of gizmos and controlling visibility. Furthermore, we discussed an additional Transform Gizmo node which would target the common combination of location, rotation, and scale inputs.

For the backward flow we want to explore double links which kind of shows that the gizmo controls its input value. A few alternatives have been proposed like showing a gizmo icon on the links however the double links seemed better in the end. We still need to try how this will look in practice. If the link visualization works, we can reduce the amount information we currently show in sockets in the gizmo patch because it’s redundant. The gizmo icon should only remain visible where the actual value is that is controlled.

Gizmo visibility can be controlled in a new gizmo panel on group nodes and the Geometry Nodes modifier. We started preferring this option compared to showing the gizmo visibility directly at each input value. This also works better if a gizmo controls more than one input. An optional gizmo panel could be added to node groups which could allow defining groups of gizmos whose visibility can be controlled together.

Baking

Baking in Geometry Nodes was also discussed in the last workshop. Since then, we got the Bake node and Simon was able to gather some initial experience of using it in production. That made it all the more clear that higher level tooling is necessary to manage multiple bakes. The centerpiece of that tooling will be a new display mode in the outliner that lists all the bakes, allows filtering, and makes it easy to perform batch actions.

Being able to tag bakes is necessary for organizing them. One can either add tags on the bake node directly, or on the “bake instance” which is stored on the modifier (the same bake node can be used in different contexts). When to use each kinds of tag depends on the use-case and how a production environment is set-up.

For ease of use and sharing of files, we do want to add the ability to store the baked data in the .blend file directly. This functionality should ideally be integrated with the more general packing and unpacking features Blender already has.

Path Variables

One thing that often comes up when talking about baking is the ability to control file paths at a higher level. For example, one may want to specify a bake directory at the scene or even project level. Individual bakes should be able to specify their path relative to these paths. To support this, we would like to introduce so called path variables like ${SCENE_BAKE_DIR}. Those can be be used as part of filepaths elsewhere and are replaced by the corresponding value when the path is used.

The internal and Python API for path variables requires some more thought. The tricky aspect is that often these variables are not global but depend on context. An additional complexity for bake paths is that we also want to remember the path that we actually baked to last time separately from the path that contains the path variables.

Rename Sockets in Nodes

It’s inconvenient to have to go to the side bar to change the name of various sockets, especially when the name is created in the node editor itself rather than the sidebar, by connecting a link to an extension socket. The solution is support for renaming sockets directly in nodes. In theory, we could also support removing and moving sockets, but we ignored that part for now.

Renaming should not happen with a single click on the label because that would make it annoying to move nodes. Ideally, we would support double-click and ctrl+click but unfortunately double-click is hard to do in the current state of UI code. Furthermore, it would be great to have some indicator in the UI that shows when a label is renameable. A box on hover, similar to the one in list views can work. This could potentially also be used on the breadcrumbs in the node-editor.

Tools for Node Tree UX

For this topic we went over various ideas to improve the UX when working with larger node trees. Among others ideas, we talked about multi-line comments, custom zone colors, link portals, frames, improved search, drag-and-drop for attributes, separators in node groups, viewer node position, and huge add menus. See the meeting notes for more information.

We also went a bit deeper into discussing a new interactive “slide” operator which is supposed to become a more controlable and predictable alternative to the auto offset feature that automatically moves nodes apart to make space. Development on this has started, but we are still working out the exact desired behavior.

It’s planned that Blender 4.2 comes with the new extension system, which increases the separation of features that are core of Blender and part of third party add-ons that were only shipped with Blender. The most important add-on for us is Node Wrangler. We went over some of its features because it’s now more urgent to move some of them into core.

Asset Embedding

Asset embedding started being a bigger topic last year when we noticed that our existing import methods of linking and appending don’t work so well for assets, especially also for assets we want to ship with Blender. We already had a meeting a month prior where we discussed an asset linking proposal. As a follow up to that, a new proposal was written which we discussed this time.

There was much more agreement on the proposed approach now. However, there was still some disagreement on what libraries in Blender are or should be. We concluded that we will avoid changing the definition of a library (which is just a file reference) at the cost of some additional complexity in order to avoid name collisions when different data-blocks with the same name come from the same source file at different points in time. Also the case when the same data-block comes from different source files has to be considered.

The next step for this topic is to better define how these issues should be addressed and to then prototype the solution.

Menu Socket

Menu sockets and the Menu Switch node were introduced after the last workshop in Blender 4.1. It was known already that this first implementation is limited in the sense that one needs a workaround to switch multiple different types with the same menu. That limitation has not been resolved yet.

There is another issue that feels like a limitation which is that the Menu Switch can’t be copied without making the menus incompatible. This misses that the way to reuse the same Menu Switch on different values is to make it a node group. The limitation that one can’t simply copy the menu switch node and have menu items automatically match by name is very important for us to be able to use these menu sockets for built-in nodes in the future. It also allows for renaming and reordering of menu items without the fear of breaking node trees that depend on it.

The main immediate problem is that it’s not obvious enough that one can use node groups to reuse a menu socket. One way to address is this show this information to the user when attempting to reuse a menu the invalid way. We’ll likely further generalize this by always showing extra information when a link is invalid which would help in other places too.

Being able to dynamically change socket visibility based on which menu item is selected is not yet supported. However, we checked over the design from the last workshop again, and came to the same conclusions.

Field Context

When working with non-trivial node-trees, it’s sometimes hard to tell which context a specific field is evaluated on. We discussed different ways this can be addressed. A simple first measure is to show the domains a field is evaluated on in a tooltip.

More importantly however, we want to be able to highlight where a field flows and which geometries it’s evaluated on. For that we want to extend the ability to select sockets and to visualize extra information based on the socket selection. This can also become a more general feature that can be controlled in the overlay popover. For example, at one point one might want to see where a field is evaluated and at another time all the nodes that are affected by a specific socket.

Even on simpler node-trees this can be a useful learning and teaching tool.

Lists

Lists as a general feature for Geometry Nodes have been planned for quite a while already but they were never discussed as part of a workshop. The general idea is to support standalone lists and to perform various operations on them. In this context, standalone means that they are not tied to any geometry. That also means that they can easily go out of sync with a geometry when some topology-changing operation is applied.

Besides lists that are just data, we also want to support list fields. Nowadays, a field computes a single value per geometry element. With list fields it would be possible to work with multiple values per element. This can also simplify working with topology nodes a fair amount.

Initially, we’ll likely only have flat lists instead of also supporting nested lists. Those would complicate the type system and one needs to be much more careful with how to expose them to users to avoid many performance pitfalls. We could still add them later on if required.

Socket Shapes

Currently, we use three different socket shapes to represent different kinds of values in Geometry Nodes. A circle means that something is a single value, diamonds mean fields, and diamonds with dots reference a single value but can become a field.

This works reasonably well so far, but things become more difficult with the introduction of volume grids and lists. As the volume proposal shows, we intended to use different socket shapes for volume grids, and later also for lists. However, this turned out to be more challenging than expected. It would be easy if every socket only had one specific kind of value. In reality, many nodes accept different kinds of values requiring an additional more general socket shape like the proposed asterisk. However, depending on the node or node group, the set of allowed types can be fairly arbitrary subsets of all types which the asterisk shape can’t communicate at all.

More complexity arises within node groups where on the inside we don’t really know what will be passed in. Currently, we always fall back to assuming a field is passed in but this assumption might not work anymore when we have more types. Yet more complexity comes from supporting list fields and potentially nested lists which by the current logic would require different socket shapes too. The final piece of complexity comes from the desire to show what value a socket currently has and also what it could be (like the diamond with a dot).

We spent quite some time trying to figure out rules for what these socket shapes should mean exactly and how they propagate through nodes. Unfortunately, we didn’t solve this yet even when we relaxed some of our goals.

The most promising direction currently seems to be to simplify what we intend to show with socket shapes a lot and to rely on other means to communicate the necessary details of what types a socket has and could have. For example, we could keep the current set of socket shapes but say that the circle can represent any kind of data including grids and lists whereby the diamond shape can mean fields and also list fields. The downside is that now it’s often less clear which sockets are allowed to be linked just by the socket shape alone. For example, a float grid and single float value would have the same socket shape, but it’s not valid to connect a grid to a single value. Additional information in tooltips, better error reporting for invalid links, graying out sockets that one can’t connect to when dragging a link and other overlays might solve this well enough. The upside is that this makes the type system much easier to extend without sacrificing the socket shapes we have already.

Local Node-Groups

The primary purpose of node groups is to share a node-tree in multiple places. However, this focus on share-ability makes use cases where sharing is not the main goal more cumbersome. For example, when using node groups just to organize a larger tree without worrying about reusability, it is unexpected that the node group becomes available in the add menu, can be linked to, and generally clutters data-block lists where it can also lead to name collisions.

The idea with “local” node groups is to have node groups that can be used within a specific context but don’t leak out of it. The most constrained approach would be to support single-use node groups that are completely embedded in their parent data block. Copying the group node would then also copy all the nodes within. We could support reusing embedded node groups multiple times within the same parent group. The next step up would be to allow using groups in multiple other groups without adding them to the Add menu. That can already be done by prefixing data-block names with a dot which hides them in various places.

So far, all of the mentioned solutions only apply to node groups. However, it turns out that the same problem also exists for many other data-block types. For example, a character might consist of multiple sub-objects which should really stay local to this character. Or a node group might use some separate curve object as profile shape and the object is not expected to be used separately from the node group. Ideally, these data-blocks that live in different contexts also shouldn’t be able to have name collisions. This more general solution has not been explored further by us yet, but will likely become a topic again.

Grease Pencil

One of the big new advances with Grease Pencil 3 is the ability to use Geometry Nodes. The most challenging aspect is how to integrate the idea of grease pencil layers with how plain curves work in Geometry Nodes. Most of these things have been discussed at the previous workshop already.

This time we focussed more on how to turn grease pencil into separate curves and how to put them back together, ideally without losing too much information. This will allow using the full power of Geometry Nodes to modify grease pencil data but also to generate it from scratch. The fairly straight forward solution is to have nodes that turn grease pencil layers into curve instances and back.

Custom Viewer

The primary goal of custom viewers is to allow building node groups which act like the Viewer node but can do additional processing. For example, one could build a vector field viewer that evaluates the field at various positions and draws arrows. Density grids could be displayed as boxes containing the individual voxels and tiles.

The main difficulty is that now there are different kinds of viewers which makes it less obvious which one ctrl+shift+click should connect to. A simple and probably good enough solution is to connect to the viewer which was active the most recently.

We went over different ways to mark node groups as viewers. For example, a simple heuristic could be that node groups which don’t have output sockets are treated as viewers. That would work fine for now in Geometry Nodes, but breaks down if we get export nodes which also wouldn’t have outputs. Of the proposed solutions, just a simple option on the node group that the user can control manually seemed best. This flag could also be set automatically when creating a new group from just a viewer node.

Besides custom viewers we also briefly talked about the ability to show multiple viewer columns in the spreadsheet and about other kinds of built-in viewers that we’ll likely need eventually. Especially once we have lists, it would be good to have a viewer node that can show the lists content directly in the node editor without having to use the spreadsheet. Another topic was the ability to have multiple active viewers but we did not get far with that yet.

For-Each Zones

For a couple of releases, we have repeat zones which allow running nodes in series an arbitrary number of times. What’s missing is the ability to iterate over many elements in parallel. For example, one might want to generate 10 different trees or modify each unique instance in an instance hierarchy individually.

There are different kinds of for-each zones. The most common one is the For-Each Geometry Element zone (name still pending). It has a domain dropdown and allows iterating over e.g. all points of a geometry individually and producing some value for each which are then joined together. We noticed that there are two different kinds of outputs in this zone. One is the original geometry with newly added attributes which are initialized by the zone. The other are new geometries which are generated from each geometry element and then joined together.

The initial implementation would iterate over the geometry elements one by one, based on a selection. This could be extended later to support iterating over groups defined by a group ID. For now, one would need the Split to Instances node to iterate over groups. A problem with supporting groups directly is that they break one of the core mechanics of for-each zones which is that they can turn fields from the outside into single values on the inside which allows them to be used in many nodes which don’t support fields. This problem could be aleviated by turning fields into lists instead of single values on the inside.

Other kinds of for-each zones could iterate over a set of indices, list elements, grid voxels and unique instances of an instance hierarchy.