General remarks on node groups

Node groups will hopefully get more attractive and useful in the future. A new asset manager is being worked on, which i hope will make it much easier to efficiently share and re-use complex node group setups too, in addition to objects, rigs, materials and the like. This may not be very important for compositing work, my impression is that in that domain artists usually “stack up” effects one after the other without reusing specific setups a lot. However, in the case of shader nodes (Cycles) and the upcoming particle nodes as well as future modifier nodes this scenario is much more likely. There are basically two different ways to make use of node groups (apart from simply saving layout space):
  1. Build up complex effects out of small elementary parts
    Examples:
    - Generic diffuse/specular/environment shader from different bsdf nodes, to be used as a “classic” material
    - Particle emitters from basic components, such as create new particles -> distribute on mesh surface -> initialize velocity from normals, etc. Combine these in a single node group and you have a complete particle emitter asset.
    - Procedural objects with modifier nodes, e.g. a “window” asset from simple base shapes and user settings like dimensions, number of panes, frame thickness, and of course different material slots for all the parts
    The advantage of generating such complex effects from simple components instead of adding them as a new node type is customization: You can take an existing shader/particle effect/object, make a copy of it and then tweak any internal part to make a new one. All this can be done by artists or TDs, without having to write C code and either use their own Blender builds or get a patch accepted into trunk.
  2. Interface reduction
    Often a node can be quite cumbersome to set up, especially if it is a complex node group with lots of different user options. Reducing the interface of such a node or node tree is a good way to create a usable tool by only exposing those settings that artists need frequent access to. Any setting that is not exposed will just use a default constant value internally.

Problems with editing node groups

The current paradigm of “opening” node groups in a single top-level node tree has a couple of flaws:

1. Only a single level of node group depth

You can actually add node groups inside other node groups by now, but you can not actually edit them on that deeper level. The only way to work around this is by adding a second instance of that node group at the top level and edit it there.

2. Only a single node group can be edited at a time

This design is supposed to mimick the edit mode feature used for objects in the 3D view. However, the edit mode paradigm does not really apply to them. While objects are composed of different internal elements (vertices, edges, faces) and do not contain other objects in turn, node groups are recursive by nature (a node containing other nodes).
Removing this limitation is difficult: On the design level it would mean a lot of graphical overhead to edit groups inside other groups (inside other groups etc.). Each group currently has a list of interface sockets, taking up space. Also opening a group node usually leads to overlapping with nodes in the background, so the visual outcome is ugly and confusing. Furthermore the simple “tab-tab” switching between open/closed node group would not work any longer, since it can mean to open an internal node group as well as closing the parent group.
On a technical code level this change would have big implications on virtually every single node operator, since these work by detecting the active node tree and the active/selected nodes from the context. If any of the open group nodes (which all have their internal node tree) can be eligible for editing then each operator must be told explicitly which tree/nodes to work on.

3. Input/Output columns in display

Currently a node group has a number of sockets on the left and right displayed in columns, which translate to sockets on all instances (group nodes) of that tree. These “tree sockets” (as opposed to node sockets) are connected to the internal nodes.

Complex Cycles node tree for a snowy mountain shader

The problem with this is that the tree sockets placement is completely automatic, unlike other nodes which can be placed freely. What’s more, the overall size of the node group editing windows (and therefore the tree socket columns) changes when moving nodes around (bounding box).

Moving nodes around also moves the tree sockets

With a lot of group sockets you end up with lots of long connections, especially when some input is used at the far right of the group tree (and vice versa for outputs). Add to this the new “drop on link” operator that automatically inserts nodes on links and your node group editing becomes very hard to control and frequently does unintended things.

Stupid long links getting in the way, i didn't want to insert that there

Just for the sake of completeness: The shader node tree above uses a combination of Z height limit and normal angle to Z axis in order to create masks for a snow-capped mountain. The details are not really important here, it’s just one of the more complex node trees i had lying around that is suited to demonstrating node group features. This is what the result looks like:


Solutions

Make a usable “frame” node

There is a new node type called “Frame” in the Layout category that provides functionality expected from node groups (the naming should be changed as well, one suggestion was “Tray”). The current implementation is still experimental, but with a little bit of work it can be a great feature.

The basic idea is to use this as a kind of local grouping helper. Any node (including other “frame” nodes, groups, etc.) can be attached to it, much like object’s parent/child attachment. Moving a frame node also moves all attached nodes. They can be linked to nodes outside the frame. Hierarchies of frame nodes can be created as well.

Unlike “real” node groups these frames can not be instantiated (shared), but they could allow simple duplication of all attached nodes. Since this is a very common use case for group nodes they seem to be a good replacement for local grouping of nodes.

Here’s a list of proposed additional features:

  1. Clear indication of attachment (currently not visible)
  2. Beside explicit operators for attaching/detaching nodes to frames: Moving nodes outside of a frame detaches them.
  3. Moving nodes into a frame while pressing a modifier key attaches them to the frame (currently always attaches automatically).
  4. Frames can be locked, i.e. no attached nodes can be moved, all buttons are inaccessible, no linking, etc. This way the frame can be handled as a fixed, local group and makes editing easier.
  5. Frames can be collapsed, to free space in the layout. This has some design challenges though, like how to deal with links to/from the attached nodes.
  6. Standard operators for duplication and deleting work both for a selected frame node alone as well as the frame plus all attached nodes (using a modifier key)
  7. New operator for “make frame around nodes”, similar to current “make group from selected nodes”. Ungrouping simply means deleting the frame node (see above).

Not included here are all sorts of UI display improvements, such as easier resizeable borders, custom color schemes and so on.

Frame node hierarchy, active node highlights parent frames

Edit node groups as separate trees

Reusable, standalone node groups would be edited just like any standard node tree would be. When “opening” a node group the editor display will switch to the internal node tree, instead of pretending to display a specific instance of that group in its parent tree (which is in fact just the shared node tree). This way the editing of nested node groups works in a consistent way across all possible levels.

No more nodetree/edittree distinction in the node space struct is required, there’s just a single active nodetree. The node editor window can keep a history of node paths in order to allow a web-browser-like behavior:

  • Opening a node group pushes a node path onto the history stack, before switching the node tree over to the group internals. (“Follow a link” in web browsing terms)
  • Closing the node group means popping a node path from the history and go back to the previous tree.

This behavior may not be quite as immediate as the tab-tab edit mode currently possible, but it seems that this a tradeoff between flexibility and simplicity. For simple grouping the frame node type described above would be used instead. Node groups on the other hand will become the choice for long-term reusable node assets.

It may be a good idea to actually implement these nodes as a completely new node type “Asset”, instead of remodeling the existing Group node type. This way the usability can be tested and if successful the Group node type can be removed at some later date.

Detach interface sockets from columns

Instead add an actual node for each socket in the group tree interface. These would be specialized nodes with a single input/output respectively. All settings for a particular interface socket can be either displayed on the node itself or in the extended setting in the side bar (active node panel). This includes the socket name, data type, subtype (currently not directly editable), default values and optional custom limits. This has a couple of advantages:
  • It reduces clutter in the layout. Interface nodes can be placed by the user just like any other node, thereby cleaning up the layout and making it much easier to understand. Inputs/outputs can be closer to the internal nodes that make use of them.
  • No more automatic unintended movement when the bounding box changes. Less problems with unintentional inserting of nodes in links.
  • Less space wasted on interface settings. Once the group socket has been set up properly the node can be collapsed and only takes a small amount of space.

Current way of linking sockets to the input/output columns

Proposed input/output interface nodes

Some questions:
  • How to set up the order of inputs/outputs?
    The column stacks make this quite intuitive, so i suggest to keep socket display there, but just display names. Changing the ordering there should use drag&drop instead of buttons (much easier + also saves space).
  • Only one node for each input/output, having multiple nodes for the same socket would be too confusing. Using the same input in different places can still lead to long links, but it’s much more controllable then and is at least not worse than the current system.
  • It would be nice to still have some visual indication of which tree socket in the column associates to which internal node. This is not as important as it may seem at first, since there is no real functional connection between the order of sockets in the columns and their nodes. However, some sort of highlighting indicator when selecting the node or the column item would make it easier to recognize them quickly.

Generating top-level interfaces from node trees

This feature is not actually a cure for an urgent problem, but will help increase usability for node assets. The idea is that any top-level node tree used as part of a material or texture or (later) modifiers and particles would be more usable with a compact interface (in buttons window) with just the important changeable settings.
As an example, consider a node tree setup for Cycles. Every material in Cycles consists of nodes, there is no hardcoded, non-node material system any longer – the node tree essentially is the material. Now, when using an existing material node tree there are usually a plethora of possible input parameters to consider: any unconnected node input in the tree must be considered by the user. There has been some effort by Brecht to implement a more “classic” list-style view of the nodes in the material buttons panel, but imho this is not of much help for large and complex node trees.
What i want to propose is to use the existing options for group node sockets as a means of creating an “outer” interface for such node trees. Basically what we are doing in node groups is defining an interface out of all the possible parameters used by internal nodes. Until now the only place where this interface is presented to the user are the group node instances: all exposed inputs and outputs of the group tree are translated to simple sockets on the group nodes linking to it. It would be quite simple to use the same system for displaying such an interface as buttons in the button space. Not only does this reduce the vast amount of possible input parameters to a sensible necessary subset (which can be extended when needed), but it also allows us to use meaningful names for parameters.

The snow shader example with current node tree list view

With just a few input nodes the interface can be reduced to meaningful parameters

UPDATE

After implementing the first two of the features mentioned above (interface proxy nodes and browser-like tree editing), i’ve made a little video tour to show them in action:

Node Groups Editor Tour