How Node Groups can be made useful

Node groups are one of the largely neglected features in Blender. If done right, they could increase the power of node trees by magnitudes, because there are actually things you can do best, if not only, with groups. At this time, nodes in Blender have two applications:

• Save space in the layout
  This is currently the most important (and often only) reason to use groups in Blender. This is partly because the current trees types (compositing, materials and textures) have less use for the other features groups have to offer, but even more because they are poorly implemented.
• Instantiation
  Grouped nodes are just a special type of node, which is internally linked to another node tree. This tree is an ID data block, which can be reused by several nodes at the same time. In programming terms: makes it possible to basically “write a function” and execute it where needed. The parameters and return values are the inputs and outputs of the node, which are a linked to sockets in the internal node tree.

The instantiation feature is currently very hard to use, mainly due to the way sockets are created for a group node. The current system automatically flags all visible, unlinked sockets in the group node tree. These sockets are then mirrored on the outside of the node tree in all instances of that group.

It can become a p.i.t.a. to make sure the right sockets of a tree are exposed in the group nodes, especially if a group is modified during development and in the process looses connections of instance nodes. Often you will rather want to use default values in the internal nodes than expose them as parameters. This requires a tedious sequence of entering values, linking sockets arbitrarily, hiding unlinked sockets, then unlink sockets you want exposed for the group.

Another big problem is that you have no way to rename group sockets to anything useful. Many basic nodes have exchangeable names such as “Value”, “Factor” or “Image” and having a dozen of these names on a group node gives you no idea of what these parameters are used for or what an output will contain after execution. Even the order of the group sockets is uncontrollable (simply resulting from the ordering of the nodes in the tree). Whaaa!

A First Patch

So i came up with a group nodes improvement patch, which aims at fixing these most annoying issues:

1. Sockets appear in a fixed order on either side of the group edit box.
2. The order can be changed by clicking up/down buttons next to each socket.
3. They can be renamed.
4. New sockets can be selectively exposed by dragging a link outside of the group box.
5. Sockets can be removed from the group by clicking the x icon next to the socket.

The default behavior of node groups can easily be restored, if that should be desirable for compatibility reasons or preference. Personally i find it much easier to simply start with an blank group without any sockets and add them one by one, but this should be up to the user.
Having regular links replace the automatic mapping between group and node sockets means that any kind of mapping can be created:

• A group input be linked to several node inputs without intermediate nodes, i.e. the same input parameter can be used for multiple internal nodes.
• A node output can be linked to several group outputs. This will probably not be used very often, but it is possible.
• Group sockets can be unlinked. This means that you can also create sockets for a group and only use them when needed. It also makes it much easier to relink internal nodes without losing the connection to outside nodes.

Note that this patch is written against the SVN trunk, not the more heavily customized particles-2010 branch! It will be ported to the branch of course, but the standalone patch means it can be applied to trunk and be made available sooner.

Some Internals:

This section deals with some of the technical details of the patch. If you”re not a coder it may not be very interesting, but you are allowed to read it anyway ;)
A lot of the improvements in the code are not directly visible in the interface, but they make an important change to the way group sockets work. Originally group nodes in Blender work like this:

1. Whenever a group node is created or modified by adding/removing nodes or links to the internal tree, the sockets in the internal tree are flagged as either intern or extern. This currently makes all unlinked sockets automatically extern.
2. A completely new node type definition is generated just for this group. This contains a list of all sockets that should be exposed by the group.
3. All existing node group instances (= nodes of type “Group” using this group tree) are “validated”, which means that their sockets are synchronized to the type definition of the group. After that all instances have sockets that match those of the group definition.

This way of defining group sockets has some limitations, which are a consequence of the simplistic 1:1 mapping inside the node tree: each socket in the tree is mapped to exactly one socket in the group definition. Here”s how the patch changes that:

• Instead of generating a new node type definition for each group, node trees themselves now store a list of input/output sockets. This list is just as unique to the group as the type definition was, but it removes the redundant group types (groups already had to know their group tree before knowing their full type).
• The tree socket lists are using regular bNodeSocket sockets, which means that regular links can be created between internal sockets and those owned by the group tree.

The patch stops here for now, but there are a lot more things that could be improved.

Comparison of Group Nodes before and after the patch

Current Node Groups in Blender

New Customizable Group Sockets

Using Groups for Control

As outlined in my previous personal blog post there is some redesign on the way, which will make use of a multi-level node tree approach. The (preliminary) top-level tree will define sequences of operators that work on a common data set (particles or mesh vertices). Each operator can be a hard-coded “black box” node, but it can also be a custom operator that defines vertex deformation. These modifier nodes can use a different tree type much in the way regular groups are used now. On the lower “per-vertex” level, the tree is executed in parallel for each of the vertices.

Other nodes can be used to create high-level control in the operator tree:

• Loop nodes (“For” and “While”) repeat the execution of a group tree a number of times or until a condition breaks respectively.
• “If” nodes only execute a tree when a condition holds true. They might even get a second group tree for the “else” case.
• Regular groups define simple sequences, with the additional benefit of instantiation.
• Limited recursion can be very useful for things like L-systems and procedural content generation. This can be implemented by special nodes that define a maximum recursion level or use a different tree for each level.

For these features to be combined nicely, it is mandatory that nodes can be grouped on more than just one level (groups in groups in …). Group nodes cannot currently be added inside other group trees. While this avoids the problem of infinite recursion when a group node is added to it”s own tree, it also limits the grouping level to 1. A better way of avoiding infinite recursion problems but allowing a deeper grouping level is this: Only allow nodes in a tree if they are not using this tree themselves, i.e. are not group instances of that tree or contain instances of it on deeper levels.

Groups in the Node Editor

The other problem with nested groups is the way node groups are displayed and manipulated in the node editor. When you select a group node, you can “open” it and edit the group nodes in a sub-window displaying the group tree. One problem is that you”re limited to the currently open group when editing. You also cannot have multiple groups opened at the same time, so editing groups is a constant series of opening a group, moving stuff, closing the group and working on the parent tree, etc. All made worse by a long-lived bug in the node editor interface, which activates buttons that should be hidden behind other nodes. That happens inevitably as soon as you open a larger node group.

A possible solution would be to allow multiple groups to be open at the same time. The question remains how in that case node selection and editing would work. Selecting a group internal node would select it in all open instances of the group and working on a selection of nodes from different levels is bound to cause more problems. The easiest way to avoid this would be to retain the notion of an “active” editing tree and only work on nodes in that tree at a time. Selection would either be limited to the currently active tree, or only the active (last selected) node defines the active tree. Some experimentation will be useful to find out if this makes for a convenient workflow.

That”s all folks! Hope to see you next time :)