Inside the Blender Cloud addon

In the first week of May we released the Blender Cloud addon. This addon provides an interface to browse the texture library of the Blender Cloud, download textures, and load them into the current scene. In this article we describe some of the more interesting technical aspects of the addon: the use of the asyncio module with the async/await syntax introduced in Python 3.5.

Before we go on to the details: subscribe to the cloud! Tell all your friends to subscribe! We need subscribers to be able to keep up development of the cloud and the addons.

Asynchronous communication

The addon communicates with Pillar, the back-end service of the Blender Cloud. It is a REST service that lives at https://cloudapi.blender.org/. Pillar is a Python service we built on top of Eve, and talks JSON with the Blender Cloud addon. Pillar provides all the project management and metadata, while Google Cloud Storage stores the actual texture files.

One of the core design principles of the addon was that it should not block the Blender user interface. We want communcation to be performed in the background, while we update the GUI as soon as new data is available. There are multiple approaches to this:

1. Non-blocking socket I/O. This allows the code to download data that is available, and do other stuff (like drawing the GUI and responding to user events) instead of waiting. This often results in overly complex code, as you have to maintain a download queue yourself.
2. Multi-threading. This allows threads to run in the background and use the simpler blocking I/O. The code becomes simpler, but you cannot call any Blender code from threads other than the main thread.
3. Multi-processing. Processes take longer to create than threads, especially on platforms without fork() call (I look at you, Windows). You also cannot call any Blender code from other processes.

We chose a fourth, more modern option: asyncio. It allows co-routines to run concurrently, even on the same thread. These are the motivations to choose asyncio in favour of the above alternatives:

1. Bundled with Python and supported by new syntax, most notably the await and async def statements.
2. Allows for clear “handover points”, where one task can be suspended and another can be run in its place. This provides for a much more deterministic execution flow than possible with multi-threading.
3. Support for calling callbacks in the same thread that runs the event loop. This allows for elegant parallel execution of tasks in different threads, while keeping the interface with Blender single-threaded.
4. Support for wrapping non-asyncio, blocking functionality (that is, the asynchronous world supports the synchronous world).
5. Support for calling async def methods in a synchronous way (that is, the synchronous world supports the asynchronous world).
6. No tight integration with Blender, making it possible to test asynchronous Python modules without running Blender.

The asyncio event loop

The event loop is the central execution device provided by asyncio. By design it blocks the thread, either forever or until a given task is finished. It is intended to run on the main thread; running on a background thread would break the ability to call Blender code. For integration with Blender this default behaviour is unwanted, which is solved in the blender_cloud.async_loop module as follows:

1. ensure_async_loop() starts AsyncLoopModalOperator.
2. AsyncLoopModalOperator registers a timer, and performs a single iteration of the event loop on each timer tick. As only a single iteration is performed per timer tick, this only blocks for a very short time — sockets and file descriptors are inspected to see whether a reading task can continue without blocking.
3. The modal operator stops automatically when all tasks are done.

Other addons will likely be able to use this functionality as well. However, as the addon is still under heavy development, the internals might change. Be sure to catch up with Sybren if you’re interested in following development.

More info can be found at the wiki.

The future of the GUI

The GUI of our Blender Cloud addon will change. The current GUI is based on Manu Järvinen’s Asset Flinger addon, which draws itself on top of Blender’s UI using OpenGL. As a proof of concept this is fine. A future version of the addon will integrate nicely with the Asset Engine that Bastien Montagne is working on.