Improving Alembic support in 2.79 and 2.8

Alembic logoBlender 2.78 saw basic Alembic support for both import and export. The initial patch was provided by DwarfLabs, and was then integrated into Blender by Kévin Dietrich. For the previous month, I (Sybren) have been working on bringing Alembic to the next level, and that work isn’t done yet. This means that things are going to change; for the better, of course, but it may also mean requiring a re-export of your Alembic files. In this blog post I want to sketch my plans, and collect feedback from you, our dear Alembic users.

Broad plans

The final goal of my work is to be able to take an animated character, like Hendrik IJzerbroot a.k.a. Agent 327, and export it to Alembic for further processing in either Blender or other software. Furthermore, we want to be able to take animated characters from other software and load them into Blender. This includes support for:

  1. Instanced geometry
  2. Dupli-groups and dupli-objects
  3. Hair and particle systems

Furthermore, we want to change the workflow so that it will be easier to re-export characters by coupling export settings and collections of exported objects, giving the artist more control over what is exported exactly, and automatically (re-)apply certain modifiers and materials upon import.

Part of this work can be done for Blender 2.79 already, while other parts will have to wait until Blender 2.8.

Ideas for Blender 2.79

Blender 2.79 should be a stable release. As such, I’ll only try and do bugfixes and non-invasive improvements here. For example, I’ve fixed some bugs with coordinate transformations (we use Z=up in Blender, but Y=up in Alembic files). As a result, once those fixes are in the master branch (and thus in Blender 2.79), files will export & import differently. If you exported your files from Blender, you may have to re-export them to have any benefit of those fixes.
My questions to you are:

  1. Is it okay for you if you have to re-export your Alembic files when switching to Blender 2.79?
  2. What does your pipeline look like? What software do you use to generate and import your Alembic files?

More detailed plans are available on my personal wiki page.

Ideas for Blender 2.8

Blender 2.8 will introduce the concept of collections, which we could use to determine which objects get exported to which Alembic file. Furthermore, these collections could contain information about which modifiers to bake into the mesh (“skip the final subdivision modifier”), or to toggle between render-enabled and viewport-enabled modifiers.

We also want to work on a tighter integration between Blender’s data model and Alembic (or other cache formats), to be able to stream data from Alembic into the scene, without requiring the complex setup of constraints and modifiers that the current importer sets up. Together with other Blender devs we’ll look into an improved in-memory caching system for Blender 2.8.

I’ve already been working on supporting instanced geometry (i.e. multiple objects using the same mesh data) and dupligroups (i.e. empties that duplicate a group). These changes are more likely to be included in Blender 2.8 than 2.79, since they really introduce new features and should be tested thoroughly before release.

My question to you: what are important features for your workflow?

Conclusion

Work on Alembic support in Blender is an ongoing process. What is really important at this moment, are your feedback and actual production-level Alembic files. We need both to be able to support actual film production scenarios. Of course we will treat all files as confidential, although we would also welcome some files we can use in public test cases & benchmarks.

Please leave your feedback here in a comment or mail me at sybren@blender.studio. To send files, send me a link or send them through a service like WeTransfer.

28 comments 8,244 Views
  1. Sounds nice. Maybe its possible to import lights from different tools too. Right now the lights import from houdini dosent work.Also important is a vdb volume importer via alembic.

    • Importing the transformations of the lights shouldn’t be a problem. I’d have to investigate into other lamp properties to see what’s actually stored in Alembic, though. Do you have an example file for me with lights exported from Houdini, together with the Houdini file?

  2. Something that would be really noces is the ability to easily loop and offset animation data of alembic files, this is useful when you want to populate a scene with characters that have the same animation, at this point is quite cumbersome even more when there are multiple meshes involved in one character

    • Yeah being able to offset would really help. check out Maya and Houdini as they handle offsets really well. The alembic node should be able to change the start/end time along with time offset and speed. (Also cycling.)

      • The free version of Houdini doesn’t support Alembic, and I haven’t checked Maya yet. So if you want me to check them out, please donate a license to the Blender Institute.

    • This is definitely something we have in mind, since that’s also what we did (with another cache format) for Caminandes Llamigos to get the crowd of penguins.

  3. A couple of thoughts:

    Support for more than just geometry would really help. Cameras, lights, transforms, meta_data, etc will all help in adopting a more deeper pipeline integration.

    Reading/writing should be simple and scriptable. I like what I am hearing about the collections as a lot of the baking might have to happen on the farm and not done interactively by the animator. Being able to script the collections to export in batch and then reassemble inside the lighting shot with material information would be great.

    The idea is that you have a pipeline that supports animators updating anim scene files, exporting the abc files and a lighter importing them with texture information (material.json???) to be able to light the scene without any connection back to the animation blender file.

    Something like this:
    Rig.blend –> AnimScene.blend –> (Asset.abc + MarterialInfo.json) –> LightScene.blend –> Images.####.exr

    Camera export followed by lighting export is really important. Camera is the big one where you want to export the camera along with the geometry (separate files) to be able to reassemble in the lighting shot.

    Materials and how to have an Asset.blend export the Geometry Collections along with the material connection information so that you just load into your lighting scene the geometry.abc along with some level of material information is key.

    • Corban (Corban G?). You’re basically talking about USD 😉 Which does a bit more than alembic in this pipeline regard.

      Brian

      • # yeah corban g

        I am not looking for USD exactly. (While I do want USD support in Blender, that is a different topic. 🙂

        Sorry I am not suggesting that this project support Surface information. I would expect that to happen at the pipeline level. (Something we would do here at Nimble.)

        What we are hoping for here at Nimble is for Blender to support the key definitions of geometric data types for Curves, Nurbs patches, Face sets, and Cameras. If we have geometry and set/group information then we can build the surfacing pipeline on our side. We just need to make sure that we can setup the same structure in the lighting application (blender, katana, maya,etc) as we had when animating in Blender and exporting the geometry (via alembic) and surfacing information (via Nimble pipeline).

        It’s really just supporting the standards that is currently setup by the the Alembic OpenSource community. Camera support is important to take camera information in and out of Blender. Transform information is used for exporting data beyond just geometry. Blend data is used for… well whatever the pipeline needs.

        Hope that clears things up a bit.

    • A defined workflow for shader re-assignment would definately be good. Having worked with Alembics for the last few years in production I’ve seen a number of solutions

      *Shader hookup file (say your json)
      When the objects are loaded into the scene their materials are loaded and assigned using a lookup table. This is really nice but can break if the information your using to do the hookups is modified (say someone renames a mesh).

      *Dummy Rig.
      A dummy heirarchy is present in the scene before import of alembic. this has the latest shaders on it. when the alembic meshs load they’re replace the dummy model and keep its shaders. Sometimes people dont replace the meshs they simply bind then shaded mesh to the alembic and hide it.

      And yeah Metadata is absolutely essential. all channel data should be embed-able, along with any arbitrary stuff you want to attach like UUIDs, or tags.

  4. this is probably not part of the project, but for my wrokflow as a freelance artist compatibility with other apps its really important, i was able to import alembics send to me from max but i wasn’t able to sent them back because i couldn’t send animated empties(try to bake some on boxes but it was no good).
    Other serious problem i had presented when i modified a model that was sent again from max, and when i wanted to send it back i discover that the mesh went right but with no uVS and the vertex numbers or indices changed, so i wasn’t able neither to take the uvs in an fbx model and transfer the uvs from there, so alembic would be a really practical solution but again wasn’t usable.

    The propose here its great, having this kind of pipeline will sure improve stand alone productions, would be great to know if companies like the foundry, maxon ¿autodesk? will provide native compatibility support for blender alembic files, that would be actually awesome.

  5. Great news! I am researching a workflow using Blender to animate characters and render them in Unreal. It has been a problem not to be able to export linked/grouped mashes. Long time to export is also a problem.

    Re-export will not be a problem.

  6. My workflow requires exporting cloth simulations to Unity 5. Is there any development planned for simulations?

    • There is, but it’ll be at a lower priority than animated character meshes. However, we already use cloth simulation in the Agent 327 Barbershop short, so it’s definitely on our radar.

      • Not sure I get why any geometry is not exportable via Alembic in Blender. (Sorry don’t know Blender well enough yet.) I would assume that any selected geometry in the scene (regardless of where it comes from) should be exportable. Linked, grouped, simmed, baked, dynamically driven, etc, etc… does it matter where the geometry comes from?

        Sorry if I am missing something here.

  7. How about being able to export fluids simulations from blender using alembic? I haven’t tested that. Is it possible right now?

  8. Hi, I spoke about extra attr of .abc in this ticket https://developer.blender.org/T50725
    For lot of big studios, abc and its extra attr is the skeleton of the pipeline, I am technical director and i really want to integrate blender in our pipeline as main software, but without this abc extra attr it’s not possible… only 3ds max don’t manage that, so please become better than 3dsmax :p
    Thanks for all !

    • I agree this is really important for bigger pipeline work.

      • If it’s important to you, please help us actually implement this by providing production files that show off the features you’re looking for. We can treat those files confidentially, no worries there.

    • Colin, please join the discussion at our tracker. I’ve asked for example files that show what you want to have, but you haven’t replied yet with actual files.

  9. If it is possible to include objects custom properties in the alembic it would be fantastic.

    This would make it possible to track assets (for example by using a integer ID as a property) when assembling the scene in houdini for example.

    The scenario would be

    1. asset lookdev in houdini (or other application)

    2.asset layout of multiple assets in blender

    3.export blender layout to alembic

    4.import alembic layout to houdini and connect shading from lookdev asset to the corresponding assets in the alembic layout that is imported to houdini

    • Daniel, do you have documentation for me about how Houdini (and others) store custom properties in Alembic files? Example files would help too.

  10. A small thing that still helps with frustration is a more accurate display of the progress bar when exporting ablembics. I often experience that the progress bar runs to 100% very fast, but that blender is still exporting the abc-file afterwards and is frozen for a couple of minutes.

    In my case I’m exporting 7 objects (pretty hi res) that is instanced to +4000 objects

  11. Hey Sybren!.. please talk more about the PBR viewport.. this is something Im really excited to see in Blender and would be cool to follow on updates on it 😀

  12. Hello Sybren,

    A couple of months ago I attempted to make use of blender as part of our animation pipeline, but unfortunately I failed due to the way the alembic integration behaved.
    I will explain what I was trying to do in the hope that this will be useful to you, and that I hope in future I will be able to get it working.

    We are mainly using Maya, but we also use Nuke and 3dsMax. We publish cameras and geo as abc’s, and our layout publishes are json files containing a list of assets and there transform.

    in this way we can rebuild the same scenes in maya, nuke, and 3ds max.

    my plan for blender was to have a script that would build a layout and then render it. this would then be an automated process that would run whenever there was a new camera/layout/animation cache.

    while it is possible to import abc files via a script, there was no way that I could find to wait for this process to finish and then do something else.

    so if i have a script that imports a scene and then renders it, i find that the render process happens before the abc is actually imported. I also cannot assign a shader to what I have imported from the abc – since it is not actually in the scene yet.

    Another small annoyance is knowing what has just been imported. I started writing code to check everything that is in the scene, then do the import and compare the difference to find out what I have actually imported.

    an ideal function would complete only when the import is finished, and return a list of the new nodes that have been created in the scene.

    Regards

    • That sounds like a useful workflow indeed.

      As I wrote in the article: we’re still looking for actual production files. We can treat them confidentially. It’s really important to have such files, rather than constructed simple examples, to thoroughly build & test new features.

  13. I’m not very experienced with ABC but I would love the ability to export/import cameras following curves or curves in general (bezier or point)

  1. Leave a Reply

    Your email address will not be published. Required fields are marked *

     

share this story on