A New Developer Documentation Platform

In this proposal, Julian Eisel highlights how a more accessible and powerful documentation platform can help the push towards a development culture more centered around documentation and design.

The Current Platform

To start off easy, let’s analyze the current wiki-based documentation system.

The Editing Experience

The MediaWiki text editor feels outdated, and uses a specific syntax (not Markdown), but most of all provides a very cumbersome workflow for uploading images. These aspects affect the motivation of developers when using it.

A more modern alternative called HackMD is used much more often instead. That is a Markdown editor with live-preview and collaborative editing, where multiple people can work on a document at the same time.

Some developers used this for technical documentation too and it works well. Eventually, the content needs to be moved to the wiki, which means having to convert the Markdown syntax to the wiki syntax. Sometimes this meant that content wouldn’t end up getting ported.

Navigation and Structure

More effort can be put in categorizing content as well, making entry-level docs easier to find.

Disconnected From the Code

The wiki is a separate website that you typically don’t have open unless you consciously decide to use it. Other organizations found that such platforms tend to see less activity from developers; they require a context switch. The closer documentation is to the code, where developers do their actual work, the more likely they are to work on it as well.

Imagine if descriptions for tooltips would live on a separate webpage. They would probably get outdated quite quickly. Currently they live in the source code, and developers maintain them actively (even if there’s a lot of room for improvement still).

Invite-only Platform

To prevent spam and abuse, the wiki has been an invite-only platform for several years now. This makes extremely hard for community members to get involved and contribute.

Unrelated to the Platform

There is a shortage of technical documentation. While there are a few well documented areas, and more general information like build instructions are covered well, a lot of Blender’s designs are not documented, and much of the existing documentation is not up-to-date or simply not that useful.

The practical difficulty of contributing improvements, and the lack of good examples has led many developers to not open up the wiki and contribute to it.

---

Documentation Culture

Let’s take a step back, and look for more intrinsic motivations. In fact, this article isn’t just about a change in platform. It is looking for something bigger: Can we build a new developer documentation culture?

That is a big question, isn’t it? Well, here’s some good news: others have done it. Check out “How Google, Twitter and Spotify built a culture of documentation”.

This change was driven by technical writers. Originally technical writers that felt like they failed in addressing the problem they were supposed to solve. They would just jump from project to project, leaving behind brand new documentation that would become outdated in no time. Nowadays, their technical documentation gets used and updated all the time – by engineers! Can it be true?

How did they do it?

Various resources from Twitter, Google and Spotify tell their story. They closely followed the same approach, even if specifics differ. Briefly, here are some key findings:

• Culture of docs: Regular documentation sprints. Educate developers on technical writing. Lead by example.
• Standardize and centralize: Provide answers about writing documentation before they arise. Clear examples and templates. Share the same platform for all documentation.
• Feedback loops: Make it easy to give feedback (e.g. mark text in documentation > right click > “Report a Bug”).
• Keep it simple: Don’t overcomplicate; avoid/remove any points of friction so the actual content is the focus.

The technical writers are still there, but their role changed to helping and empowering developers. For example, they maintain the documentation infrastructure and make relevant strategic decisions. They make sure documentation just works for the developers/engineers.

This brings us to what appears to be the most important change: Fiercely optimize for the engineer.

Twitter, Google and Spotify have committed to a documentation infrastructure that is based on the idea of Docs as Code.

Docs as Code

The idea is simple: Treat documentation like code. Employ a workflow that developers are comfortable with; make them feel at home. This includes features like version control, collaboration and automation.

More specifically, Docs as Code typically features the following:

• Simple markup language (Markdown, ReStructured text, etc.)
• Version controlled repository, close to the code
• Pull requests, versioning, branching
• Forge integration (Gitea, Github, etc.)
• Continuous delivery, automated checks

We actually have some experience with such platforms. The Blender Manual, Blender Studio pipeline documentation and Flamenco documentation follow a similar approach. Let’s try to bring this to the developer documentation. This is where it can really shine.

New Platform Proposal

Things should be simple, so they key elements of the new platform are simple as well:

• Material for MkDocs
• Continuous delivery via buildbot
• Edit with preview in Gitea
• Git LFS (Large File Storage) repository
• Possibly pulled with make update
• URL: developer.blender.org/docs

Material for MkDocs is a Material based theme and framework around the MkDocs static site generator. Essentially it turns Markdown files into HTML pages that look great and offer a great browsing experience. It has been chosen over alternatives (such as Sphinx, VitePress and Hugo) since it seems like the best fitting platform. The main features are:

• Easy to set up and use. Developers will have it up and running in an instant.
• Markdown is readable in source, widely used (for example on most developer platforms) and well supported. Many IDEs have builtin Markdown editing and previewing support. Copy & paste from HackMD possible.
• Modern looking, rich in features (client side search, dark mode, … see below) and a big amount of common markdown extensions and plugins.
• Live reloading.
• Python based.
• Good search.

Overall it’s a great documentation experience for both writing and browsing.

Impressions

There is an experimental setup of the new documentation platform available under developer.blender.org/docs.

This is how a documentation page looks like:

Note the top-level navigation at the top, the navigation for the current category on the left, and the current page navigation on the right.

A simple toggle in the header switches between light and dark mode:

Material for MkDocs supports Mermaid to define diagrams as text in Markdown, and render them on the fly using the theme colors:

Who would’ve thought that creating such diagrams can actually be fun (for developers)!

Lastly, Material for MkDocs comes with a great working search feature:

Editing

There are two main workflows for editing pages with the new platform. Online and offline editing.

To edit pages online, simply click the editing icon at the top of a page:

This brings you to Gitea, where you can immediately edit the page’s Markdown.

Although a side-by-side view would be a lot nicer, a simple click on the Preview tab gives a preview of the rendered result:

This is Gitea’s Markdown preview, so there might be smaller differences to the output of Material for MkDocs. But it works extremely well. Plus, as you can see, Gitea supports the Mermaid diagrams too!

The second way to edit is offline. Many editors support Markdown editing and previewing. Here is a setup in Visual Studio Code:

MkDocs supports live reloading. Every time you save the file, the preview on the right gets updated. This is using the live preview extension for Visual Studio Code, but the builtin Markdown preview works well too.

When done, you can simply commit and push the edits (or create a pull request). Either via the Gitea UI when editing online, or manually, using the Git interface of your choice. Buildbot will update the documentation output on developer.blender.org/docs within few minutes.

Since this is Markdown, there is another convenient way to edit the docs: HackMD. Work with live preview and in collaboration with others, then simply copy the result from there to the documentation!

Goodbye Wiki?

The new platform would replace the current wiki. All the documentation, as well as the release notes can move to the new platform. Personal pages like the weekly reports of developers can be moved to personal repositories on Gitea, also using simple Markdown.

Although converters from Wiki syntax to Markdown are available, the transition will require manual work. Mostly to fix issues from the conversion, fix links, add missing images, etc. A few people volunteered to help already, more help would be welcome.

Status & Next Steps

Most preparation work is done:

• Research, testing and experiments
• Buy-in from developers and stakeholders
• Buildbot continuous delivery setup (push changes, Buildbot generates and publishes output to developer.blender.org/docs)
• Hosting on developer.blender.org/docs
• blender-developer-docs Git LFS repository with Material for MkDocs setup & customizations

Follow the latest status on the dedicated task: #116055: New Developer Documentation Platform

This means we are ready to prepare the actual transition, whereby the wiki will be archived. Once this is done we can decide if we want to include the developer documentation as part of the source code or its additional resources (updated together with the source code via make update). This would move the documentation closer to the code, and ensure there’s easy and continuously updated access. Further, we can investigate ways to include internal API documentation (generated from C/C++ API comments using Doxygen) as part of the platform.

---

Building a developer documentation culture is challenging but possible, like other organizations show. Maybe all it takes is a few clever changes, and a bit of pushing to reach some tipping point; and then it snowballs!

The truth is, developers know that they should be writing technical documentation. They just don’t feel empowered enough to do so, there are too many hurdles and too few incentives. It is time to do some changes:

• A new documentation platform that is optimized for the engineer (docs as code) and removes friction from the workflow.
• Make it easy to write documentation by providing clear examples, templates, tools and education.
• Rethink how we write documentation. Make it actually useful for new and experienced developers alike.
• Foster a culture of documentation. Push until using and writing technical documentation becomes part of every day life for a developer — like it happened with unit testing.
• Open up to the community. A new culture can be a lot more exciting for technical writers or the general community to get involved.

Or, how the technical writers who revolutionized technical documentation at Google put it:

First tooling, then culture. […] Focus on the Engineer.

The Knowledge: Towards a Culture of Engineering Documentation – Riana MacNamara, Google

---