Building maintainable step-by-step tutorials with Git

Meteor’s goal is to make building great apps easier than ever before, and the experience for new users is a very important part of that mission. We found that walking people through all of the Meteor’s best features one-by-one is the most effective way to get people excited about the platform. When you try using Meteor to build a simple app and compare it to your previous experience with other platforms, it becomes very clear why a lot of people are very excited about what Meteor brings to the table. I’d like to share with you some of our experiences building a tutorial that is easy for us to update, and easy for the readers to follow. This includes:

1. The maintenance problems that arise when you have a step-by-step code tutorial that you want to update often
2. How we solved that for the Meteor tutorials while improving the tutorial user experience
3. How we can make these tools available for everyone!

The result: a new workflow for updating the tutorial

Before I go into all of the details, here is a simple diagram of what we ended up with — a system that generates tutorial code snippets with code highlighting, context, and links to GitHub from Git commits:

The life of a tutorial commit: in Git, included as a Meteor Blaze template inside the content, and displayed to the reader on meteor.com

Expanding the amount of tutorial content increased the need for maintainability

When the original tutorial was written in October of last year, every step was painstakingly polished and tested to ensure a consistently good experience. In addition to the core work of writing the content and designing the app built by the reader, there were many hours of extra overhead just to make sure the code snippets were consistent and correct. In Meteor 1.2, one of the big messages is going to be first-class support for using Meteor with any frontend rendering library, and in particular we are focusing on supporting Angular and React in addition to Blaze, Meteor’s default rendering engine. This means we will now have three official tutorials, raising the maintenance burden threefold. In the future, we’ll increase the number of tutorials as we add new features to the platform and start talking about the best practices for building an app. We will need to maintain all of this content to keep it up to date with updates to Meteor, changes in best practices, and feedback from new developers.

What’s so hard about maintaining a tutorial?

First, let’s enumerate the parts of a great tutorial:

1. Code snippets that let the user easily follow along by modifying their project according to each step.
2. Text content that describes the goal of each step, tells the developer what to do, and explains what the code means.
3. GitHub repository with all of the code in the tutorial, in a step-by-step format. This way the reader has an “escape hatch” to make sure they have been doing everything correctly — they can check what all of the code is supposed to look like after every step.

The biggest technical challenge in building a good tutorial is keeping all of these parts consistent and easy to update. Here are some problems that come up often if you are editing a step-by-step code tutorial:

Interdependent steps

Let’s say you updated some API, and you now want to change a function call in step 4 out of 12. You would think you could just go to that step, and update the code snippet. But that doesn’t work so well, because each later step builds on that code. It’s possible that step 8 also needs to be changed as a result. Let’s look at an example.

Here’s a code snippet from step 4 from the old tutorial:

Here’s a code snippet from step 8, which depends on the previous code in a way that is hard to detect. The comment references the <header> and <h1> tags from the code above, so if those elements change in the previous steps it will no longer be possible to apply this one.

It can quickly become a nightmare just to ensure that the reader can actually apply each step on top of the previous one.

Hard to tell where the code goes

Most step-by-step code tutorials give the developer small code snippets with directions about where to put the code in their project. You end up with somewhat vague directions, and people who aren’t expert coders or super familiar with the platform might not be able to understand these directions. Here is a code snippet that we used to have in our tutorial:

If you aren’t familiar with Meteor or JavaScript, which you might not be if you are following the beginner tutorial, this could be a very hard step to follow.

Hard to keep code snippets and Github in Sync

We want the reader to be able to follow along with your tutorial by looking at a GitHub repo where you can see all of the context for the changes. It can be very difficult to keep the repository and the code snippets in the tutorial content in sync. You probably need to first update the repo, then update the code snippets, then make sure that all of the links are pointing to the correct commit on GitHub. This process is very laborious, and it’s easy to make mistakes. However, having this escape hatch of being able to see all of the code in context is super valuable so we don’t want to lose it in the name of maintenance.

The solution: use Git as the source of truth

It turns out that there is already a great system for encoding incremental changes to code — Git itself! So we encoded all of the code snippets into a finely-grained step-by-step repository for each tutorial. Here’s an example: https://github.com/meteor/simple-todos/commits/master

Keeping the main source of truth in git immediately solved all of the problems with consistency across steps, and made it trivial to maintain a step-by-step GitHub repository. There was just one question left — how do we generate code snippets for the tutorial content from Git?

Getting the commit data out of Git with git format-patch

It turns out that Git already has great mechanisms for getting commit data. The main ones you might have heard of are git log for listing commits, and git diff for showing the changes for a single commit. But I needed something that would essentially dump the entire state of the repository so that I could parse that into code snippets. The command that worked for me after some hours of digging through Git documentation was git format-patch --stdout, which just dumps all of the data for all of the commits in your repository.

Here’s an example of what the output looks like for one of the tutorial repositories. Now, all I needed to do was parse this file into a format that could be used to render something via JavaScript.

Parsing the Git Patch file into JSON data

Thankfully, the Git patch format isn’t super complicated. It didn’t take too long to write a simple JavaScript parser that could read the format and return a nice JSON representation of all of the commit data. You can see the logic on GitHub. The package I wrote is not yet fully documented or tested, but it works fine for basic use cases.

Annotating the code snippets with consistent step numbers

Git already has a unique identifier for each commit — the SHA of the changes. However, this didn’t work well for us when trying to design a tutorial where we could update the commits frequently. We needed a way to tag each commit so that we could consistently reference it in the tutorial content. The solution was to have a standard format for commit messages that encoded the step number at the beginning of the message — it was simple and worked great.

Using a Meteor Blaze template to render each Diff with context

With all of the data I gained from the Git patch, it was possible to build a much better and smarter code snippet box. Here’s a comparison of an old and new code snippet:

Before:

After:

As you can see, with all of the new information we had from Git, we were able to display which lines were changed, the context around them, a link to a diff for that specific file, and a description of the change being made.

See the entire tutorial with the new code snippets live now on meteor.com!

The new workflow for updating the tutorial

Switching to this new system enabled a much simpler workflow for updating the tutorials. You can now do a simple git rebase to update the repository, run one script to generate the tutorial, and all of the code snippets are updated automatically. I now actually enjoy updating the tutorial rather than dreading it and worrying that I will break something. Now that we’ve explained each part, let’s revisit the diagram that puts it all together:

The life of a tutorial commit: on GitHub, included as a Meteor template inside the content, and displayed to the reader

What’s next?

Now that the data for the tutorial is stored in a structured form, it will be possible to make additional improvements:

1. Showing a side-by-side diff where necessary
2. Automated testing for each step
3. Building a more interactive experience that involves pulling code from GitHub and following along

Before, it was very hard to improve anything because all of the data was stuck in raw markdown files, but this new approach unlocks a whole lot of new potential!

Making this tutorial building workflow available to everyone

The Meteor tutorial source code is completely open-source on GitHub and there’s a fair amount of documentation. But just making something open source doesn’t mean it’s easy for people to use it and contribute. So Uri, Tom, and I — all of whom you can imagine having a real interest in tutorial excellence! — are working on a set of generalized tutorial tools that can work for a wider variety of tutorials. Once we can make it work for some new test cases, I hope that we can release it to the wider community to be used for Meteor tutorials and really any kind of programming tutorial. The ultimate goal is to create a better experience both for tutorial creators and the readers following along!

Follow along and contribute on GitHub in the Meteor tutorial tools repository!