Making nice documentation for a Julia Package

[2016-02-17 Wed] [comments on julia-users]

Note that this method is outdated. Use Documenter.jl instead.

Over the last few days I improved the documentation for my Julia package Parameters.jl1. Before I just had a README.md on GitHub (which was probably just fine, considering that the package is tiny). Now I have:

This path is well trodden, at least another 13 packages use this approach to documentation2. However, I was a bit confused on how to go about this and on how the different pieces fit together. Compounding was that I couldn't find one place which describes the whole process. (Although, just now I found this brief description, hidden away in Lexicon.jl's documentation.) Naturally, I thought, I'd better write this up.

The pieces which are needed to make this all work:

(The first an last item are not necessary if you just want local documentation.)

Automatic API-docs generation

Since Julia 0.4, inline doc-strings are supported. The Lexicon.jl package by Michael Hatherly3 can gather these doc-strings and puts them, nicely formatted, into one or several markdown files; it also adds an index file. Here are the generated API and API index file for my package. To build them I use the following build.jl script (slightly adapted from Lexicon's build.jl script):

using Lexicon, Parameters

const api_directory = "api" # where to put the API-docs, relative to this file.
const modules = [Parameters]

cd(dirname(@__FILE__)) do

    # Run the doctests *before* we start to generate *any* documentation.
    for m in modules
        failures = failed(doctest(m))
        if !isempty(failures.results)
            println("\nDoctests failed, aborting commit.\n")
            exit(1) # Bail when doctests fail.
    # also execute examples

    # Generate and save the contents of docstrings as markdown files.
    index  = Index()
    for mod in modules
        update!(index, save(joinpath(api_directory, "$(mod).md"), mod))
    save(joinpath(api_directory, "index.md"), index; md_subheader = :category)

    # Add a reminder not to edit the generated files.
    open(joinpath(api_directory, "README.md"), "w") do f
        print(f, """
        Files in this directory are generated using the `build.jl` script. Make
        all changes to the originating docstrings/files rather than these ones.
        Documentation should *only* be built directly on the `master` branch.
        Source links would otherwise become unavailable should a branch be
        deleted from the `origin`. This means potential pull request authors
        *should not* run the build script when filing a PR.

    info("Adding all documentation changes in $(api_directory) to this commit.")
    success(`git add $(api_directory)`) || exit(1)

This does the following

  • runs the doctests (a feature of Lexicon.jl)
  • runs the example in Parameters.jl/examples/ex1.jl
  • creates the API-doc files in Parameters.jl/docs/api
  • adds a reminder-file to that folder that these files are automatically created
  • adds the created markdown files to git

The auto-generated API-doc files need to be checked-in, as Read the Docs cannot generate them automatically. This also means that the embedded links to the source files link to one commit before the API-doc markdown files are committed: thus it is best to commit the API-doc in a separate commit.

The manual

Apart from the auto-generated API docs, my package also needs the manual proper, also written in markdown. The files are in the Parameters.jl/docs/ folder: index.md gives an overview of the package, manual.md is the actual manual.

Turning it into HTML: MkDocs

As many other Julia and non-Julia packages, I use the static site generator MkDocs (which is geared towards documentation generation). It's super simple, all that is needed is a mkdocs.yml file in the package top directory, mine is this:

site_name:           Parameters.jl
repo_url:            https://github.com/mauro3/Parameters.jl
site_description:    Types with default field values, keyword constructors ...
site_author:         Mauro Werder
theme:               readthedocs
markdown_extensions: [tables, fenced_code]
- Introduction: index.md
- Manual: manual.md
- API: api/Parameters.md
- API Index: api/index.md

This sets a bunch of parameters and then the document structure is specified under the pages: header. All the files which are part of the documentation need to be listed here to be included in the build. (A more complicated example from Docile.jl)

The HTML can then be built with

$ mkdocs build --clean

and it gets put into a directory site/. It can viewed locally with

$ mkdocs serve

which will serve it to the local web site:

Putting the HTML on the web

After checking that all is in order, the HTML can be put on a server. Two options are probably easiest: using gh-pages (or the equivalent for bitbucket/gitlab) or using Read the Docs, a free documentation-hosting service. The former approach is described here and here. I use Read the Docs which has the advantage that the docs are automatically built for you when pushing to the GitHub (or bitbucket/gitlab) repo. However, the API docs still need to be built manually!

The setup with Read The Docs was a breeze, it supports MkDocs out of the box. The only hiccup was that my project needed "Import Manually" as the "+" button did not work.

Don't forget to link to it from the README on the GitHub page. There is even a badge to be had: Sorry, your browser does not support SVG.

Putting it all together

This shell script, in the project root, does it all

# Builds the doc, starts the mkdocs server and opens the page in
# firefox

# Build API-docs
cd docs
julia build.jl
cd ..
# Build mkdocs. Only needed to check locally:
mkdocs build --clean
(sleep 1 && firefox # waits for mkdocs server to start up
mkdocs serve

apart from the git commit and git push.



This post references commit 9e14db5 of Parameters.jl


What I describe here, should work with any Julia package, it does not need to be a registered package (maybe it works for just a plain module too).


Michael Hatherly was instrumental in creating the in-line documentation system of Julia which was modelled after Docile.jl, his other documentation package. In a reply to this post on julia-users, he gave some interesting additional information. The second instrumental person was Mike Innes (@one-more-minute), here at the JuliaCon 2015 Hackathon (when the in-line docs were merged).