autodoc

Table of Contents

Autodoc docs

Automate the publishing of generated docs.

Imagine the following scenario: you have files in a git repo, from which you generate HTML, which you want to make available on-line.

For example: you have source code from which you generate API docs, or you have markdown files that you run through a static site generator.

If you're using Github then getting these resulting files on-line is as easy as pushing to the gh-pages branch. This isn't hard, but you have to first generate the HTML, put it aside, switch branches, commit the change, push it to Github, and switch back. There are many little things that can go wrong in that process. It's also few minutes of your life that you just wasted doing a tedious, mechanical job, every time again.

Enter autodoc, a single shell script that automates this process as best as it can.

1 Installation

The recommended, "evergreen" way of using autodoc is to create a small shell script in your repository, you can call it generate_docs, that looks like this:

#!/bin/bash

# Command that generates the HTML.
export AUTODOC_CMD="lein codox"

# The directory where the result of $AUTODOC_CMD, the generated HTML, ends up. This
# is what gets committed to $AUTODOC_BRANCH.
# export AUTODOC_DIR="gh-pages"

# The git remote to fetch from and push to.
# export AUTODOC_REMOTE="origin"

# Branch name to commit and push to
# export AUTODOC_BRANCH="gh-pages"

\curl -sSL https://raw.githubusercontent.com/plexus/autodoc/master/autodoc.sh | bash

At a minimum you must set AUTODOC_CMD, this is the command that gets called to generate the HTML files.

If you're not comfortable running a shell script straight off the internets, you can also just copy autodoc.sh to your project, and change the variables at the top of the script.

2 Usage instructions

Call ./generate_docs at any time, and the HTML will immediately be updated and made available on-line. This is safe to do no matter the state of your repository. More on why that is below.

The other variables are optional, the values you see in the script above are their defaults.

You can have local changes, untracked files, etc. The script does not actually switch branches, and does not change the current "working tree" beyond running $AUTODOC_CMD. autodoc does use the "git index" (also known as the "staging area"), so this needs to be clean. If you did a git add before then the script will complain and refuse to continue until you commit or reset.

DO WATCH OUT: The $AUTODOC_DIR directory will be removed and re-created on every run. It should only contain generated artifacts, and should be in your .gitignore. If you point this to your homework then it will eat your homework.

3 How it works

The procedure that autodoc follows has been tweaked over time to be as reliable and fool proof as possible. Here is roughly what it does, in order.

  • Check if the git index is clean, otherwise exit
  • Check if $AUTODOC_CMD is set, otherwise exit
  • Do a git fetch, to know what the target branch looks like on the remote (e.g. origin/gh-pages)
  • Clear out $AUTODOC_DIR. It deletes it if already there, and then creates it anew, to make sure you don't commit stale files.
  • Run $AUTODOC_CMD. This should put the HTML artifacts in $AUTODOC_DIR
  • Check if $AUTODOC_DIR is empty. If so then the command didn't create any output, and the script will complain and exit.
  • Create a git "tree object" of the contents of $AUTODOC_DIR
  • Generate a commit message which includes the source branch and commit hash, as well as an overview of any local changes
  • Create a git "commit object" with this tree and message, and with as parent commit the latest commit on the target branch on the target remote, or as an orphan commit if the target branch does not yet exist.
  • Check if the created commit differs from its parent. If not then there was no change and the script exits.
  • Push this commit to AUTODOC_REMOTE/AUTODOC_BRANCH (e.g. origin/gh-pages)
  • Display the commit with diff stats so you get some feedback on what happened

4 Generating API docs

autodoc is really a cry to library maintainers across continents and languages: please publish API docs. Here I'll try to collect some quick tips on how to do that. Please add your favorite language and tool, each section should be a concise summary on how to get started, plus links to the tool's documentation.

If this was wikipedia this part would be called a stub. Do dive in!

4.1 Clojure

The main tool you're looking for is codox, although Marginalia could also be of use.

Use DOC_CMD="lein codox" or DOC_CMD="boot codox <options> target".

5 Improving autodoc

This script has already seen a few iterations of polish, but it can without a doubt be further improved. It needs to be reliable and safe, either doing its job, or bailing out and telling the user what the problem is.

If you can make it more reliable and safe then it currently is, then we'd love to get a pull request.

6 Projects using autodoc

7 Credits

Initially written by Arne Brasseur, improved with the help of Martin Klepsch and Jack Rusher

8 License

Copyright © Arne Brasseur and contributors

Available under the Mozilla Public License 2.0. See LICENSE.

9 Source

#!/bin/bash

# Keep a separate branch of generated API docs.
#
# This script generates API documentation, commits it to a separate branch, and
# pushes it upstream. It does this without actually checking out the branch,
# using a separate working tree directory, so without any disruption to your
# current working tree. You can have local file modifications, but the git index
# (staging area) must be clean.

############################################################
# These variables can all be overridden from the command line,
# e.g. AUTODOC_REMOTE=plexus ./generate_docs

# The git remote to fetch and push to. Also used to find the parent commit.
AUTODOC_REMOTE=${AUTODOC_REMOTE:-"origin"}

# Branch name to commit and push to
AUTODOC_BRANCH=${AUTODOC_BRANCH:-"gh-pages"}

# Command that generates the API docs
#AUTODOC_CMD=${AUTODOC_CMD:-"lein with-profile +codox codox"}
#AUTODOC_CMD=${AUTODOC_CMD:-"boot codox -s src -n my-project -o gh-pages target"}

# Working tree directory. The output of $AUTODOC_CMD must end up in this directory.
AUTODOC_DIR=${AUTODOC_DIR:-"gh-pages"}

############################################################

function echo_info() {
   echo -en "[\033[0;32mautodoc\033[0m] "
   echo $*
}

function echo_error() {
   echo -en "[\033[0;31mautodoc\033[0m] "
   echo $*
}

if [[ -z "$AUTODOC_CMD" ]]; then
    echo_error "Please specify a AUTODOC_CMD, e.g. lein codox"
    exit 1
fi

if ! git diff-index --quiet --cached HEAD ; then
    echo_error "Git index isn't clean. Make sure you have no staged changes. (try 'git reset .')"
    exit 1
fi

VERSION=0013

echo "//======================================\\\\"
echo "||          AUTODOC v${VERSION}               ||"
echo "\\\\======================================//"

MESSAGE="Updating docs based on $(git rev-parse --abbrev-ref HEAD) $(git rev-parse HEAD)

Ran: $AUTODOC_CMD
"

if [[ ! -z "$(git status --porcelain)" ]]; then
  MESSAGE="$MESSAGE
Repo not clean.

    Status:
$(git status --short)

    Diff:
$(git diff)"
fi

# Fetch the remote, we don't care about local branches, only about what's
# currently on the remote
git fetch $AUTODOC_REMOTE

# Start from a clean slate, we only commit the new output of AUTODOC_CMD, nothing else.
rm -rf $AUTODOC_DIR
mkdir -p $AUTODOC_DIR

echo_info "Generating docs"
echo $AUTODOC_CMD | bash

AUTODOC_RESULT=$?

if [[ ! $AUTODOC_RESULT -eq 0 ]]; then
    echo_error "The command '${AUTODOC_CMD}' returned a non-zero exit status (${AUTODOC_RESULT}), giving up."
    exit $AUTODOC_RESULT
fi

if [[ $(find $AUTODOC_DIR -maxdepth 0 -type d -empty 2>/dev/null) ]]; then
    echo_error "The command '$AUTODOC_CMD' created no output in '$AUTODOC_DIR', giving up"
    exit 1
fi

# The full output of AUTODOC_CMD is added to the git index, staged to become a new
# file tree+commit
echo_info "Adding file to git index"
git --work-tree=$AUTODOC_DIR add -A

# Create a git tree object with the exact contents of $AUTODOC_DIR (the output of
# the AUTODOC_CMD), this will be file tree of the new commit that's being created.
TREE=`git write-tree`
echo_info "Created git tree $TREE"

# Create the new commit, either with the previous remote HEAD as parent, or as a
# new orphan commit
if git show-ref --quiet --verify "refs/remotes/${AUTODOC_REMOTE}/${AUTODOC_BRANCH}" ; then
    PARENT=`git rev-parse ${AUTODOC_REMOTE}/${AUTODOC_BRANCH}`
    echo "Creating commit with parent refs/remotes/${AUTODOC_REMOTE}/${AUTODOC_BRANCH} ${PARENT}"
    COMMIT=$(git commit-tree -p $PARENT $TREE -m "$MESSAGE")
else
    echo "Creating first commit of the branch"
    COMMIT=$(git commit-tree $TREE -m "$MESSAGE")
fi

echo_info "Pushing $COMMIT to $AUTODOC_BRANCH"

# Rest the index, commit-tree doesn't do that by itself. If we don't do this
# `git status` or `git diff` will look *very* weird.
git reset .

# Push the newly created commit to remote
if [[ ! -z "$PARENT" ]] && [[ $(git rev-parse ${COMMIT}^{tree}) == $(git rev-parse refs/remotes/$AUTODOC_REMOTE/$AUTODOC_BRANCH^{tree} ) ]] ; then
    echo_error "WARNING: No changes in documentation output from previous commit. Not pushing to ${AUTODOC_BRANCH}"
else
    git push $AUTODOC_REMOTE $COMMIT:refs/heads/$AUTODOC_BRANCH
    # Make sure our local remotes are up to date.
    git fetch
    # Show what happened, you should see a little stat diff here of the changes
    echo
    git log -1 --stat $AUTODOC_REMOTE/$AUTODOC_BRANCH
fi

Author: Arne Brasseur

Created: 2017-12-21 Thu 21:11

Validate