Using Org-mode to Publish a Web Site

Org-mode is an Emacs outliner, markup language, personal organizer, spreadsheet, and literate programming system.¹ [¹ Yes.] I use it for my blog because of the rich markup language it provides. Org-mode's markup language can be compiled to HTML, and its publish system allows systematically compiling a whole interlinked website.

Since Org-mode is an Emacs package, I use Emacs in batch mode to run it, instructing it to open the index page index.org, load the list of tasks publish.el, and run the tasks (org-publish-all).

emacs -Q --batch \
    -l /tmp/www-in/etc/publish.el \
    /tmp/www-in/index.org \
    --funcall org-publish-all

Note that the index page and the list of tasks are located in www-in. That directory is a clean checkout of my website's git repository:

cd /tmp

# Copy repository and make directories
[ -d /tmp/www-in  ] || git clone /home/git/repositories/www.git /tmp/www-in 
[ -d /tmp/www-out ] || mkdir /tmp/www-out

This script is run every time I push to the git repository as a post-receive hook. If /tmp/www-in already exists, it fetches any new changes:

cd /tmp/www-in
unset GIT_DIR
GIT_WORK_TREE=/tmp/www-in/ git fetch
GIT_WORK_TREE=/tmp/www-in/ git checkout -f origin/master

The reason for explicitly setting GIT_WORK_TREE is that that variable, which git reads, is explicitly set in hooks; that’s also why I unset GIT_DIR.

After the hook copies/updates the repository to www-in, it runs the emacs invocation above and then copies the HTML output from /tmp/www-out to the web server root.

cp -r /tmp/www-out/* ~www/www

There’s one final trick involved for updating the hook itself. I keep the hook in the same repository as the website, and copy it over on every push:

cp /tmp/www-in/etc/post-receive.hook /home/git/repositories/www.git/hooks/post-receive

I make sure edits to the hook are in separate commits from edits to anything else, so that I'm always running the current version of the hook.

The hook above ultimately centers around calling the Emacs command org-publish-all. This command executes a list of tasks, each of which calls a different publishing function. My site has three tasks: one master task and then two child tasks for compiling Org files to HTML and for publishing static pages.

(setq org-publish-project-alist
      `(("www"
         :components ("www-pages" "www-static"))
        ("www-pages" ...)
        ("www-static" ...)))

The task for static pages is simple; it simply copies files from www-in to www-out using the org-publish-attachment command:

("www-static"
 :base-directory "/tmp/www-in"
 :base-extension "css\\|js\\|png\\|..."
 :publishing-directory "/tmp/www-out"
 :publishing-function org-publish-attachment
 :recursive t)))

The properties here should be quite self-explanatory; base-extension indicates which files to copy, and I've replaced my long list of file extensions with a ... there, but the list is quite long.² [² Every once in a while I need to add another extension to the list. I should fix that at some point, because every time I forget I have a broken link on my site.]

The task for Org-mode pages is fairly similar, using the org-html-publish-to-html command instead:

("www-pages"
 :base-directory "/tmp/www-in"

 :base-extension "org"
 :recursive t

 :publishing-directory "/tmp/www-out"
 :publishing-function org-html-publish-to-html
 ...
 )

My ellipsis hides a long list of options that format the output—let's talk about that.

The HTML Org-mode generates is ugly, but it gets the job done, and it's pretty configurable using options in the task definition.

The simple tag system on my home page is also implemented using Org-mode.

My home page is an Org-mode file with an outline node for each blog entry. Those blog posts have Org-mode tags:

* Blog

** [[file:blog/major-key.org][Major Key (a puzzle)]]                                                  :misc:
** Age-aware Data Structures                                           :algs:
*** [[file:blog/age-aware/array.org][Age-aware Array Search]] (Part 1 of 4)
*** [[file:blog/age-aware/tree.org][Tree Lookup]] (Part 2 of 4)
** [[file:blog/stream-fusion.org][Stream Fuse Carefully]]                                                :plt:

I make sure that only the top level headings (like “Blog”) are treated as headings with OPTIONS:

#+OPTIONS: H:1 toc:nil num:nil

Finally, I use some JavaScript, which you can find in /etc/blog.js, to make the tag-chooser, and some CSS (found in main.css:216–270) to style it.

I used to generate a full RSS feed for my site using ox-rss. However, recent versions of Org-mode seem to have broken ox-rss. Plus, I like to generate full-text RSS feeds and that meant ox-rss re-exporting every blog post to the RSS file. That re-export took forever and also couldn't be cached. As a result, editing a published post took between ten and twenty minutes—I can't believe I just lived with that for years!

I now generate an RSS file by parsing the front page HTML to get a list of posts and then parsing each post's HTML to get its contents. A little Python script does all of this in seconds, and since Org-mode caches its HTML output, publishing takes a few seconds.

My Python code is quick-and-dirty: it neither parses the HTML nor uses a proper XML library to generate the RSS. Instead, everything is string templates. Luckily, I am not dealing with adversarial HTML, so this seems to work.³ [³ It is often not valid RSS because I embed scripts, SVGs, or IFRAMEs into the document, but RSS readers seem to handle those oddities without issue, so I'll leave it as is.]