Using Emacs and Org-mode as a static site generator
Published: December 29, 2024
Introduction
For reasons that I don't yet fully understand11 That was a lie. Of course I know why. It's because playing with this website is fun and during the holidays I actually have time to have fun 😊. By the way, this is the first article that I am writing with my new, tufte-css inspired style. I hope it works out. , I spend a considerable amount of time during the holidays tweaking and updating this website. The resulting static site generator has now grown in complexity to a point where my previous posts are woefully outdated and rather confusing to read. So why not make an updated "This is how this site is built" post for the year 2025? Alright, saddle up.
The first iteration of this website was generated using Pelican. This was mostly due to the fact that I was not an Emacs user yet and that Markdown seemed like a reasonable choice for a simple static site at the time22I still think that Markdown + Pelican is a decent choice for a personal website. But if you are like me, that's juuust not emacsy enough…. I rebooted the website at some point in 2018, which coincided with me moving from the old Markdown + Pelican setup to using the Org-mode publishing features exclusively. The current design of the site is essentially the same as this first iteration, but the build process and especially the CSS style sheets have changed considerably over the years. At this point, I believe that I have arrived at a somewhat stable point with the build system, so now might be a good time to give an in-depth overview of how this website is built.
The build process—a 1000-foot view
Project structure
The site starts as a run-of-the-mill private GitHub repo containing all of the source files:
website ┣ bib // BibTeX files for my CV and Publications page ┣ bin // Python source code for CSS and JS minifiers ┣ blog // .org files for blog articles ┣ css // CSS and JS source files ┣ cv // LaTeX project for my CV ┣ dl // misc. files to download ┣ html // misc. HTML (website header and footer) ┣ img // images ┣ lisp // Emacs Lisp code to build the site ┣ pages // .org files for non-blog pages, like index.html ┗ README.org // smol README
With the exception of the actual Emacs source code, this is entirely self-contained; there are no other external dependencies33This is important to me. I even make sure to serve all fonts (and even MathJax) from my server. I just don't like the idea of some external content going away and breaking my site.. From here, I use the Org-mode publishing machinery to generate the HTML you are viewing. At first, I started out using the standard Org-mode HTML export code, but I have since resulted to defining my own HTML export backend to generate this site. I will describe this backend in detail below.
A single page of the site
A blog article or static page (static pages are the landing page, the about page, etc.) starts as an org-mode file either in the pages or blog subdirectory. As an example, the listing below contains the first few paragraphs of this post as I am writing it.
A screenshot of me editing this article in Emacs:
Org-mode source for this blog post (abridged)
#+title: Using Emacs and Org-mode as a static site generator #+AUTHOR: Dennis Ogbe #+EMAIL: [email protected] #+DATE: <2024-12-29 Sun> * Introduction :PROPERTIES: :CUSTOM_ID: intro :END: #+BEGIN_PREVIEW For reasons that I don't yet fully understand[fn::That was a lie. Of course I know why. It's because playing with this website is *fun* and during the holidays I actually have time to have fun 😊. By the way, this is the first article that I am writing with my new, tufte-css inspired style. I hope it works out.], I spend a considerable amount of time during the holidays tweaking and updating this website. The resulting static site generator has now grown in complexity to a point where my previous posts are woefully outdated and rather confusing to read. So why not make an updated "This is how this site is built" post for the year 2025? Alright, saddle up. #+END_PREVIEW The first iteration of this website was generated using Pelican. This was mostly due to the fact that I was not an Emacs user yet and that Markdown seemed like a reasonable choice for a simple static site at the time[fn::I still think that Markdown + Pelican is a decent choice for a personal website. But if you are like me, that's juuust not emacsy enough...]. I rebooted the website at some point in 2018, which coincided with me moving from the old Markdown + Pelican setup to using the Org-mode publishing features exclusively. The current design of the site is essentially the same as this first iteration, but the build process and especially the CSS style sheets have changed considerably over the years. At this point, I believe that I have arrived at a somewhat stable point with the build system, so now might be a good time to give an in-depth overview of how this website is built. * The build process---a 1000-ft view :PROPERTIES: :CUSTOM_ID: build-process :END: The site starts as a run-of-the-mill private GitHub repo containing all of the source files: #+begin_src fundamental website ┣ bib // BibTeX files for my CV and Publications page ┣ bin // Python source code for CSS and JS minifiers ┣ blog // .org files for blog articles ┣ css // CSS and JS source files ┣ cv // LaTeX project for my CV ┣ dl // misc. files to download ┣ html // misc. HTML (website header and footer) ┣ img // images ┣ lisp // Emacs Lisp code to build the site ┣ pages // .org files for non-blog pages, like index.html ┗ README.org // smol README #+end_src With the exception of the actual Emacs source code, this is entirely self-contained; there are no other external dependencies[fn::This is important to me. I even make sure to serve all fonts (and even MathJax) from my server. I just don't like the idea of some external content going away and breaking my site.]. From here, I use the Org-mode publishing machinery to generate the HTML you are viewing. At first, I started out using the standard Org-mode HTML export code, but I have since resulted to defining my own HTML export backend to generate this site. I will describe this backend in detail below. A blog article or static page (static pages are the landing page, the about page, etc.) starts as an org-mode file either in the =pages= or =blog= subdirectory. As an example, the listing below contains the first few paragraphs of this post as I am writing it.
There are already a few idiosyncrasies that I can comment on at this point. First, note the #+HTML_HEAD_EXTRA: line at the top of the file:
This line is adding the margin-style.css stylesheet to the generated HTML. Normal blog posts and pages on this site use the style.css sheet, which is linked using the code in the generator script I will describe below. However, this post (and, e.g., my test page) use this new style where I am chopping off 30% of the text area on the right side for footnotes and margin notes. This style is inspired by tufte-css and David Mackay's absolutely amazing textbook on Information Theory, Inference, and Learning Algorithms, one of my all-time favorite books. It's all hand-coded/copied CSS of varying quality, but it seems to work fine so far. This effectively lets me have two different styles for pages on this site. Either the "normal", full-width style that you are used to, or the new style mimicking the works of the great Edward Tufte.
Another non-standard aspect of the raw .org files in my repo is contained in the #+BEGIN_PREVIEW ... #+END_PREVIEW block. I use this custom block to mark the part of the post which I want to show as a preview on the blog landing page (or sitemap, as it is called by the org-publish infrastructure). As we will see below, I have some custom code in my sitemap generator which extracts the text from this block and inserts it together with the publishing date and a "Read More…" link on the sitemap page. I have had a custom sitemap generator function since the first version of the website, and this preview feature was actually the first custom code I added.
\(\LaTeX\) CV and BibTeX publications list
Other than the previously-mentioned quirks and features, my article source files are pretty standard. However, since I am also building my academic CV from this repo, the cv subdirectory contains a \(\LaTeX\) project which I usually build using a small Makefile and Latexmk. My CV, displayed in the margin, is a pretty standard, relatively boring \(\LaTeX\) document, so I don't have much to comment on there other than maybe on the list of publications.
I keep a list of conference papers, journal papers, and other invited talks as .bib BibTeX files in this project. Since I don't want to repeat myself (especially not when it comes to mangling BibTeX files), I use those .bib files both to generate a list of publications in my CV as well as to generate the list of publications on my publications page44For a very long time, I used bibtex2html for this purpose. This required me to carry around a binary for a program written in OCaml (!). Fortunately, now that citeproc.el exists and is integrated with the new Org-mode citation system, I can generate beautiful HTML bibliographies in pure elisp.. This keeps both in sync and I only have to keep track of a single source of truth.
I'll comment on the Emacs Lisp code for this later. Overall, I am very impressed with the new Org-mode citation system and how relatively flawless the HTML export works. For my publications page, I did end up defining my own XML citation style based on the IEEE style, because I don't like how the official IEEE citation style obliterates title casing for the journal and conference names55IYKYK.. So that's another source file in the repo for the website, but this is a) human-redable XML and b) I haven't had to touch this file since I created it.
Building the site using make and Emacs batch mode
A very simple Makefile ties the build together. It builds the CV and the website and optionally "deploys" both to a directory from which it can be served via HTTP:
Makefile
# build the CV and website DEPLOYDIR=/path/to/webroot all: cv site cv: cd cv && $(MAKE) cp -f cv/cv.pdf dl/ site: bib/*.bib blog/*.org pages/*.org lisp/*.el css/*.in html/*.html dl/*.* img/*.* -mkdir -p www cd lisp && WEBSITE_OUT_DIR=$(shell readlink -f www) BUILD_TYPE=full ./build.sh deploy: cv site cp -f -u -r -v www/* $(DEPLOYDIR) chmod -R 755 $(DEPLOYDIR) clean: cd cv && $(MAKE) clean rm -rf www/* rm -rf blog/blog.org .PHONY: all cv clean
build.sh (called from the Makefile)
#!/bin/sh BASEDIR=$(cd $(dirname "$0"); pwd) [ -z "$WEBSITE_OUT_DIR" ] && \ OUTDIR=$(mktemp -d) || \ OUTDIR="$WEBSITE_OUT_DIR" WEBSITE_OUT_DIR="$OUTDIR" \ WEBSITE_BUILD_TYPE="$BUILD_TYPE" \ emacs --batch -l "./project.el" --eval="(org-publish \"blog\" t)" [ -z "$WEBSITE_OUT_DIR" ] && \ echo "Output written to $OUTDIR." \ || true
I don't remember why I split the build between the bash script and the Makefile. I could probably refactor that. But it works and is not really presenting a problem.
The Makefile makes the distinction between make-ing the website and deploy-ing it. The deploy step simply copies the output of the make step (the final website) to the folder that my web server points to66Actually, I am also serving the intermediate www directory to the web. But I am serving that behind a password-protected subdomain to allow me to preview the site while I am editing it.. My Emacs configuration has some helper functions and keybindings to kick off a website build. Here, there are basically two different ways I can rebuild the site from Emacs. The first and simple method is to use Emacs' async-shell-command to shell out to run the make command. This rebuilds the entire site and as of the time of this writing takes about 20 seconds. Another method I added recently is to use emacs-async to spawn another Emacs process directly and to only re-publish the current .org file. This only takes a couple of seconds and makes editing a blog post much nicer. The Emacs Lisp code for this looks roughly like the following:
(defvar do.website/base-dir nil "Path to the base directory of the website.") (defconst do.website/project-script (file-name-concat do.website/base-dir "lisp/project.el") "Path to the project.el script containing my static site generator.") ;; ... snip ... (defun do.website/republish-current-file () "Re-publish the currently edited file asynchronously." (interactive) (save-buffer) (async-start `(lambda () (load-file ,do.website/project-script) (org-publish-file ,(buffer-file-name))) (lambda (result) (beep) (message "republish-current-file: Done"))))
When editing .org files for my website, I have the full make command bound to C-c m and the command to republish the current file only bound to C-c c (which happens to be my generic "compile this" keybinding in other modes).
A deep dive into the build script
Let's recap what we have gone over so far: With the exception of my CV, which is using \(\LaTeX\), I am using plain Emacs and Org-mode as my static site generator. I am using the org-publish infrastructure to generate HTML from .org files and other source files. All of this is configured and run from a self-contained, roughly 750 line long Emacs Lisp script. In this section, let's examine this build script from top to bottom. This section will be filled with lots of code listings and commentary, so don't say I didn't warn you!
Preamble and early setup
;; This file defines the org-publish project for my web site. -*- eval: (flycheck-mode -1) -*- ;; I can either run this from the build.sh script / Makefile or ;; evaluate this buffer and publish from within Emacs while editing ;; the page. (defun generate-website (arg) "Generate my website. Call with prefix argument for a complete rebuild." (interactive "P") (message "Generating website for staging...") (if arg ;; force rebuild everything (org-publish "blog" t nil) ;; only rebuild what changed (org-publish "blog" nil nil)) (message "Done. Check output in %s" my-website-out-dir))
Nothing much to see on the first few lines. This is just a function that calls org-publish interactively. It is not used much, probably a candidate for deletion.
I promise this gets better… I used to be pretty terrible at Emacs Lisp back in the day.
;; I am not sure why I have to do it this way... This snippet finds ;; the parent directory of this file, which is the base directory of ;; the project. (setq my-website-base-dir (file-name-as-directory (file-name-directory (directory-file-name (file-name-directory (or load-file-name buffer-file-name)))))) ;; we do not need backup files for this (setq make-backup-files nil) ;; set up the rest of the directory tree (defmacro my-website-set-path-var (name) (list 'setq (intern (format "my-website-%s-dir" name)) (list 'file-name-as-directory (concat my-website-base-dir name)))) (my-website-set-path-var "bin") (my-website-set-path-var "bib") (my-website-set-path-var "blog") (my-website-set-path-var "css") (my-website-set-path-var "cv") (my-website-set-path-var "dl") (my-website-set-path-var "html") (my-website-set-path-var "img") (my-website-set-path-var "lisp") (my-website-set-path-var "pages") ;; we pull the output directory out of an environment variable. If this ;; variable is not set, we bail (setq my-website-out-dir (getenv "WEBSITE_OUT_DIR")) (unless my-website-out-dir (setq my-website-out-dir (file-name-concat my-website-base-dir "www")) (message "Using default WEBSITE_OUT DIR: %s" my-website-out-dir)) (setq my-website-out-dir (file-name-as-directory my-website-out-dir))
These early lines are remnants of my very early clunky attempts of doing this. If I were to re-write this today, I'd probably find a nice way to find the path to the project directory and I'd definitely find a less roundabout way of setting variables for the different subdirectories. I also wouldn't use environment variables. These are some of my earlier attempts at Emacs Lisp, and they have aged accordingly. But they still work, so rewriting them is not a priority.
;; [2022-12-27 Tue] This is now compatible with emacs 28.1. it ;; requires the `htmlize' and `org-contrib' packages. (package-initialize) (require 'org) (require 'htmlize) (require 'org-contrib) (require 'ox-html) (require 'ox-rss) (require 'oc-csl) (require 'citeproc) (require 'parsebib) (require 'rx) ;; fix obsolete `font-lock-reference-face' ;; (see https://lists.nongnu.org/archive/html/emacs-devel/2022-09/msg01022.html) (setq font-lock-reference-face font-lock-constant-face) ;; re-build the entire project if $WEBSITE_BUILD_TYPE=FULL (when (and (getenv "WEBSITE_BUILD_TYPE") (string-equal (downcase (getenv "WEBSITE_BUILD_TYPE")) "full")) (setq org-publish-use-timestamps-flag nil))
I now load all packages using require. This could move to use-package, but it works fine in fresh Emacs processes. I don't remember whether the bug mentioned here is still open or not, but it probably doesn't matter much. Finally, it looks like I am using another environment variable to determine whether to republish all files or to use Org's built-in machinery to detect which files have changed. Of note here is that this build script depends on the htmlize and org-contrib packages77So maybe I need to amend my earlier statement about the source repo being fully self contained modulo Emacs. I guess it is fully self contained modulo Emacs, any Emacs dependencies, htmlize, and org-contrib..
Misc org-export settings
(setq org-cite-export-processors `((t . (csl ,(file-name-concat my-website-bib-dir "ieee-mod.csl")))))
Here I point Org mode to my custom CSL style, which I derived from the IEEE style. I've commented on the new Org-mode citation system already. It is very useful.
(setq org-export-html-coding-system 'utf-8-unix) (setq org-html-htmlize-output-type 'css) (setq org-html-doctype "xhtml5") (setq org-html-html5-fancy t) ;; speed up export by defaulting to not evaluating babel block. to skip this, we use ":eval yes" in the source block header (setq org-confirm-babel-evaluate nil) (add-to-list 'org-babel-default-header-args '(:eval . "never-export")) (setq org-export-time-stamp-file nil) (setq org-rss-use-entry-url-as-guid nil) ;; massage org-time-stamps (setq org-time-stamp-custom-formats '("%B %d, %Y" . "%B %d, %Y, %H:%M")) (setq org-display-custom-times t) (setq org-export-date-timestamp-format (car org-time-stamp-custom-formats)) (defun my-org-export-ensure-custom-times (backend) (setq-local org-display-custom-times t) (setq-local org-export-date-timestamp-format (car org-time-stamp-custom-formats))) (add-hook 'org-export-before-processing-hook 'my-org-export-ensure-custom-times) ;; <2024-03-10 Sun> Need to fix `org-html-timestamp', since it calls ;; `org-timestamp-translate', which does not strip brackets.. why. (defun my-format-timestamp (ts) "Format a time stamp using org-mode according to `org-time-stamp-custom-formats'." (org-format-timestamp ts (org-time-stamp-format (org-timestamp-has-time-p ts) 'no-brackets t) t)) (defun override-org-html-timestamp (timestamp _contents info) (let ((value (org-html-plain-text (my-format-timestamp timestamp) info))) (format "<span class=\"timestamp-wrapper\"><span class=\"timestamp\">%s</span></span>" (replace-regexp-in-string "--" "–" value)))) (advice-add 'org-html-timestamp :override #'override-org-html-timestamp)
In this section I am setting some variables of the Org-mode HTML export backend. For one, I am aspiring for this site to be XHTML5 compliant. I think that it mostly is. Next, I disable evaluation of source code blocks to speed up the export. If I want to evaluate a source block (which I sometimes do), then I need to opt-in using an :eval yes block header argument.
For some reason, I need to mess with the org-html-timestamp function, because it doesn't strip the brackets ([ and ] for org-time-stamp-inactive, < and > for org-time-stamp). Somehow, the last time I worked on this, I could never get my time stamps to appear as January 1, 1970 for example. So it looks like I had to resort to some hack here. It may be time to check whether this is still an issue and/or submit an upstream fix.
Website header and footer
;; our common CSS and JS for each page. this is used in the :html-head argument. (setq website-head (concat "<script src='/res/code.js'></script>\n" ; some very simple javascript "<link rel='stylesheet' href='/res/style.css'>\n" ; minified style sheet "<link rel='shortcut icon' type='image/x-icon' href='/img/favicon.ico'>\n" ; favicon "<link rel='alternate' type='application/rss+xml' title='RSS Feed for ogbe.net' href='/blog-feed.xml'>\n")) (defun website-header (info) (with-temp-buffer (insert-file-contents (concat my-website-html-dir "header.html")) (buffer-string))) (defun website-footer (info) (with-temp-buffer (insert-file-contents (concat my-website-html-dir "footer.html")) (buffer-string))) (defun my-blog-org-export-format-drawer (name content) (concat "<div class=\"drawer " (downcase name) "\">\n" "<h6>" (capitalize name) "</h6>\n" content "\n</div>"))
Here I define the HTML header and footer for the website. The variable website-head holds the links to my JavaScript sources, my default CSS style, my favicon, and the RSS Feed. This website does not use a lot of JavaScript (check it), the only thing that I am currently loading is an EventListener which automatically opens the Footnotes section on narrow screens. This is part of my attempt at a "responsive" style for this site.
The remainder of this section just loads my header and footer from .html files on disk. Again, you can use your browser's "View Source" functionality to inspect them. There is nothing special here. The last function ensures that any custom Org drawers are exported using a H6 headline. This is rarely used, since I don't really use Org drawers when writing.
CSS and JavaScript minification
(defun minify (what file) (let ((tool (file-name-concat my-website-bin-dir (cond ((eq what 'js) "rjsmin.py") ((eq what 'css) "rcssmin.py") (t (error "unknown format: %s" what)))))) (shell-command-to-string (format "python3 %s < %s" tool file)))) (defun minify-css (file) (minify 'css file)) (defun minify-js (file) (minify 'js file))
This is the first spot where it gets interesting. In an effort to save some bandwidth, I minify the CSS and JavaScript that I serve as part of the website. Emacs does not have a built in CSS minifier (that I know of). In earlier versions of the site, I used CSSTidy to minify my CSS. This was fine, but it required me to carry around the CSSTidy binary in my repo, which also meant that the website could only be built on x86 Linux machines. Once I added some JavaScript to the website, I hit a roadblock, since CSSTidy didn't support JavaScript. After searching around, I found rcssmin and rjsmin, two fast CSS and JavaScript minifiers which are contained in a single Python file each. This is the perfect solution, save for some native Emacs Lisp implementation. The code in this snippet just wraps the call to the Python interpreter and defines the two helper functions minify-css and minify-js. I use them during the export process later.
MathJax
(setq my-blog-local-mathjax '((path "/mathjax/tex-chtml.js") (scale "1.0") (align "center") (indent "2em") (tagside "right") (tags "all") (mathml nil))) (setq my-blog-local-mathjax '((path "/mathjax/tex-chtml.js"))) (setq my-blog-extra-mathjax-config (concat "<script>" "MathJax = { tex: { inlineMath: [['$', '$'], ['\\\\(', '\\\\)']], tags: 'ams' }, svg: { fontCache: 'global' }};" "</script>"))
This snippet sets some MathJax constants to be used later. Nothing special, other than the fact that I serve MathJax from my server instead of a CDN.
Helper functions
(defun my-blog-get-preview (file) "The comments in FILE have to be on their own lines, prefereably before and after paragraphs." (with-temp-buffer ;; I admit that this is a very dirty hack. but we want to sanitize the ;; previews a little. FIXME: add things to remove from the preview blurb as ;; I use them. (let ((raw-preview-string (with-temp-buffer (insert-file-contents file) (goto-char (point-min)) (let ((beg (+ 1 (re-search-forward "^#\\+BEGIN_PREVIEW$"))) (end (progn (re-search-forward "^#\\+END_PREVIEW$") (match-beginning 0)))) (buffer-substring beg end))))) (insert raw-preview-string) ;; remove any footnotes from the preview blurb (goto-char (point-min)) (replace-regexp "\\[fn:.*?\\]" "") (buffer-string))))
The first helper function is the aforementioned function which extracts the posts' preview string using my custom block definition. It used to be a relatively straightforward regex search, but I had to hack this once I realized that I needed to remove any footnotes or margin notes from the preview for the sitemap. For now, this can only remove footnotes. Whenever I write a post with a margin note in the preview, I'll add a feature to remove the margin note.
(defun my-blog-parse-sitemap-list (l) "Convert the sitemap list in to a list of filenames." (mapcar #'(lambda (i) (let ((link (with-temp-buffer (let ((org-inhibit-startup nil)) (insert (car i)) (org-mode) (goto-char (point-min)) (org-element-link-parser))))) (when link (plist-get (cadr link) :path)))) (cdr l)))
This function, covered in a previous post on the topic, converts the list of links to blog posts are they are handed to the sitemap function into a list of filenames. It is used in the sitemap generator.
The sitemap generator function
(defun my-blog-sitemap (title list) "Generate the landing page for my blog. This actually generate two pages. one for the RSS output and one for the actual landing page." (let* ((filenames (my-blog-parse-sitemap-list list)) (project-plist (assoc "blog-articles" org-publish-project-alist)) (rss-output-file (file-name-concat my-website-blog-dir "blog-feed.org"))) ;; first generate the org file for the RSS output (with-temp-buffer (insert "#+AUTHOR: Dennis Ogbe\n") (insert "#+EMAIL: [email protected]\n") (insert (mapconcat (lambda (file) (let* ((abspath (file-name-concat my-website-blog-dir file)) (relpath (file-relative-name abspath my-website-base-dir)) (title (org-publish-find-title file project-plist)) (date (format-time-string (car org-time-stamp-formats) (org-publish-find-date file project-plist))) (rss-permalink (file-name-sans-extension relpath)) (preview (my-blog-get-preview abspath))) (with-temp-buffer ;; need to make this an org-mode buffer (org-mode) ;; insert the link to the article as h2 (insert (concat "* [[file:" relpath "][" title "]]\n")) ;; add properties for `ox-rss.el' here (org-set-property "RSS_PERMALINK" rss-permalink) (org-set-property "PUBDATE" date) (org-set-property "RSS_TITLE" title) ;; insert the preview (insert preview) (buffer-string)))) filenames "\n")) (write-file rss-output-file)) ;; now generate the actual Blog landing page (with-temp-buffer ;; insert a title and save (insert "#+TITLE: Blog - Dennis Ogbe's Personal Website\n") (insert "#+AUTHOR: Dennis Ogbe\n") (insert "#+EMAIL: [email protected]\n\n") (insert "#+OPTIONS: title:nil\n") (insert "#+begin_export html\n<h1>Blog</h1>\n#+end_export\n\n") ; this way the browser's tab shows ^ but the site shows < ;; only display a full preview for the first 10 posts (let* ((nfull 10) (title-preview (seq-subseq filenames 0 (1- nfull))) (title-only (seq-subseq filenames nfull))) (insert (mapconcat (lambda (file) (let* ((abspath (file-name-concat my-website-blog-dir file)) (relpath (file-relative-name abspath my-website-base-dir)) (title (org-publish-find-title file project-plist)) (date (format-time-string (car org-time-stamp-custom-formats) (org-publish-find-date file project-plist))) (preview (my-blog-get-preview abspath))) (with-temp-buffer ;; insert the link to the article as h2 (insert (concat "* [[file:" relpath "][" title "]]\n")) ;; insert the date, preview, and read more link (insert (concat "/Published: " date "/\n\n")) (insert preview) (insert "\n") (insert (concat "[[file:" relpath "][/Read More.../]]\n")) (buffer-string)))) title-preview "\n-----\n")) ;; For the remaining articles, show them as a list (insert "\n-----\n") (insert "#+begin_export html\n<h1>Archive</h1>\n#+end_export\n\n") (insert "#+begin_blog_archive\n") (insert (mapconcat (lambda (file) (let* ((abspath (file-name-concat my-website-blog-dir file)) (relpath (file-relative-name abspath my-website-base-dir)) (title (org-publish-find-title file project-plist)) (date (format-time-string "%Y-%m-%d" (org-publish-find-date file project-plist)))) (format "- %s: [[file:%s][%s]]" date relpath title))) title-only "\n")) (insert "\n#+end_blog_archive\n")) (buffer-string))))
My rather long sitemap generator function actually generates two sitemaps. The first is an .org file which is the input to the ox-rss.el export backend which generates my RSS feed. It adds a drawer with the properties RSS_PERMALINK, PUBDATE, and RSS_TITLE to each entry.
The second .org file that is generated is the main sitemap. Here, I first add my regular header info to stay consistent with the other pages of the website. I then add the preview for each blog post. As an added wrinkle, I only add the preview for the most recent ten posts. All posts older than that get listed under a special "Archive" section to avoid filling up the page.
Pre- and post-processing
(defun my-blog-pages-preprocessor (project-plist) (message "In the pages preprocessor.")) (defun my-blog-pages-postprocessor (project-plist) (message "In the pages postprocessor.")) (defun my-blog-articles-preprocessor (project-plist) (message "In the articles preprocessor.")) (defun my-blog-articles-postprocessor (project-plist) "Massage the sitemap file and move it up one directory. for this to work, we have already fixed the creation of the relative link in the sitemap-publish function" (let* ((sitemap-fn (concat (file-name-sans-extension (plist-get project-plist :sitemap-filename)) ".html")) (sitemap-olddir (plist-get project-plist :publishing-directory)) (sitemap-newdir (expand-file-name (concat (file-name-as-directory sitemap-olddir) ".."))) (sitemap-oldfile (expand-file-name sitemap-fn sitemap-olddir)) (sitemap-newfile (expand-file-name (concat (file-name-as-directory sitemap-newdir) sitemap-fn)))) (with-temp-buffer (goto-char (point-min)) (insert-file-contents sitemap-oldfile) ;; massage the sitemap if wanted ;; delete the old file and write the correct one (delete-file sitemap-oldfile) (write-file sitemap-newfile))))
Most of these pre- and post-processor functions do nothing, with the exception of the post-processor of the articles subproject, which just moves the freshly generated sitemap file around.
(defun my-website-rss-postprocessor (info) "Add a stylesheet tag to the RSS feed exported by `ox-rss.' This is included in more recent versions of `ox-rss', but seemingly did not make it into Emacs 28.1." (let ((outfile (file-name-concat (plist-get info :publishing-directory) "blog-feed.xml")) (stylesheet (plist-get info :my-rss-stylesheet))) (when stylesheet (with-temp-buffer (insert-file-contents outfile) (goto-char (point-min)) (replace-regexp (rx "<?xml version=\"1.0\" encoding=\"" (* anything) "\"?>") (format "\\&\n<?xml-stylesheet type=\"text/xsl\" href=\"%s\"?>" stylesheet)) (write-file outfile)))))
This postprocessor runs on the RSS feed XML generated by ox-rss. Happy acronym salad: "XLS loads CSS to style RSS" It adds my custom XSL style sheet rss.xsl to my RSS feed to style it. This sheet, in return, then loads a CSS sheet for the RSS feed called ../res/rss.css to make the RSS feed look somewhat like the rest of the website.
(defun my-blog-articles-add-subheader (plist filename pub-dir) "Called after the publishing function, this adds a subheader to each blog post." (let* ((outfile (file-name-concat pub-dir (concat (file-name-base filename) ".html"))) (date (format-time-string (car org-time-stamp-custom-formats) (org-publish-find-date filename plist))) (author (org-publish-find-property filename 'author plist)) ; unused (re (regexp-quote "<h1 class=\"title\">"))) ;; open the outfile and splice publishing date into the generated HTML (with-temp-buffer (insert-file-contents outfile) (when (re-search-forward re nil t) (end-of-line) (insert (format "\n<div class=\"subheader\"><p><i>Published: %s</i></p></div>" date))) (write-file outfile))))
This extra post-processor is run after the main publishing function88To see how this is configured, check the org-publish-project-alist variable below and inserts the publication date on each page. I wrote this just so I don't have to set the publication date twice for each article.
Processing CSS and JavaScript sources
(defvar my-website-cssjs-files nil "A list of alists defining which files from the `css' directory to concatenate or minify.") (setq my-website-cssjs-files '(;; the main CSS style for pages ((output . "style.css") (contents . ("fonts.css.in" "code.css.in" "main.css.in")) (processor . minify-css)) ;; the CSS style for pages with margin ((output . "margin-style.css") (contents . ("margin.css.in")) (processor . minify-css)) ;; the CSS style for the RSS feed ((output . "rss.css") (contents . ("fonts.css.in" "code.css.in" "rss.css.in")) (processor . minify-css)) ;; the JavaScript code ((output . "code.js") (contents . ("code.js.in")) (processor . minify-js)))) (defun my-website-process-cssjs (project-plist) "Process the variable `my-website-cssjs-files' and produce outputs in `my-website-css-dir'." (let ((base-dir (expand-file-name (plist-get project-plist :base-directory)))) (mapcar (lambda (spec) (let ((output (file-name-concat base-dir (alist-get 'output spec)))) (with-temp-buffer (insert (mapconcat (symbol-function (alist-get 'processor spec)) (mapcar (lambda (f) (file-name-concat base-dir f)) (alist-get 'contents spec)) "\n")) (write-file output)))) my-website-cssjs-files))) (defun my-website-clean-cssjs (project-plist) "Clean up the generated outputs in `my-website-css-dir'." (let ((base-dir (expand-file-name (plist-get project-plist :base-directory)))) (mapcar (lambda (spec) (let ((output (file-name-concat base-dir (alist-get 'output spec)))) (delete-file output))) my-website-cssjs-files)))
This one is kind of fun: The CSS files I have been mentioning are actually concatenations of the minified true source files in the repo. In the variable my-website-cssjs-files, I define a list of alists, where each alist contains directions on how to bake an output file. For example, take the output file style.css, the main CSS style for the website. The alist is:
((output . "style.css") (contents . ("fonts.css.in" "code.css.in" "main.css.in")) (processor . minify-css))
The code in my-website-process-cssjs takes this recipe and creates the output file by first running the processing function (minify-css from earlier) on each of the files listed in contents and then concatenating the result. Together, the functions my-website-process-cssjs and my-website-clean-cssjs make up the :preparation-function and :completion-fnuction for the RSS subproject.
Generating a list of publications using citeproc.el
(defun generate-bib-html (relfile) (let* ((infile (file-name-concat my-website-bib-dir relfile)) (csl-style (file-name-concat my-website-bib-dir "ieee-mod.csl")) (csl-locale (file-name-concat my-website-bib-dir "locales-en-US.xml")) (bib-entries ;; get all keys in the file as a list (let (keys) (maphash (lambda (k v) (push k keys)) (with-temp-buffer (insert-file-contents infile) (parsebib-collect-bib-entries :fields '("key")))) keys)) (cproc (citeproc-create csl-style (citeproc-hash-itemgetter-from-any infile) (citeproc-locale-getter-from-dir my-website-bib-dir) "en-US" t))) ;; add all keys to the citation processor (citeproc-append-citations (mapcar (lambda (k) (citeproc-citation-create :cites `(((id . ,k)))) ) bib-entries) cproc) ;; render the bibliography (with-temp-buffer (insert (car (citeproc-render-bib cproc 'html nil nil nil))) ;; highlight my name ;; FIXME should be possible with ;; `citeproc-name-postprocess-functions', but this seems to not ;; be supported in my version. (see ;; https://andras-simonyi.github.io/org-csl-cv-bib-tutorial/) (goto-char (point-min)) (replace-regexp (rx "D." (or (* " ") (* whitespace)) "Ogbe") "<b>D. Ogbe</b>") (buffer-substring (point-min) (point-max)))))
This function converts a .bib file into HTML code and in the process, highlights my name using bold font using some regexp logic. I use this in evaluated code blocks on the publications page to generate the list of publications for each subsection. For example, to generate the section containing conference papers, I have the following:
* Conference Talks and Proceedings #+BEGIN_SRC emacs-lisp :exports results :results value html :eval yes (generate-bib-html "conf.bib") #+END_SRC
My custom HTML export backend
Now on to the fun stuff. In this section, I define a custom Org Export backend, derived from the original HTML export backend, which I use to generate the HTML for non-standard things like footnotes and margin notes. I wrote this specifically to support the Tufte-inspired margin style. The code is borrowing heavily from ox-tufte, which is a custom export backend designed to work with the original tufte-css project. It's a lot of code for a relatively small gain99Totally worth it, tbh., but don't forget, you are still here, 15 minutes into reading a post on using Emacs as a static site generator 😎.
(org-export-define-derived-backend 'ogbe-html 'html :translate-alist '((footnote-reference . ogbe-html-footnote-reference) (link . ogbe-html-maybe-margin-note-link) (special-block . ogbe-html-special-block))) (defun org-html-publish-to-ogbe-html (plist filename pub-dir) "Publish an org file to using my custom backend. PLIST is the property list for the given project. FILENAME is the filename of the Org file to be published. PUB-DIR is the publishing directory. Return output file name." (org-publish-org-to 'ogbe-html filename (concat "." (or (plist-get plist :html-extension) org-html-extension "html")) plist pub-dir))
I start by defining a derived org-export backend named ogbe-html. It mirrors the html backend, but provides its own exporters for footnotes (obviously), links (I encode inline margin notes as special links) and special blocks (for named, non-inlined margin notes). I then define the corresponding :publishing-function.
It's been a while, so I don't remember in detail anymore, but the next few functions are either completely new developments or borrowed/souped-up versions of similar ones found in ox-tufte, which I think I was drawing a lot of inspiration from at the time of writing. The basic idea behind a lot of these functions is to assign footnotes and margin notes special class-es, which I then subsequently style in the CSS stylesheets.
(defun ogbe-html-footnote-reference (footnote-reference contents info) (let ((text (replace-regexp-in-string ;; footnotes must have spurious <p> tags removed or they will not work "</?p.*>" "" (org-trim (org-export-data (org-export-get-footnote-definition footnote-reference info) info)))) (num (org-export-get-footnote-number footnote-reference info))) (format (concat "<span id=\"fnr.%d\" class=\"sidenote-number\">" "<a href=\"#fn.%d\"><sup>%s</sup></a></span>" "<span class=\"sidenote\" id=\"sn.%d\">" "<span><sup>%d</sup></span>" "%s</span>") num num num num num text)))
This function formats a footnote reference into a) superscript number (the footnote index) of class sidenote-number and b) a block of text containing the footnote text of class sidenote.Example: in style.css, I disable the display of the sidenote class like this:
.sidenote {
display: none;
}
Conversely, in margin-style.css, I enable the display of the sidenote:
.sidenote {
display: block;
}
Note that the footnote text will be visible in the auto-generated "Footnotes" section in either case! The only difference between the default and the margin style is that in the margin style, the text is essentially displayed twice. I think this is a small price to pay.
I can switch between the default style and the margin style by choosing to hide the sidenote element or display it in the margin.
(defun ogbe-html-format-margin-note (text id) (format "<span class=\"marginnote\" id=\"mn.%s\">%s</span>" id text)) (defun ogbe-html-maybe-margin-note-link (link desc info) ;; check if a link points to a #+BEGIN_margin ... #+END_margin ;; block. if yes, insert it as margin note inline. (let* ((path (org-element-property :path link)) (type (org-element-property :type link)) ;; check whether we are at a margin link. (margin-link (and (string= "fuzzy" type) (string= "mn" (car (split-string path ":"))))) ;; attempt to follow the link and check if we are looking at a margin block (margin-block (when (and (string= "fuzzy" type) (not margin-link)) (let ((elem (org-export-resolve-fuzzy-link link info))) (when (and (eq (org-element-type elem) 'special-block) (string= "margin" (org-element-property :type elem))) elem))))) (cond (margin-link ;; for a link-style margin note, replicate the behavior of ;; `ox-tufte.el' and just copy the link text without any ;; processing. (let ((text (replace-regexp-in-string "</?p.*>" "" desc)) (id (cadr (split-string path ":")))) (ogbe-html-format-margin-note text id))) (margin-block ;; for a rich block-style margin note, we need to export ;; the contents of the block and insert it inline. since at ;; this point, we do NOT want to skip the translation of ;; the "margin" blocks, we need to export the block with an ;; anonymous backend (see ;; https://orgmode.org/worg/dev/org-export-reference.html) (let ((text (org-export-data-with-backend margin-block (org-export-create-backend :parent 'ogbe-html :transcoders `((special-block . ,(lambda (block contents info) contents)) (paragraph . ogbe-html-margin-paragraph) (src-block . ogbe-html-margin-src-block))) info)) (id (org-element-property :name margin-block))) (ogbe-html-format-margin-note text id))) (t (org-html-link link desc info))))) (defun ogbe-html-special-block (special-block contents info) "Transcode a SPECIAL-BLOCK element from Org to HTML. CONTENTS holds the contents of the block. INFO is a plist holding contextual information. This function implements some special cases. For example, for `margin' blocks, we do not output anything, since they are handled by `ogbe-html-maybe-margin-note-link.'" (let ((block-type (org-element-property :type special-block))) (cond ((string= block-type "margin") "") (t (org-html-special-block special-block contents info)))))
This code essentially does two things. First, it checks whether any links in the .org file are special links of the form
[[mn:name_of_note][Text of the margin note]]
which get transcoded into margin notes. These are "inline" margin notes.
It's margin-ception!
Second, it checks whether any links contain links to a named special block defined by #+begin_margin ... #+end_margin tags. If yes, the contents of this block are inserted in the margin. In the .org file, I would insert such "named" margin note as follows:
mn-ception Second, it checks whether any links contain links to a named special block defined by =#+begin_margin ... #+end_margin= tags. If yes, the contents of this block are inserted in the margin. In the .org file, I would insert such "named" margin note as follows: #+begin_margin It's margin-ception! #+end_margin
This lets me insert more complicated things in the margin, like, e.g., figures and code blocks. There are some more sophisticated things going on, for example, I have define another org-export backend to transcode the contents of the margin note before I can then insert the HTML in the output, but I'll let the code do the talking.
(defun ogbe-html-margin-paragraph (paragraph contents info) "Transcode a PARAGRAPH element from Org to HTML. CONTENTS is the contents of the paragraph, as a string. INFO is the plist used as a communication channel. This is a modified version of `org-html-paragraph' which does not wrap the figure in a <figure> tag, but instead uses a custom span element." (let* ((parent (org-export-get-parent paragraph)) (parent-type (org-element-type parent)) (style '((footnote-definition " class=\"footpara\"") (org-data " class=\"footpara\""))) (attributes (org-html--make-attribute-string (org-export-read-attribute :attr_html paragraph))) (extra (or (cadr (assq parent-type style)) ""))) (cond ((and (eq parent-type 'item) (not (org-export-get-previous-element paragraph info)) (let ((followers (org-export-get-next-element paragraph info 2))) (and (not (cdr followers)) (memq (org-element-type (car followers)) '(nil plain-list))))) ;; First paragraph in an item has no tag if it is alone or ;; followed, at most, by a sub-list. contents) ((org-html-standalone-image-p paragraph info) ;; insert an image with the img tag (format "<span class=\"margin-img\">%s</span>" contents)) ;; Regular paragraph. Don't insert <p> tag, since we are making a margin note (t contents)))) (defun ogbe-html-margin-src-block (src-block contents info) "Transcode a SRC-BLOCK element from Org to HTML. CONTENTS holds the contents of the item. INFO is a plist holding contextual information. This outputs a <code> tag in a <span> instead of a <pre> tag in a <div> to make this work in the margin." (let ((lang (org-element-property :language src-block)) (code (org-html-format-code src-block info)) (label (let ((lbl (org-html--reference src-block info t))) (if lbl (format " id=\"%s\"" lbl) "")))) (format "<code class=\"src src-%s margin-src-block\"%s>\n\n%s\n</code>" lang label code)))
I promised that the code would get more fun to read! The above helper functions ensure that we can fit entire paragraphs with figures and source code blocks into the margin. They are part of the anonymous backend I mentioned in the previous paragraph.
(defun ogbe-html-footnote-section (info) "Format the footnote section. INFO is a plist used as a communication channel. This is a respin of `org-html-footnote-section.'" (let ((defs (org-export-collect-footnote-definitions info)) ;; make a table within a details block (fmt (concat "<details id=\"footnotes-details\">" "<summary class=\"footnote-summary\">Footnotes</summary>\n" "<div class=\"footnote-table\">\n" "<table style=\"margin: 0 0 0 0; max-width:100%%\">\n"; table header "%s\n" ; 1 row/footnote (see later) "</table></div></details>\n"))) (when defs (format fmt (mapconcat ;; most of this stolen from `org-html-footnote-section.' (lambda (definition) (pcase definition (`(,n ,_ ,def) (let* ((inline? (not (org-element-map def org-element-all-elements #'identity nil t))) (anchor (org-html--anchor (format "fn.%d" n) n (format " class=\"footnum\" href=\"#fnr.%d\" role=\"doc-backlink\"" n) info)) (contents (org-trim (org-export-data def info))) (text (if (not inline?) contents (format "<p class=\"footpara\">%s</p>" contents)))) ;; create a HTML table row instead. (format (concat "<tr class=\"fn-tbl-row\">\n" ; row "<td class=\"fn-tbl-number\">%s</td>\n" ; number "<td class=\"fn-tbl-item\">%s</td>" ; text "</tr>") anchor text))))) defs "\n"))))) (advice-add 'org-html-footnote-section :override #'ogbe-html-footnote-section)
Lastly, I override the original footnote section function to generate a footnote section wrapped in a <details> element. The footnotes themselves are formatted as a big HTML table, which I style using CSS.
The org-publish-project-alist
Finally, we have arrived at the end. The variable org-publish-project-alist pulls everything together. The main project consists of a variety of subprojects, each of which I have commented on extensively. This variable simply points the Org-mode publishing machinery to the right functions. Presented with little comment.
During development, I did not have my nice republish-current-file functionality. So I hacked together this switch based on an environment variable that would only build the test page. It is no longer necessary, but I have not removed it yet.
;; define a test mode in which we only build the test page (defvar my-website-test-mode nil "Enable test mode") (when (getenv "WEBSITE_TEST_MODE") (setq my-website-test-mode t)) ;; finally, pull the project together in the `org-publish-project-alist' (setq org-publish-project-alist `(("blog" :components ,(if my-website-test-mode '("blog-css" "blog-rss" "blog-pages") '("blog-articles" "blog-rss" "blog-pages" "blog-css" "blog-images" "blog-dl"))) ("blog-articles" :base-directory ,my-website-blog-dir :base-extension "org" :publishing-directory ,(concat my-website-out-dir "blog") :publishing-function (org-html-publish-to-ogbe-html my-blog-articles-add-subheader) :preparation-function my-blog-articles-preprocessor :completion-function my-blog-articles-postprocessor :htmlized-source t ;; this enables htmlize, which means that I can use css for code! :headline-level 4 :section-numbers nil :with-toc nil :with-drawers t :with-sub-superscript nil ;; important!! ;; the following removes extra headers from HTML output -- important! :html-link-home "/" :html-head ,website-head :html-head-include-default-style nil :html-head-include-scripts nil :html-format-drawer-function my-blog-org-export-format-drawer :html-home/up-format "" :html-mathjax-options ,my-blog-local-mathjax :html-mathjax-template ,(concat my-blog-extra-mathjax-config "\n<script type=\"text/javascript\" src=\"%PATH\"></script>") :html-link-up "" :html-link-home "" :html-preamble website-header :html-postamble website-footer ;; sitemap - list of blog articles :auto-sitemap t :sitemap-filename "blog.org" :sitemap-title "Blog" :exclude "blog-feed.org" ;; custom sitemap generator function :sitemap-function my-blog-sitemap :sitemap-function org-publish-sitemap-default :sitemap-sort-files anti-chronologically :sitemap-date-format "Published: %a %b %d %Y") ("blog-pages" :base-directory ,my-website-pages-dir :base-extension ,(if my-website-test-mode "nothing" "org") :include ("test-page.org" "publications.org") :publishing-directory ,my-website-out-dir :publishing-function org-html-publish-to-ogbe-html :preparation-function my-blog-pages-preprocessor :completion-function my-blog-pages-postprocessor :htmlized-source t :headline-level 4 :section-numbers nil :with-toc nil :with-drawers t :with-sub-superscript nil ;; important!! ;; the following removes extra headers from HTML output -- important! :html-link-home "/" :html-head ,website-head :html-head-include-default-style nil :html-head-include-scripts nil :html-format-drawer-function my-blog-org-export-format-drawer :html-home/up-format "" :html-mathjax-options ,my-blog-local-mathjax :html-mathjax-template ,(concat my-blog-extra-mathjax-config "<script src=\"%PATH\"></script>") :html-link-up "" :html-link-home "" :html-preamble website-header :html-postamble website-footer) ("blog-rss" :base-directory ,my-website-blog-dir :base-extension "org" :publishing-directory ,my-website-out-dir :publishing-function org-rss-publish-to-rss :with-author t :html-link-home "https://ogbe.net/" :html-link-use-abs-url t :title "Dennis Ogbe" :rss-image-url "https://ogbe.net/img/feed-icon-28x28.png" :my-rss-stylesheet "/res/rss.xsl" ; custom style sheet :completion-function my-website-rss-postprocessor :section-numbers nil :exclude ".*" :include ("blog-feed.org") :table-of-contents nil) ("blog-css" ; see my-website-cssjs-files as example for how these are published :base-directory ,my-website-css-dir :base-extension ".*" :exclude ,(rx (* anything) (or "~" "#" ".in")) :publishing-directory ,(concat my-website-out-dir "res") :publishing-function org-publish-attachment :preparation-function my-website-process-cssjs :completion-function my-website-clean-cssjs :recursive t) ("blog-images" :base-directory ,my-website-img-dir :base-extension ".*" :publishing-directory ,(concat my-website-out-dir "img") :publishing-function org-publish-attachment :recursive t) ("blog-dl" :base-directory ,my-website-dl-dir :base-extension ".*" :publishing-directory ,(concat my-website-out-dir "dl") :publishing-function org-publish-attachment :recursive t)))
Conclusion
That was a lot! In this post, I described in detail how this website is built using Emacs and Org-mode as my static site generator. Although it may look like I have done only one major redesign over the last six years, the nuts and bolts behind this static site generator have grown and evolved significantly. I hope that the information in this post is useful to others on their quest of generating a website using Emacs and Org mode. Fingers crossed 🫰🏾 that the build process I presented here remains stable for the next half-decade or so. I will try my best to provide revisions to this article should they become necessary.
Footnotes
| 1 | That was a lie. Of course I know why. It's because playing with this website is fun and during the holidays I actually have time to have fun 😊. By the way, this is the first article that I am writing with my new, tufte-css inspired style. I hope it works out. |
| 2 | I still think that Markdown + Pelican is a decent choice for a personal website. But if you are like me, that's juuust not emacsy enough… |
| 3 | This is important to me. I even make sure to serve all fonts (and even MathJax) from my server. I just don't like the idea of some external content going away and breaking my site. |
| 4 | For a very long time, I used bibtex2html for this purpose. This required me to carry around a binary for a program written in OCaml (!). Fortunately, now that citeproc.el exists and is integrated with the new Org-mode citation system, I can generate beautiful HTML bibliographies in pure elisp. |
| 5 | IYKYK. |
| 6 | Actually, I am also serving the intermediate |
| 7 | So maybe I need to amend my earlier statement about the source repo being fully self contained modulo Emacs. I guess it is fully self contained modulo Emacs, any Emacs dependencies, |
| 8 | To see how this is configured, check the |
| 9 | Totally worth it, tbh. |