How to Build a Static Site Generator in Python From Scratch
Build a working static site generator in Python from scratch, covering Markdown, front matter, Jinja2 templates, a build cache, and automated deployment with GitHub Actions.
Most blogging platforms, including the one hosting this article, render every page on demand: a request comes in, a database gets queried, a template engine assembles HTML, and the response goes out. A static site generator skips all of that at request time. It runs once, ahead of time, turns a folder of Markdown files into plain HTML files, and then a plain web server (or GitHub Pages, or any file host) just hands those files out. There is no database to query and no template engine running while a visitor waits.
Table Of Content
- What a Static Site Generator Actually Does
- Prerequisites
- Step 1: Set Up the Project
- Step 2: Parse Front Matter, and Get the Error Messages Right
- Step 3: Turn Markdown Into HTML, and Meet a Silent Renderer Bug
- Step 4: Add a Custom Shortcode, and Corrupt Your Own Documentation
- Step 5: Build a Cache That Tracks Its Real Inputs, and Silently Serve Stale HTML
- Step 6: Generate an Index Page
- Step 7: Escape Untrusted Front Matter Without Breaking Your Own HTML
- Step 8: Deploy on Every Push With GitHub Actions
- A Gotcha If You Ever Parse This File in Python
- Common Mistakes and Gotchas
- How to Verify Everything Works
- Next Steps
In this tutorial you will build one from scratch in Python: a generator that reads Markdown posts with YAML front matter, renders them through Jinja2 templates, supports a small custom shortcode syntax, skips rebuilding pages that have not changed, and deploys itself to GitHub Pages on every push. Every script below was written and personally run in a real sandbox for this article, and five separate bugs showed up along the way, in the front matter parser, the Markdown renderer, the shortcode preprocessor, the build cache, and the template engine’s default security setting. Each one is reproduced here with the exact broken output before you see the fix, not just described after the fact.
What a Static Site Generator Actually Does
Strip away the branding around tools like Jekyll, Hugo, or Eleventy, and a static site generator does five things, in order:
- Read a Markdown file plus a small block of metadata at the top (the front matter): title, date, which template to use.
- Convert the Markdown body to HTML.
- Drop that HTML into a reusable page template (the templating engine‘s job) so every post shares the same header, footer, and styling.
- Write the result to an output folder as a plain
.htmlfile. - Optionally skip step 2 to 4 entirely for files that have not changed since the last run (the build cache), so a large site does not re-render everything on every edit.
That is the whole model. Understanding it by building it, rather than just pointing a CLI at a folder, is worth doing even if you end up using an off-the-shelf generator afterward: every bug you hit below is a bug real generators have to solve too, and knowing why the solution looks the way it does makes the next YAML front matter error message or stale-cache mystery much faster to diagnose.
Prerequisites
- Python 3.10 or newer (this tutorial was built and run on Python 3.13.14). Anything with the
tuple[dict, str]style type hints used below needs 3.9+; the code has no other version-specific requirements. pipfor installing packages, and comfort creating a virtual environment.- Basic familiarity with Markdown syntax. No prior YAML or Jinja2 experience is required; both are explained as they show up.
- A GitHub account and
git, only needed for the deployment step near the end. Everything before that runs entirely on your own machine with no external accounts.
Step 1: Set Up the Project
Create a project folder with a place for your Markdown source files and a place for your templates, then set up an isolated environment:
mkdir my-blog
cd my-blog
mkdir content templates
python -m venv .venv
# Windows PowerShell:
.venv\Scripts\Activate.ps1
# macOS / Linux:
source .venv/bin/activate
pip install markdown PyYAML Jinja2 pytest
Pin what you installed so the build is reproducible on another machine (or in CI, which you will set up in Step 8):
markdown==3.10.3
PyYAML==6.0.3
Jinja2==3.1.6
pytest==9.1.1
Save that as requirements.txt. Three small libraries are doing all the real work here: markdown turns Markdown text into HTML, PyYAML parses the front matter block, and Jinja2 renders the page templates. Everything else in this tutorial, front matter parsing, shortcodes, the build cache, is code you will write yourself, on purpose, so you understand exactly what it is doing and why.
Step 2: Parse Front Matter, and Get the Error Messages Right
Front matter is a block of YAML at the top of a Markdown file, fenced by two lines that each contain exactly three hyphens. Create content/hello-world.md:
---
title: "Hello, World"
date: 2026-09-10
layout: post.html
---
This is my first post on the new generator.
It has **two** paragraphs so far, and a short code sample below:
```python
def greet(name):
return f"Hello, {name}!"
```
That's it for now.
Now write a parser for it in frontmatter.py. A regular expression that captures everything between the two --- delimiters as the metadata, and everything after the second delimiter as the body, is enough:
import re
import yaml
FRONT_MATTER_RE = re.compile(r"\A---\s*\n(.*?\n)---\s*\n(.*)", re.DOTALL)
def parse_front_matter(text: str) -> tuple[dict, str]:
match = FRONT_MATTER_RE.match(text)
if not match:
raise ValueError("No front matter found (file must start with '---')")
meta = yaml.safe_load(match.group(1)) or {}
body = match.group(2)
return meta, body
That looks correct, and it is, for a well-formed file. Now break it on purpose. Save a post that opens with --- but never closes the front matter block:
---
title: "Broken Post"
date: 2026-09-11
layout: post.html
This post forgot to close its front matter.
Running the parser against it raises:
ValueError: No front matter found (file must start with '---')
That message is actively misleading. The file does start with ---. The real problem is a missing closing delimiter, and a writer staring at that error while their file visibly begins with three dashes will waste time looking in the wrong place. The regex’s single failure branch cannot distinguish “there is no opening delimiter at all” from “there is an opening delimiter but the match never found a matching close,” because both cases just mean the whole pattern failed to match.
Fix it by checking the two failure modes separately, with a cheap string check before the expensive regex:
import re
import yaml
FRONT_MATTER_RE = re.compile(r"\A---\s*\n(.*?\n)---\s*\n(.*)", re.DOTALL)
def parse_front_matter(text: str) -> tuple[dict, str]:
if not text.startswith("---"):
raise ValueError(
"No front matter found: file must start with '---' on its own line"
)
match = FRONT_MATTER_RE.match(text)
if not match:
raise ValueError(
"Front matter opened with '---' but never closed: "
"add a second '---' line after the metadata"
)
meta = yaml.safe_load(match.group(1)) or {}
body = match.group(2)
return meta, body
Re-running the same broken file now raises a message that actually points at the fix:
ValueError: Front matter opened with '---' but never closed: add a second '---' line after the metadata
One more thing worth knowing before moving on: yaml.safe_load does not hand you back a plain string for date: 2026-09-10. It recognizes the ISO 8601 shape and parses it straight into a real Python datetime.date object. That is convenient for sorting posts by date later (Step 6), and it is why the front matter shown above quotes the title ("Hello, World") but leaves the date bare.
Step 3: Turn Markdown Into HTML, and Meet a Silent Renderer Bug
With front matter parsed out, the body text needs to become HTML. Write a first version of the build script, build.py, that loads every Markdown file, converts it, and renders it through a Jinja2 template:
from pathlib import Path
import markdown
from jinja2 import Environment, FileSystemLoader
from frontmatter import parse_front_matter
CONTENT_DIR = Path("content")
TEMPLATE_DIR = Path("templates")
OUTPUT_DIR = Path("_site")
def render_post(md_path: Path, env: Environment) -> tuple[dict, str]:
text = md_path.read_text(encoding="utf-8")
meta, body = parse_front_matter(text)
html_body = markdown.markdown(body)
template = env.get_template(meta.get("layout", "post.html"))
return meta, template.render(meta=meta, content=html_body)
def build() -> None:
env = Environment(loader=FileSystemLoader(TEMPLATE_DIR))
OUTPUT_DIR.mkdir(exist_ok=True)
for md_path in sorted(CONTENT_DIR.glob("*.md")):
meta, html = render_post(md_path, env)
out_path = OUTPUT_DIR / (md_path.stem + ".html")
out_path.write_text(html, encoding="utf-8")
print(f"built {out_path} from {md_path.name}")
if __name__ == "__main__":
build()
You also need a template. Create templates/post.html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>{{ meta.title }}</title>
</head>
<body>
<article>
<h1>{{ meta.title }}</h1>
<p class="date">{{ meta.date }}</p>
{{ content }}
</article>
</body>
</html>
Run python build.py and open _site/hello-world.html. The title, date, and first paragraph all look right. The fenced code block does not:
<p>It has <strong>two</strong> paragraphs so far, and a short code sample below:</p>
<p><code>python
def greet(name):
return f"Hello, {name}!"</code></p>
<p>That's it for now.</p>
The triple backticks and the python language hint never turned into a proper <pre><code> block. Instead the whole fenced block got treated as one long inline code span, the same way `a single backtick` would be, with the word “python” and both lines of the function crammed inside a single <code> tag wrapped in a stray <p>.
This is not a bug in the markdown library. Fenced code blocks are an optional extension, not core Markdown syntax, and markdown.markdown() only parses core syntax unless you explicitly ask for more. Python-Markdown’s own documentation for the extension confirms the shape of the feature but not that it is on by default: “The Fenced Code Blocks extension adds a secondary way to define code blocks, which overcomes a few limitations of indented code blocks. This extension is included in the standard Markdown library.” Included in the library and enabled are two different things.
Turn it on, along with two other extensions worth having from day one (tables for Markdown tables, and toc for automatic heading anchors you will likely want later):
MARKDOWN_EXTENSIONS = ["fenced_code", "tables", "toc"]
html_body = markdown.markdown(body, extensions=MARKDOWN_EXTENSIONS)
Rebuild, and the same source now produces a real code block:
<p>It has <strong>two</strong> paragraphs so far, and a short code sample below:</p>
<pre><code class="language-python">def greet(name):
return f"Hello, {name}!"
</code></pre>
<p>That's it for now.</p>
A real <pre><code class="language-python"> block, with the language name promoted to a CSS class the way the HTML5 spec recommends, ready for a syntax highlighter to hook into later.
Step 4: Add a Custom Shortcode, and Corrupt Your Own Documentation
Markdown does not have a built-in syntax for things like margin notes or callout boxes. A small custom “shortcode” syntax fills that gap. Define one: {{ note: some text }} anywhere inside a paragraph should turn into <span class="sidenote">some text</span>.
The obvious first implementation is a single regex substitution run over the raw Markdown text before it reaches the Markdown parser:
import re
SIDENOTE_RE = re.compile(r"\{\{\s*note:\s*(.*?)\s*\}\}", re.DOTALL)
def apply_shortcodes(text: str) -> str:
return SIDENOTE_RE.sub(r'<span class="sidenote">\1</span>', text)
Try it on a post that both uses a sidenote and documents the sidenote syntax for other writers, inside a fenced code block. Save this as content/shortcodes-demo.md (front matter omitted here for brevity; give it a title, a date, and a layout the same way hello-world.md does):
Here's a real sidenote: {{ note: This text should float into a span,
not stay literal. }}
If you want to document the syntax itself, you might show it like this:
```text
Use {{ note: your text here }} anywhere in a paragraph.
```
That code block should show the raw syntax, untouched.
Running apply_shortcodes() on this before handing it to markdown.markdown() produces:
<p>Here's a real sidenote: <span class="sidenote">This text should float into a span,
not stay literal.</span></p>
<p>If you want to document the syntax itself, you might show it like this:</p>
<pre><code class="language-text">Use <span class="sidenote">your text here</span> anywhere in a paragraph.
</code></pre>
<p>That code block should show the raw syntax, untouched.</p>
The sidenote outside the code block worked. The code block that was supposed to show the literal {{ note: ... }} syntax got rewritten too, because a plain regex has no idea what a fenced code block is. It matched the shortcode pattern wherever it appeared in the raw text, including inside three backticks that were supposed to protect that text from exactly this kind of transformation. Any documentation of your own templating syntax, written using the syntax it documents, will self-destruct under this implementation.
The fix is to make the preprocessor fence-aware: split the text on fenced code blocks first, and only run the shortcode substitution on the pieces that are not inside a fence.
import re
SIDENOTE_RE = re.compile(r"\{\{\s*note:\s*(.*?)\s*\}\}", re.DOTALL)
FENCE_RE = re.compile(r"(```.*?```)", re.DOTALL)
def apply_shortcodes(text: str) -> str:
pieces = FENCE_RE.split(text)
for i, piece in enumerate(pieces):
if piece.startswith("```"):
continue # inside a fenced code block: leave it exactly as written
pieces[i] = SIDENOTE_RE.sub(r'<span class="sidenote">\1</span>', piece)
return "".join(pieces)
re.split() with a capturing group keeps the delimiter (the fenced block itself) in the returned list, alternating between “text before a fence,” “the fence,” “text after,” and so on. Skipping every piece that starts with three backticks means fenced content is stitched back in byte for byte. Rerunning the identical input now produces:
<p>Here's a real sidenote: <span class="sidenote">This text should float into a span,
not stay literal.</span></p>
<p>If you want to document the syntax itself, you might show it like this:</p>
<pre><code class="language-text">Use {{ note: your text here }} anywhere in a paragraph.
</code></pre>
<p>That code block should show the raw syntax, untouched.</p>
The sidenote outside the fence still converts. The literal syntax inside the fence now survives untouched.
Step 5: Build a Cache That Tracks Its Real Inputs, and Silently Serve Stale HTML
Re-rendering every post on every run is fine for three files and painfully slow once a site has a few hundred. An explicit build cache, a small file that remembers what was already built and skips it next time, solves that without needing a filesystem watcher running in the background. The natural first design keys the cache on a hash of each Markdown file’s contents:
import hashlib
import json
from pathlib import Path
CACHE_PATH = Path(".build_cache.json")
def load_cache() -> dict:
if CACHE_PATH.exists():
return json.loads(CACHE_PATH.read_text(encoding="utf-8"))
return {}
def save_cache(cache: dict) -> None:
CACHE_PATH.write_text(json.dumps(cache, indent=2), encoding="utf-8")
def content_hash(md_path: Path) -> str:
return hashlib.sha256(md_path.read_bytes()).hexdigest()
Wire it into the build loop: compute the hash, compare it to what was cached last time, skip if they match, render and save if they do not. Build once, and everything renders as expected. Build a second time with nothing changed, and both posts correctly print skip.
Now edit only the template, not any content file. Add a word to the heading in templates/post.html:
<h1>UPDATED THEME: {{ meta.title }}</h1>
Rebuild. Both posts still print skip hello-world.md (unchanged) and skip shortcodes-demo.md (unchanged). Open the actual output file and the heading has not changed at all: it still says <h1>Hello, World</h1>, with no “UPDATED THEME” anywhere. The generator now silently serves a stale page and reports success while doing it.
The cache key only ever covered the Markdown source file. But render_post() does not depend on the Markdown source alone; it also depends on whatever template that source gets rendered through. A cache key that leaves out an input the render step actually reads will never notice when that input changes, and there is no warning, no error, nothing in the console to suggest anything is wrong. This class of bug (a cache key that quietly does not cover everything the cached computation depends on) is one of the most common ways real build systems and web caches go stale in production.
The fix is to hash every file the render step reads, not just the one that is easiest to reach for:
def render_key(md_path: Path, template_path: Path) -> str:
hasher = hashlib.sha256()
hasher.update(md_path.read_bytes())
hasher.update(template_path.read_bytes())
return hasher.hexdigest()
With the cache key built from render_key() instead of content_hash() alone, repeat the exact same experiment: build once, build again with nothing changed (both skip), then edit only the template again. This time the rebuild correctly prints build hello-world.md -> _site\hello-world.html for every post that uses that template, and the output file’s heading correctly shows the updated theme. Revert the template back before moving on.
Step 6: Generate an Index Page
Add a third post before building a page that lists them, so the listing has something worth looking at. Save this as content/second-post.md:
---
title: "Why Cache Keys Need to Match Reality"
date: 2026-09-14
layout: post.html
---
A cache is only as correct as the key you chose for it. If the key
leaves out an input your build depends on, the cache will happily
serve a stale result forever and never tell you it's wrong.
{{ note: This is the bug this generator hit in Step 5, and the fix
was to hash the template alongside the content. }}
A blog needs a page listing every post, not just individual post pages. The natural place to collect that list is inside the same loop that builds each post, but it has to happen for every post regardless of whether that post’s own HTML was skipped by the cache: the index depends on every post’s title and date, and those do not become stale just because the post’s body did not change.
posts = [] # collected for the index, whether or not each page was rebuilt
for md_path in sorted(CONTENT_DIR.glob("*.md")):
text = md_path.read_text(encoding="utf-8")
meta, body = parse_front_matter(text)
posts.append({"title": meta["title"], "date": meta["date"], "slug": md_path.stem})
# ... cache check and render happen here, as in Step 5 ...
posts.sort(key=lambda p: p["date"], reverse=True)
index_template = env.get_template("index.html")
(OUTPUT_DIR / "index.html").write_text(index_template.render(posts=posts), encoding="utf-8")
Add templates/index.html:
<!DOCTYPE html>
<html lang="en">
<head><meta charset="utf-8"><title>My Blog</title></head>
<body>
<h1>My Blog</h1>
<ul>
{% for post in posts %}
<li>
<a href="{{ post.slug }}.html">{{ post.title }}</a>
<span class="date">{{ post.date }}</span>
</li>
{% endfor %}
</ul>
</body>
</html>
With three posts dated 2026-09-10, 2026-09-12, and 2026-09-14 in content/, running the build produces an index listing all three newest first, correctly sorted using the real datetime.date objects PyYAML parsed out of each file’s front matter back in Step 2, with no string-to-date conversion needed anywhere.
Step 7: Escape Untrusted Front Matter Without Breaking Your Own HTML
Every template above interpolates meta.title directly. Jinja2’s own API documentation flags exactly this as a trap for newcomers: “In future versions of Jinja we might enable autoescaping by default for security reasons. As such you are encouraged to explicitly configure autoescaping now instead of relying on the default.” The default, today, is off.
Prove it. Render a post whose title contains an HTML tag:
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("templates"))
template = env.get_template("post.html")
meta = {"title": "<script>alert(1)</script>", "date": "2026-01-01"}
html = template.render(meta=meta, content="<p>safe pre-rendered content</p>")
The rendered output contains:
<title><script>alert(1)</script></title>
<h1><script>alert(1)</script></h1>
The script tag lands in the output completely unescaped. Front matter is something you write yourself today, so this feels academic right up until the moment you accept a guest post, pull a title from an RSS feed, or let a coding agent draft front matter for you; the moment any of that metadata is not fully trusted, an unescaped title becomes a real cross-site scripting vector.
The fix is not simply “turn autoescaping on,” though. Autoescaping applies to everything passed into a template, and the Markdown-rendered HTML from Step 3 is exactly the kind of content you want passed through as real markup, not escaped into visible angle brackets. The correct fix is to turn autoescaping on for the environment and then explicitly mark the one value that is trusted, already-rendered HTML, using Jinja2’s Markup wrapper (from the markupsafe package, a dependency Jinja2 already installs for you):
from markupsafe import Markup
env = Environment(loader=FileSystemLoader("templates"), autoescape=True)
template = env.get_template("post.html")
meta = {"title": "<script>alert(1)</script>", "date": "2026-01-01"}
html = template.render(meta=meta, content=Markup("<p>safe pre-rendered content</p>"))
Rerun it, and the output changes in exactly the two places that should change:
<title><script>alert(1)</script></title>
<h1><script>alert(1)</script></h1>
<p>safe pre-rendered content</p>
The malicious title is now inert text on the page instead of a live script tag. The legitimate Markdown-rendered paragraph is still real HTML, not escaped into visible entities, because it was explicitly wrapped in Markup() before being handed to the template. In render_html(), that means wrapping the output of markdown.markdown() the same way before returning it:
return template.render(meta=meta, content=Markup(html_body))
Everywhere else in the template, front matter fields stay auto-escaped by default, and you never have to remember to escape them yourself.
Steps 3 through 7 all touched build.py a piece at a time. Here is the complete file exactly as those changes leave it, safe to diff your own copy against:
"""The full generator: front matter, shortcodes, markdown, a cache-aware
render step per post, and an index page listing every post by date."""
from pathlib import Path
import markdown
from jinja2 import Environment, FileSystemLoader
from markupsafe import Markup
from cache import load_cache, render_key, save_cache
from frontmatter import parse_front_matter
from shortcodes import apply_shortcodes
CONTENT_DIR = Path("content")
TEMPLATE_DIR = Path("templates")
OUTPUT_DIR = Path("_site")
MARKDOWN_EXTENSIONS = ["fenced_code", "tables", "toc"]
def render_html(meta: dict, body: str, env: Environment) -> str:
processed = apply_shortcodes(body)
html_body = markdown.markdown(processed, extensions=MARKDOWN_EXTENSIONS)
template = env.get_template(meta.get("layout", "post.html"))
# meta.title and friends come from front matter and get auto-escaped.
# html_body is trusted output from our own markdown pipeline, so it is
# wrapped in Markup() to tell Jinja2 not to escape it a second time.
return template.render(meta=meta, content=Markup(html_body))
def build() -> None:
env = Environment(loader=FileSystemLoader(TEMPLATE_DIR), autoescape=True)
OUTPUT_DIR.mkdir(exist_ok=True)
cache = load_cache()
new_cache = {}
posts = [] # collected for the index, whether or not each page was rebuilt
for md_path in sorted(CONTENT_DIR.glob("*.md")):
text = md_path.read_text(encoding="utf-8")
meta, body = parse_front_matter(text)
layout_name = meta.get("layout", "post.html")
template_path = TEMPLATE_DIR / layout_name
out_path = OUTPUT_DIR / (md_path.stem + ".html")
posts.append({"title": meta["title"], "date": meta["date"], "slug": md_path.stem})
key = str(md_path)
digest = render_key(md_path, template_path)
new_cache[key] = digest
if cache.get(key) == digest and out_path.exists():
print(f"skip {md_path.name} (unchanged)")
continue
html = render_html(meta, body, env)
out_path.write_text(html, encoding="utf-8")
print(f"build {md_path.name} -> {out_path}")
posts.sort(key=lambda p: p["date"], reverse=True)
index_template = env.get_template("index.html")
(OUTPUT_DIR / "index.html").write_text(
index_template.render(posts=posts), encoding="utf-8"
)
print(f"build index.html ({len(posts)} posts)")
save_cache(new_cache)
if __name__ == "__main__":
build()
frontmatter.py, shortcodes.py, and cache.py hold the pieces built in Steps 2, 4, and 5 respectively, each imported here rather than duplicated inline.
Step 8: Deploy on Every Push With GitHub Actions
The last piece is a CI pipeline that runs build.py and publishes _site/ to GitHub Pages automatically. GitHub Pages supports deploying straight from a workflow run rather than committing built HTML to a branch, which keeps generated output out of your git history entirely. Save this as .github/workflows/deploy.yml in a real GitHub repository (creating one is outside the scope of this tutorial, but everything up to this point has already run entirely on your own machine with nothing to sign up for):
name: Build and deploy site
on:
push:
branches: [main]
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: pages
cancel-in-progress: true
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: "3.13"
- run: pip install -r requirements.txt
- run: python build.py
- uses: actions/upload-pages-artifact@v5
with:
path: _site
deploy:
needs: build
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- id: deployment
uses: actions/deploy-pages@v5
The action versions above (checkout@v7, setup-python@v7, upload-pages-artifact@v5, deploy-pages@v5) were confirmed against each action’s own GitHub releases API at the time of writing; check the releases page for each action if you are reading this later, since these projects tag new major versions periodically. The permissions block is not decorative. The actions/deploy-pages project’s own documentation is explicit about why both lines are required: pages: write exists “to deploy to Pages,” and id-token: write exists “to verify the deployment originates from an appropriate source.” Without both, the deploy job fails with a permissions error rather than silently doing less than you expect.
Two things in a real repository, done once, before this workflow will do anything: go to the repository’s Settings, then Pages, and under “Build and deployment” set the Source dropdown to “GitHub Actions” instead of the default “Deploy from a branch.” Then push this workflow file on the branch named in the on: push: branches: list. From that point on, every push to main rebuilds the site and republishes it, with no manual step in between.
A Gotcha If You Ever Parse This File in Python
The workflow above runs fine on GitHub’s own servers. But if you also write a small Python script to validate or lint your own workflow file (a natural thing to want, especially once you already have a build script that parses other structured files), reaching for yaml.safe_load() on it will surprise you:
import yaml
doc = yaml.safe_load(open(".github/workflows/deploy.yml").read())
print(list(doc.keys()))
['name', True, 'permissions', 'concurrency', 'jobs']
The on: key is not in that list. Trying doc["on"] raises a plain KeyError: 'on', even though the file quite obviously has an on: section. PyYAML implements YAML 1.1, and the 1.1 specification’s boolean type defines a much wider set of “true-ish” and “false-ish” tokens than most people expect. The specification’s own boolean type page lists the exact pattern: y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF. A bare on at the top of a YAML mapping is not the string “on”; it resolves to the Python boolean True. The correct key is doc[True].
It gets worse if a document happens to use more than one of these tokens as top-level keys. on and yes both resolve to the same Python value, True, so a mapping containing both silently collapses into one key, with whichever value was parsed last quietly overwriting the other with no warning and no error. The same happens between off and no, both of which resolve to False. This is sometimes called the YAML “Norway problem,” after the ISO country code “NO” suffering the identical fate in configuration files that were never meant to contain booleans at all.
Two real fixes exist, and either works for a validation script like this one. Quote the key explicitly in the source YAML ("on": instead of on:), which is unambiguous to any YAML parser and does not change how GitHub itself interprets the workflow. Or, if you cannot edit the file, load it with yaml.load(text, Loader=yaml.BaseLoader) instead of safe_load(); BaseLoader skips all implicit type resolution and returns every scalar as a plain string, keys included.
Common Mistakes and Gotchas
- Fenced code blocks need an explicit extension.
markdown.markdown(text)alone only implements core Markdown. Triple-backtick fences silently degrade into one long inline code span instead of raising an error, so the bug is easy to miss until you actually read the rendered HTML. - Any text-level preprocessor needs to be fence-aware. A regex applied to raw Markdown before parsing has no concept of code fences and will happily rewrite literal syntax examples inside them. Split on fences first, transform the rest, and stitch the fenced pieces back in untouched.
- A cache key must cover every input the cached computation reads, not just the one that is easiest to hash. Hashing only the content file and forgetting the template it renders through produces a cache that reports success while quietly serving stale output, with no error anywhere in the console.
- Metadata collection for an index or listing page must not be skipped when the individual page is cached. The index depends on every post’s title and date every time it builds, regardless of whether that post’s own HTML changed.
- Jinja2 does not autoescape by default. Turn
autoescape=Trueon for the environment, and explicitly wrap already-rendered, trusted HTML (like Markdown output) inMarkup()so it is not double-escaped. - PyYAML’s default loader implements YAML 1.1 boolean resolution. Bare
on,off,yes, andnokeys become Python booleans, not strings, which matters the moment you write your own tooling around a GitHub Actions workflow file.
How to Verify Everything Works
Lock the render, cache, and shortcode behavior in with a small test suite in test_build.py: parse both the happy path and both front matter error cases, confirm a sidenote outside a fence gets converted while one inside a fence does not, confirm the cache key changes when the template changes and stays stable when nothing does, and confirm autoescaping protects a hostile title while leaving trusted Markdown output alone. Running it should show every case passing:
python -m pytest test_build.py -v
test_build.py::test_parse_front_matter_happy_path PASSED
test_build.py::test_parse_front_matter_missing_entirely PASSED
test_build.py::test_parse_front_matter_never_closed PASSED
test_build.py::test_shortcode_replaces_sidenote_outside_code PASSED
test_build.py::test_shortcode_leaves_fenced_code_untouched PASSED
test_build.py::test_render_key_changes_when_template_changes PASSED
test_build.py::test_render_key_stable_when_nothing_changes PASSED
test_build.py::test_autoescape_protects_front_matter_but_not_rendered_markdown PASSED
8 passed in 0.10s
Then confirm the whole pipeline end to end from a completely clean checkout: delete _site/ and .build_cache.json, run python build.py once, and check three things. First, _site/index.html lists every post, newest first. Second, open one of the generated .html files directly in a browser (a plain file:// path works fine, no server needed) and confirm the code block from Step 3 renders as an actual formatted code block, not a paragraph full of visible backticks. Third, run the build a second time with nothing changed and confirm every post prints skip while index.html still rebuilds. If all three hold, the generator is behaving the way every step above proved it should.
Next Steps
The generator here covers the core loop every static site generator is built around, but a few natural extensions are worth exploring once this feels solid: a draft: true front matter flag that Step 6’s loop skips over entirely, syntax highlighting for code blocks using the codehilite extension alongside fenced_code, an RSS feed generated the same way index.html is, and tag or category pages built by grouping the same posts list from Step 6 differently.
Before you wire the deployment workflow from Step 8 into a real repository, sxz.io has two more tutorials worth reading. How to Pin and Verify GitHub Actions to Stop Supply Chain Poisoning covers why the @v7-style version tags used above are convenient but not the safest option for a workflow you actually rely on, and what pinning to a commit SHA buys you instead. How to Scope GitHub Actions Permissions to Least Privilege With actionlint goes deeper on exactly the kind of permissions: block this workflow’s deploy job depends on. And if the autoescaping lesson from Step 7 was useful, How to Stop XSS Attacks in a Python Web App With a Content Security Policy covers a second, independent layer of defense against the same class of bug for anything you build that serves untrusted input directly.








No Comment! Be the first one.