Colophon

Architecture

How a docs directory becomes a bundle

The pipeline is shaped by two decisions: store Markdown rather than HTML, and keep routing out of the stored artifact. Everything below follows from those.

01

The pipeline

CI publishes; the backend ingests; three surfaces read the index. Each stage hands the next a smaller, more structured artifact than it received.

The Colophon publish and serve pipeline Five stages left to right: the repository's docs Markdown files; CI running colophon publish; object storage holding content-addressed blobs and per-revision manifests; the Backstage backend which ingests and chunks; and a Postgres index of navigation, pages and chunks. The index fans out to three consumers: the frontend, the search collator, and the MCP actions. Repository docs/**/*.md CI colophon publish Object storage blobs/ · bundles/ Backend ingest · chunk Postgres index nav · pages · chunks Frontend React + remark/rehype Search collator portal-wide search MCP actions via Actions Registry
The pipeline, stage by stage.
publish
Scan, validate, assemble a manifest, upload the blobs it references, then write the manifest. validate and publish share one code path, so they cannot drift into disagreeing about what is publishable.
register
CI tells the backend a revision exists and which channel should point at it. This is the only mutating call in the whole system that CI makes.
ingest
The backend reads the manifest from storage, records the revision and its pages, and — only if a channel points at it — reads every page blob and cuts retrieval chunks.
serve
The HTTP router, the search collator, and the MCP actions all read the same index. There is no second source of truth for any of them.

02

Two storage systems, split by artifact type

The split is not a deployment convenience. The two kinds of artifact have opposite properties, and forcing either into the other's store makes something expensive.

Object storage Postgres
Holds Markdown bodies, images, attachments Manifests, nav, pages, chunks, channels, entity links
Shape Large, immutable, content-addressed Small, relational, constantly queried
Written by CI, once per new blob, never again The backend, on every ingest and channel move
Key blobs/<ab>/<sha256> Revision id, bundle id, slug

Content-addressing is what makes retained history affordable. Every page body and every asset is stored once, under the sha-256 of its bytes, and shared by every revision that references it.

Two revisions sharing content-addressed blobs Two revision manifests on the left — one built from the main branch, one from release-1.x — both reference the same two blobs holding architecture.md and guides/publishing.md. Each also references its own distinct copy of reference/configuration.md. The shared blobs are stored once, not twice. MANIFESTS BLOB STORE — KEYED blobs/<ab>/<sha256> revision source.ref = main revision source.ref = release-1.x architecture.md SHARED guides/publishing.md SHARED reference/configuration.md main only reference/configuration.md 1.x only
Two revisions of one bundle. The pages they share (highlighted) are stored once; only the page that actually differs costs a second blob. Illustrative — the paths are this repository's, the arrangement is drawn for the explanation.

The two-character fan-out prefix in the key is there so that no single prefix accumulates the entire corpus, which matters for listing performance and for backends that shard on key prefix.

03

Revisions are content; channels are routing

A revision is an immutable snapshot, identified by the sha-256 of its manifest. A channel is a mutable named pointer — latest, 1.x, pr-42. CI publishes a revision, then repoints a channel at it.

The channel is deliberately absent from the manifest. A manifest describes content; a channel describes where readers are sent. Keeping the two apart is what makes the next three properties fall out for free.

Channels as mutable pointers at immutable revisions Four immutable revisions sit in a row in publication order. Above them, three channel pointers: 1.x points at the first revision, pr-42 points at the third, and latest points at the fourth and most recent. A dashed arrow shows latest being repointed backwards at the second revision — a rollback performed as a pointer move, with no rebuild and no new upload. CHANNELS — MUTABLE REVISIONS — IMMUTABLE, IN PUBLICATION ORDER → 1.x latest pr-42 ROLLBACK — POINTER MOVE a17f0c92… 1.x · 8e21b0c 7f3ce014… main · 3d9f214 c25e73f7… main · 4c47e41 9d10ab4c… pr-42 · 55c1e8a
Revision ids are the abbreviated sha-256 of the manifest. Of the four shown, c25e73f7… at commit 4c47e41 is the one the publish on the home page actually produced; the rest are drawn to illustrate the arrangement.

Rollback is a pointer move

A bad docs deploy is repaired by pointing latest at the previous revision. No rebuild, no re-upload, no CI run, and nothing about the bad revision is destroyed — it stays addressable for anyone who wants to look at what went wrong.

Release branches dedupe against each other

Map a release branch onto a channel and the two lines of documentation share every page they have in common. Keeping five supported versions online costs five manifests plus the blobs that genuinely differ.

PR previews are throwaway channels

A pull request publishes to pr-42 and never touches latest. When the branch is deleted the channel goes with it, and retention reclaims the revisions nothing points at.

04

Indexing discipline

Two rules keep the index from growing in proportion to how often CI runs rather than to how much documentation exists.

Only channel-pointed revisions are chunked. Chunking is the expensive step — it reads every page blob in the revision. A revision that is published and never pointed at costs a few index rows, not a full pass over the corpus.

Only the default channel projects into Backstage Search. Otherwise the portal search box returns the same page once per supported version, and the more carefully a team maintains its release branches the worse their search gets. Other channels stay reachable through the version picker and through colophon:search with an explicit channel filter.

Retention closes the loop. Revisions a channel points at are never collected; beyond those, the most recent revisionsPerChannel per channel are kept and the rest become eligible for garbage collection.

05

The manifest is written last

Blobs go up first; the manifest goes up only once they are all present. Until the manifest exists the revision is not readable, so a crash halfway through an upload leaves orphaned blobs — which retention will collect — rather than a revision that resolves to a page whose body was never stored.

This is the same reason the revision id excludes the timestamp: a partially uploaded revision that is retried produces the same id and the same keys, so the retry completes the original rather than starting a parallel one.