LLM Wiki 101
Welcome to LLM Wiki 101. Over the next forty-five minutes we'll explore a pattern Andrej Karpathy described for building knowledge bases that a language model maintains for you — not a static wiki you update by hand, and not a black-box vector store, but something in between that stays accurate as it grows. We'll cover both the conceptual foundations and the engineering details, so whether you're here to understand the idea or to actually build one, there's a track for you.
Persistent LLM-maintained knowledge bases — Karpathy's pattern, distilled
v1 · 2026-05-09 · 2 tracks · ~45 min
EN also in ภาษาไทย
What you'll learn
The course splits into two tracks. Track A, Foundations, answers the "what" and "why" — what an LLM Wiki actually is, why it's different from plain RAG, where the idea came from with the Zettelkasten note-taking method, and the three-layer structure at its core. Track B, Engineering, is the "how" — designing your schema, the tooling with Obsidian and Git, the failure modes to watch for like knowledge drift, and how to wire it into agents. You don't have to follow both in order; pick the track that matches why you came.
Track A
Foundations
What is an LLM Wiki?
Why not just RAG?
Zettelkasten ancestry
The three layers
Page template + citation discipline
Operating it by hand
Track B
Engineering
Schema design choices
Tooling: Obsidian + Git
Failure modes (drift)
Integration with agents
Evaluation metrics
Cheatsheet
The central claim
Here's the single idea everything rests on, in Karpathy's words: the wiki is a persistent, compounding artifact. Contrast that with ordinary RAG, which is stateless — every query re-derives its answer from scratch, learning nothing between calls. An LLM Wiki is stateful: each new source adds to a body of synthesis that keeps growing. And notice what's actually new here. The architecture itself is about seventy years old, going back to paper note systems. What's genuinely new is delegating the maintenance — the linking, the updating, the bookkeeping — to the language model itself.
"The wiki is a persistent, compounding artifact." — Andrej Karpathy
RAG: stateless — re-derive every query
LLM Wiki: stateful — accumulates synthesis
The novelty isn't the architecture (70 years old)
The novelty is delegating maintenance to the LLM
Three layers, three operations
The system has three layers, and the key thing to grasp is who is allowed to change each one. The raw layer is never edited — it's the verbatim source material, curated by a human. The wiki layer is where synthesis lives; the LLM updates it on every ingest, with your review. And the schema layer — the rules of the system — changes rarely and deliberately, authored by you like a constitution. Operating on those layers are three verbs: ingest, turning a new source into wiki updates; query, pulling accumulated knowledge to answer a question; and lint, a periodic health check we'll come back to.
Layer Mutability Authored by Raw Never edited Human (curates) Wiki Updated per ingest LLM (with review) Schema Rare, deliberate Human (constitution)
Three operations
Ingest — turn a new source into wiki updates (multi-page)Query — retrieve accumulated knowledge to answerLint — periodic health check
RAG vs LLM Wiki
Let's put RAG and the LLM Wiki side by side, because they're often confused. RAG is stateless per query and retrieves raw chunks, connecting them only by embedding similarity, with fairly loose provenance. The wiki is stateful and compounds, retrieves synthesised pages, connects them with explicit wikilinks, and keeps tight provenance from page to claim to source. RAG shines for quick lookup and per-query question answering; the wiki shines for deep domain mastery and research. And in production you don't pick one — you go wiki first because it's cheap, and fall back to RAG when you need the very latest.
Dimension RAG LLM Wiki State Stateless per query Stateful, compounds Retrieves Raw chunks Synthesised pages Connections Embedding similarity Wikilinks Provenance Chunk → source (loose) Page → claim → source (tight) Best for Lookup / per-query QA Domain mastery / research
Production hybrid: wiki first (cheap), RAG fallback (current).
Ancestry — Zettelkasten
This idea has a deep ancestor: the Zettelkasten, or slip-box. The sociologist Niklas Luhmann built one out of roughly ninety thousand paper index cards, and from that system he produced around fifty books and five hundred and fifty articles. Three principles made it work: atomicity, one idea per note; cross-referencing, explicit links between notes; and unique, stable identifiers for every card. Strip away the paper and you can see it clearly — an LLM Wiki is a Zettelkasten with a language model doing the maintenance Luhmann had to do by hand.
Niklas Luhmann built ~90,000 paper index cards. ~50 books + ~550 articles came out of that system.
Atomicity — one idea per noteCross-referencing — explicit links between notesUnique IDs — stable identifiers (numbered hierarchies)
"LLM Wiki = Zettelkasten + an LLM doing the maintenance."
The genealogy
It helps to see the whole lineage. In the nineteen-fifties, the paper Zettelkasten — all maintained by hand. In the eighties, hypertext systems like NoteCards, still mostly manual. From nineteen ninety-five, Ward Cunningham's wikis added collaboration but maintenance stayed manual. In the twenty-twenties, tools like Roam and Obsidian added plugins to help. And from twenty twenty-four onward, the LLM Wiki — where the language model finally takes over the bookkeeping. Each era kept the same core idea and automated a little more of the upkeep.
Era System Maintenance 1950s+ Paper Zettelkasten All manual 1980s NoteCards (hypertext) Mostly manual 1995+ Wikis (Cunningham) Manual + collab 2020s Roam, Obsidian Manual + plugins 2024+ LLM Wiki LLM does bookkeeping
Standard page template
Here's what a single wiki page actually looks like. At the top is frontmatter — an id, a type, a status, a confidence score, and a list of sources. Then a title, a one-paragraph summary, and the heart of the page: a list of claims, where every claim carries a source marker and its own confidence number. Finally, relationships — the typed links to other entities, like "composes" pointing to another page's id. This structure is what makes the whole thing machine-maintainable and auditable.
---
id: concept:llm-wiki
type: concept
status: active
confidence: 0.5
sources: []
---
# Title
## Summary
One paragraph.
## Claims
- <Claim.> `[src: raw/...] {conf: 0.5}`
## Relationships
- composes → [[entity-id]]
Citation discipline (the rule)
If you remember one rule from this course, make it this one: every claim cites a raw file, with a confidence number. That single discipline buys you three things. It forces grounding, so the model can't pad answers with its own priors. It enables decay, because a low-confidence claim that's also old can be flagged as stale. And it surfaces contradictions honestly, because when two sources disagree, both citations stay. The confidence math is simple reinforcement: a first observation starts at zero point five, and each independent confirmation closes the gap toward one — point eight, then point nine two, then point nine seven.
Every claim cites a raw file with a confidence number.
Forces grounding (no priors padding)
Enables decay (low conf + old = stale)
Reveals contradictions (both citations stay)
conf = 0.5 (first observation)
conf = 1 - (1 - conf) * 0.6 (each independent reinforcement)
After 1 reinforcement: 0.5 → 0.8
After 2: 0.8 → 0.92
After 3: 0.92 → 0.968
Ingest workflow
Ingest is the workflow you'll run most. Five steps. First, capture the source verbatim into a dated raw file — nothing summarised yet. Second, extract the entities, using the catalogue your schema defines. Third, write or update the wiki — and note this touches multiple pages, each with its source markers. Fourth, reconcile the bookkeeping: the index, the log, the edges file for the graph. Fifth, summarise what changed — pages added or modified, contradictions opened. A healthy ingest usually touches three to ten pages. If you only ever touch one page, you're missing the cross-references that make the wiki valuable.
Capture — verbatim to raw/YYYY-MM-DD-<slug>.mdExtract — entities per SCHEMA's catalogueWrite/update — multi-page touch with [src:] markersReconcile — index.md + log.md + graph/edges.jsonlSummarise — pages added/modified, contradictions opened
A single ingest typically updates 3-10 pages. One-page ingest = missed cross-references.
Lint workflow
Lint is the periodic health check — not every ingest, but every ten or so, or weekly to monthly. It looks for orphans, pages with nothing linking to them; broken wikilinks, where you reference a page that doesn't exist; contradictions, which you resolve by supersession, never silent deletion; stale claims, where confidence has decayed past the threshold and the claim gets marked faded; and citation integrity, confirming every source marker still points to a real file. The governing principle is supersession over deletion — a good audit trail beats a tidy-looking wiki.
Periodic, not per-ingest. Every 10 ingests or weekly/monthly.
Orphans — pages with no inbound linksBroken wikilinks — [[X]] with no XContradictions — supersession, never silent deletionStale claims — decay below 0.2 + 2× half-life → status: fadedCitation integrity — every [src:] file exists
"Supersession over deletion." Audit trail beats tidiness.
— end of Track A —
Track B — Engineering patterns
That closes Track A, the foundations. If you came to understand the idea, you now have it. Track B is for the builders — the next twenty-five minutes get concrete about schema, tooling, failure modes, and wiring this into agents. If that's not you, this is a fine place to stop.
For builders. ~25 min.
Schema design — the highest-leverage decision
Of all the choices you make, the schema is the highest-leverage one — it shapes everything downstream. Four decisions. Pick an entity catalogue of five to eight types; resist the urge to define twenty. Pick a relation catalogue of six to ten, and just use sensible defaults if you're unsure. Set a decay half-life per type — fast-moving things like tools might be ninety days, stable things like decisions a full year. And write explicit privacy rules listing what must never enter the wiki. Above all, change the schema rarely. It's a constitution, not a config file.
Entity catalogue — pick 5-8 types. Avoid 20.Relation catalogue — 6-10. Use the defaults if unsure.Decay half-life per type — fast (tools): 90 days. Stable (decisions): 365 days.Privacy rules — explicit list of what never enters wiki/
Schema updates should be rare . It's a constitution, not a config.
Tooling: Obsidian + Git
You don't need exotic tooling — Obsidian plus Git covers it. The conventions map almost one to one: one markdown file per entity is native; wikilinks with double brackets are native and give you backlinks for free; frontmatter is native and queryable through Dataview; the graph view is a core plugin; and because it's all just a folder of text files, it's completely Git-friendly. The plugins worth adding are Dataview, Templater, and Obsidian Git for automatic version control.
Convention Obsidian One markdown per entity Native Wikilinks [[X]] Native + backlinks Frontmatter Native + Dataview queries Graph view Core plugin Git-friendly Just a folder
Recommended plugins: Dataview, Templater, Obsidian Git.
Failure modes
Now the honest part — how these systems break. The number one failure is claim drift: the wiki keeps asserting things the sources no longer support. Then orphan pages with no inbound links; broken wikilinks pointing nowhere; contradictions, opposite claims about the same subject; stale claims that decay should have faded; and citation integrity failures, where a cited file got moved or deleted. The important thing about drift is that it's rarely catastrophic on day one — it compounds quietly, which is exactly why the lint cadence matters.
Claim drift — wiki says things sources no longer support (#1 failure)Orphan pages — no inbound links, not in indexBroken wikilinks — targets that don't existContradictions — opposite claims, same subjectStale claims — decay applied; mark faded, never deleteCitation integrity — cited file moved or deleted
Drift is rarely catastrophic on day 1. It compounds.
Provenance audit
Here's how you actually check provenance, at two depths. The quick check is a short shell loop: for every page, for every raw file it cites, confirm that file still exists — and print anything broken. Run that often; it's cheap. The deeper check is manual and monthly: pick five claims at random and verify the cited raw source genuinely still says what the claim asserts. If even one fails, that's your signal that a full lint sweep is overdue.
# Quick — broken citation pointers
for page in wiki/*.md; do
for src in $(grep -oE 'raw/[^]]+' "$page"); do
[ -f "$src" ] || echo "BROKEN: $page → $src"
done
done
# Deeper — random claim verification
# Pick 5 claims monthly, check the cited raw still says it
# If 1+ fails → full lint sweep is overdue
Integration with agents — context engineering
This is where it pays off for agent systems. Anthropic's research makes the tension clear: as the context window grows, the model's recall actually decreases — you can't just stuff everything in. So you layer it. The wiki is your persistent memory. RAG handles just-in-time retrieval of the freshest material. Sub-agents work in clean context windows and hand back only summaries. And the composition is: try the wiki first because it's cheap, fall back to RAG when needed, and if you discover something genuinely new, file it back into the wiki so the system compounds.
"As context window increases, model recall decreases." — Anthropic
Wiki = persistent memory layer
RAG = just-in-time retrieval
Sub-agents = clean context windows, return summaries
Composition: Wiki cheap → RAG fallback → file back to wiki if novel
Five health metrics
To know whether your wiki is healthy, track five metrics. Citation density should be at least one citation per claim. Provenance integrity should be a hundred percent — every citation points to a raw file that exists. Orphan rate under five percent. Freshness, meaning twenty to fifty percent of pages touched in the last ninety days — too low means it's stagnant, too high means it's churning. And a confidence median above zero point six, with faded claims under ten percent. Baseline these today, then re-check monthly; the regular cadence is what catches drift before it compounds.
Metric Healthy Citation density ≥ 1.0 per claim Provenance integrity 100% citations point to existing raw Orphan rate < 5% Freshness 20-50% touched in 90d Confidence median > 0.6; faded < 10%
Baseline today. Re-check monthly. The cadence catches drift before it compounds.
Anti-patterns to avoid
A quick catalogue of what not to do. Don't keep your schema in chat history — it belongs in a file. Don't define twenty-plus entity types; that's over-engineering. Don't collapse everything into one mega-citation at the end of a page — it defeats the audit. Don't write claims without confidence numbers. Don't silently delete; supersede instead. Don't do single-page ingests; you'll miss cross-references. Don't lint on every ingest; it's too noisy. And never run the wiki without version control.
Schema in chat history (not in file)
20+ entity types (over-engineered)
One mega-citation at end of page (defeats audit)
Claims without confidence numbers
Silent deletion instead of supersession
Single-page ingest (missed cross-references)
Lint per ingest (too noisy)
No version control on the wiki
Production checklist
Before you call a wiki production-ready, run this checklist. On the foundation side: the three layers exist, the page template is followed, every claim is cited, and wikilinks use stable ids. On the engineering side: the schema lives in a file rather than chat, the wiki is in Git, your five metrics are baselined, privacy rules are explicit, and — crucially — you've tested it with both a real human and a real LLM driving it. That last one catches problems no static review will.
Foundation
3 layers exist
Page template followed
Every claim cited
Wikilinks use IDs
Engineering
Schema in file (not chat)
Wiki in git
5 metrics baselined
Privacy rules explicit
Tested on real human + LLM
Sources cited
Everything in this course is grounded, so here are the primary sources. Karpathy's original gist is the seed of the whole pattern. The twenty-twenty Lewis paper is the canonical reference for RAG. The Zettelkasten entry covers the historical method. The Obsidian vault docs are your tooling starting point. And Anthropic's context-engineering article underpins the agent-integration section. Fittingly, every claim in this material is itself cited inside the wiki under the wiki folder.
All claims cited inside the wiki under /wiki/.
Questions?
And that's LLM Wiki 101. We went from the core idea — a persistent, compounding, LLM-maintained knowledge base — through its Zettelkasten roots, the three-layer structure, the ingest and lint workflows, and the engineering details to run it in production. The best way to learn it is to build a small one this week and ingest five real sources. Thank you, and I'm happy to take your questions.
LLM Wiki 101 · 2026-05-09