LLM Wiki 101

What is an LLM Wiki?

A design pattern where the LLM maintains a knowledge base instead of re-deriving context per query.

An LLM Wiki is a folder of markdown files that an LLM both reads and writes — a knowledge base that compounds across sessions. The pattern was articulated by Andrej Karpathy in a short gist that opens with the central claim:

"The wiki is a persistent, compounding artifact."

That single line is the difference between this pattern and almost everything else in the AI-knowledge space. Most retrieval systems are stateless: every query re-derives an answer from raw chunks. An LLM Wiki is stateful: every query draws from accumulated synthesis, and (optionally) updates that synthesis with what it learned from the new question.

Three layers, three operations

The pattern has exactly three layers:

• Raw sources — captured documents, immutable after capture
• Wiki — synthesised entity pages the LLM writes and maintains
• Schema — the constitution: rules, entity types, lifecycle policies

And exactly three operations:

• Ingest — turn a new source into wiki updates (multi-page)
• Query — retrieve accumulated knowledge to answer a question
• Lint — periodic health check (contradictions, orphans, decay)

The division of labour

Humans curate sources and judge quality. LLMs do the bookkeeping — updating cross-references, maintaining consistency, surfacing contradictions. The split is not "LLM does the work and human reviews" — it's "human does the curation, LLM does the gardening."

Why not just RAG?

Retrieval-Augmented Generation works for lookup. LLM Wiki adds the layer above: synthesis that compounds.

Retrieval-Augmented Generation (RAG) — Lewis et al., 2020 — combines model weights (parametric memory) with an external retrievable corpus (non-parametric memory). For each query, the model retrieves relevant chunks and conditions generation on them. It works. It's the default for AI-knowledge systems today.

But the original RAG paper itself flagged two open problems:

"Providing provenance for their decisions and updating their world knowledge remain open research problems."

LLM Wiki addresses both differently than later RAG variants:

• Provenance → every wiki claim cites a specific raw file with a confidence number
• Updating → ingest re-touches every page that mentions an entity, surfacing contradictions for human resolution

RAG vs LLM Wiki — the canonical comparison

The production hybrid

Real systems use both. RAG retrieves raw chunks (fast, current); the wiki holds synthesised understanding (compound, traceable). Anthropic's context-engineering guidance phrases this as "structured note-taking provides persistent memory with minimal overhead" — note-taking is the wiki, just-in-time retrieval is the RAG.

A practical query flow:

1. Hit the wiki first — cheap, synthesised
2. If wiki is silent or low-confidence, fall through to RAG over raw documents — expensive, raw
3. If RAG produces a useful answer, optionally file it back as a new wiki page (the "save valuable findings" half of the query operation)

Ancestry: Zettelkasten and friends

The architecture is 70 years old. Niklas Luhmann did the manual version with 90,000 paper cards. The novelty is the maintenance layer.

A Zettelkasten ("slipbox" in German) is a knowledge management system based on three principles:

1. Atomicity — one idea per note
2. Cross-referencing — explicit links between notes
3. Unique identification — stable IDs (Luhmann used numbered hierarchies)

Niklas Luhmann, a German sociologist, built a Zettelkasten of approximately 90,000 index cards across his career. He credited it as the engine behind ~50 books and ~550 articles. His system was digitised in 2019 and is now publicly accessible.

The genealogy

The lineage from paper cards to LLM Wiki is direct:

The atomic notes, explicit linking, and metadata-driven retrieval that define LLM Wiki descend directly from the index card. The architecture is well-validated by 70 years of practice.

The teaching shortcut

If a student understands Zettelkasten, they understand LLM Wiki. The framing:

"LLM Wiki = Zettelkasten + an LLM doing the maintenance."

What's manual in classical Zettelkasten:

• Writing each card
• Inserting links between cards
• Flipping through the box to spot orphans
• Re-reading and re-writing as new cards arrive

All four become LLM operations. The human stays in the curation/judgement role; the LLM handles the gardening that previously consumed hours per week.

The three layers

Raw is immutable. Wiki is synthesised. Schema is constitutional. Different mutability rules per layer is the discipline that prevents drift.

Raw layer (`raw/`)

Captured primary sources — verbatim, never edited after capture:

raw/
├── 2026-05-09-karpathy-llm-wiki-gist.md
├── 2026-05-09-rag-paper.md
└── ...

Frontmatter records source_url, ingested_at, source_type. The text itself is what the source said when ingested. If the upstream changes, you re-ingest as a new dated raw file — never edit the old one. This preserves audit trail.

Wiki layer (`wiki/`)

LLM-generated entity pages. One concept per file:

wiki/
├── concept:llm-wiki.md
├── concept:rag.md
├── pattern:citation-discipline.md
└── ...

Pages are synthesised, not copied. Every claim cites at least one raw source: [src: raw/...] {conf: 0.5}. Pages get updated as new sources arrive — confidence rises, claims are reinforced, contradictions surface for human resolution.

Schema layer (SCHEMA.md / CLAUDE.md / AGENTS.md)

The constitution. Defines:

• Entity types (concept, pattern, principle, risk, tool, decision)
• Relation types (composes, depends-on, mitigates, etc.)
• Confidence + decay rules
• Lint policies
• Privacy rules

The schema is the discipline mechanism. Without it, an LLM-maintained wiki diverges within weeks. With it, the artifact stays coherent for years. Schema updates should be rare and deliberate — it's a constitution, not a config.

Why three layers, not two?

A two-layer system (raw + wiki, no schema) lacks the discipline mechanism. The LLM has no contract for how to write pages, what entities mean, when to lint. After dozens of sources, every ingest's "vibe" varies — formatting drifts, citation styles drift, page granularity drifts. The schema is what makes this maintainable, not just one-shot synthesis.

Page template & citation discipline

Every page follows the same shape. Every claim cites a source with a confidence number. The discipline is what makes the artifact trustworthy.

The standard page template

---
id: <entity-type>:<kebab-slug>
type: <entity-type>
title: <Human-readable title>
status: active                    # active | stale | faded | orphan
confidence: 0.5
sources: []                       # raw/ files that contributed
created: YYYY-MM-DD
updated: YYYY-MM-DD
updated_log:
  - YYYY-MM-DD: created
tiers: semantic
half_life_days: 180
tags: []
---

# <Title>

## Summary
One paragraph. Plain language.

## Claims
- <Claim.> `[src: raw/...] {conf: 0.5}`
- <Another.> `[src: raw/A.md, raw/B.md] {conf: 0.8}`

## Relationships
- composes → [[entity-id]] `{conf: 0.x}`

## Open questions
- [ ] <question>

## Changelog
- YYYY-MM-DD — created

The citation rule

Every claim cites at least one raw file with a confidence number. No floating assertions. If you can't cite it, it doesn't enter the wiki.

The format does three things at once:

• Forces grounding — the LLM can't pad pages with priors
• Enables decay — claims with low confidence and old timestamps mark themselves stale
• Reveals contradictions — when source B contradicts source A, both citations stay; supersession becomes explicit

Confidence math

First observation:               conf = 0.5
Reinforcement (independent src): conf = 1 - (1 - conf) * 0.6
                                 (asymptotes toward 1.0)
Contradiction:                   open supersession candidate
                                 (don't lower silently)

After 1 reinforcement: 0.5 → 0.8. After 2: 0.8 → 0.92. After 3: 0.92 → 0.968. Diminishing returns by design — once a claim is well-established, additional citations bump confidence less.

Anti-patterns

• ❌ "[src: official docs]" — meaningless. Must point to raw/<filename>.md
• ❌ One mega-citation at end of page — defeats per-claim audit. Cite per claim.
• ❌ No confidence number — without it, decay is impossible
• ❌ Synthesising claims that exceed sources — if you can't trace it to text in raw/, don't write it

Operating it by hand

Walk through one ingest, one query, one lint — without any tooling. Builds the muscle memory before automation.

Ingest workflow (manual)

1. Capture — point the LLM at a source. Save verbatim to raw/YYYY-MM-DD-<slug>.md with frontmatter.
2. Extract — identify entities mentioned (per SCHEMA's catalogue). For each: does it exist? Update. Else create.
3. Write/update — for each entity page touched, add new claims with [src:] markers. Bump confidence on reinforced claims.
4. Reconcile — update index.md (catalogue) and log.md (chronological audit). Update graph/edges.jsonl.
5. Summarise — tell the human: pages added, pages modified, contradictions opened.

Note: a single ingest typically updates 3-10 pages. If you only touched one, you missed cross-references — re-scan.

Query workflow

1. Parse — identify entities in the question
2. Retrieve — grep wiki/ for matching IDs; read those pages; follow [[wikilinks]] 1-2 hops
3. Synthesise — compose answer from accumulated claims; cite specific pages
4. (Optional) File back — if the question is novel + reusable, save Q+A as wiki/q-<slug>.md

Lint workflow

Periodic, not per-ingest. Recommended cadence: every 10 ingests, or weekly/monthly.

Checks:

• Orphans — pages with no inbound links and not in index.md
• Broken wikilinks — [[entity-id]] targets that don't exist
• Contradictions — opposite claims on same subject. Propose supersession; never silently delete.
• Stale claims — apply decay; mark status: faded when confidence falls below 0.2 and untouched for 2× half-life
• Citation integrity — every [src:] marker points to a file that exists

Output: a dated audit artefact raw/lint-YYYY-MM-DD.md listing what was found, resolved, escalated.

Schema design choices

The schema is rare-update by design. Get the entity catalogue + decay parameters right early; resist casual edits later.

Schema design is the highest-leverage decision in an LLM Wiki. Once it's set and the LLM has been operating against it for weeks, changing schema is expensive (every existing page may need migration). The first version should be deliberate.

Design checklist

1. Entity catalogue — pick 5-8 types. Avoid 20.
2. Relation catalogue — pick 6-10. The default set (composes, depends-on, mitigates, etc.) covers most domains.
3. Decay half-life per type — fast-moving (tools): 90 days. Stable (decisions): 365 days.
4. Privacy rules — explicit list of what never enters wiki/
5. Confidence formula — keep the default unless you have a reason

Common entity catalogues by domain

Anti-patterns in schema design

• ❌ 20+ entity types — over-engineered; harder for LLM to choose
• ❌ Vague types ("note", "thing") — types should clearly partition the space
• ❌ Schema in chat history — LLM forgets between sessions; commit to file
• ❌ Schema longer than the wiki — over-specified before the wiki has signal

Tooling: Obsidian + Git

Local-first, markdown-native, free. The viewer doesn't own your data — the wiki is just a folder.

Why Obsidian fits the pattern

Recommended plugin set

Git as the version layer

Treat your wiki folder as a git repo. Benefits:

• Audit trail — every change versioned with a commit message
• Branching for experiments (try a new schema in a branch)
• Diff-able edits — see exactly what the LLM changed in a given session
• Sync via GitHub if you want multi-device access

Practical setup:

cd ~/your-wiki-folder
git init
echo "raw/private/" > .gitignore   # if you have private sources
git add .
git commit -m "Initial wiki snapshot"

Then the LLM (Claude Code, Cursor, whatever) edits files; you periodically git diff to review changes; commit when satisfied.

Failure modes

Claim drift is the #1 failure. Plus orphans, contradictions, schema-conformance gaps. Mitigations are layered.

Claim drift = the wiki gradually saying things its raw sources no longer support. Rarely catastrophic on day 1; compounds. After 6 months without lint, a 50-page wiki can have 10-20% of claims that no longer match their sources.

Four ways drift happens

1. Re-write without re-read — LLM updates a page based on its priors, not the cited source
2. Source updates upstream — our raw is dated; upstream blog has been edited
3. Cascading reinforcement — Page A cites Page B which cites Page C; trail to raw thins
4. Schema evolution — schema added a type; old pages don't conform; lint flags but nobody migrates

Other failure modes

Why "supersession over deletion"

Deleting old claims feels tidy. It's wrong. The audit trail matters more than tidiness — six months from now, you may need to know why the wiki used to say X. Mark old claims status: stale, link to the superseding claim, leave them in the file.

Provenance audit

The deepest drift check — pick 5 random claims, manually verify the cited raw still says what's claimed:

for page in wiki/*.md; do
  for src in $(grep -oE 'raw/[^]]+' "$page"); do
    [ -f "$src" ] || echo "BROKEN CITATION: $page → $src"
  done
done

Run monthly. If 1+ random claim fails verification, do a full lint sweep.

Integration with agents (the production angle)

LLM Wiki fits inside Anthropic's framing of context engineering as the persistent memory layer. Compose with just-in-time retrieval.

Modern agents — "LLMs autonomously using tools in a loop" — have a fixed context budget that gets eaten by tool calls and intermediate reasoning. Anthropic's context-engineering guidance frames the constraint:

"As the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases."

This is called context rot. It motivates treating context as a scarce resource — and makes the case for LLM Wiki:

"Structured note-taking enables persistent memory outside the context window — agents write notes that get pulled back in later, providing 'persistent memory with minimal overhead.'"

The just-in-time pattern

Anthropic recommends agents maintain "lightweight identifiers (file paths, stored queries, web links, etc.) and use these references to dynamically load data into context at runtime." This is the LLM Wiki shape — the wiki holds references and synthesised pages; the agent loads what it needs on demand.

How wiki + RAG + agents compose

Query arrives
  ↓
Agent checks LLM Wiki first (cheap, synthesised, traceable)
  ↓
If wiki has the answer → cite + return
If wiki is silent or low-confidence:
  ↓
  Agent invokes RAG over raw documents (expensive, fresh)
  ↓
  If RAG produces useful answer → return
  ↓
  Optionally: file back as new wiki page (compounds)

Wiki is the cache; RAG is the source-of-truth fallback. Together they're cheaper, more traceable, and more compounding than either alone.

Sub-agent architecture

Anthropic also recommends specialised sub-agents that handle focused tasks with clean context windows, returning condensed summaries (1,000-2,000 tokens) to a coordinator. LLM Wiki composes naturally:

• Coordinator dispatches a sub-agent per query
• Sub-agent reads the wiki (clean context), produces an answer
• Sub-agent optionally writes a new wiki page if it learned something
• Coordinator gets a summary back, no context bloat

Evaluation — how do I know my wiki is healthy?

Five metrics. Baseline today, re-check monthly. Wiki health is observable — if you measure it.

An LLM Wiki has no natural "test set". Health is observable through metrics that capture the failure modes from Module 09.

Five metrics

Per-page quality scoring

For each modified page during lint, rate 0-1 on:

• Cites sources for every claim?
• Internally consistent?
• Well-structured (frontmatter + sections + relationships)?
• Atomic (one concept) vs sprawling?

Pages below 0.5 → flag for rewrite.

Sampling-based provenance audit

Each lint pass: pick 5 random claims, manually verify the cited raw file still says what's claimed. If 1+ fails, full sweep is overdue.

What you don't measure

• Page count — bigger isn't healthier; can mean sprawl
• Total words — same; the right size depends on the domain
• "Coverage" of sources — citing every raw file isn't a goal; some sources may not deserve a wiki page

Cheatsheet & resources

One-page reference. What to read next.

Quality checklist

Foundations

• ☐ Three layers exist (raw/, wiki/, SCHEMA.md)
• ☐ Raw files have frontmatter (source_url, ingested_at, source_type)
• ☐ Wiki pages follow the standard template
• ☐ Every claim cites a raw file with confidence
• ☐ Wikilinks use entity IDs ([[concept:foo]])

Operations

• ☐ Ingest touches multiple pages (rule of thumb: 3-10)
• ☐ Query is wiki-first, RAG-fallback
• ☐ Lint runs every 10 ingests or weekly
• ☐ Provenance audit runs monthly
• ☐ Supersession is used; no silent deletions

Engineering

• ☐ Schema is committed to file (not chat history)
• ☐ Wiki is in git
• ☐ Health metrics defined + baselined
• ☐ Privacy/secrets rules explicit + enforced
• ☐ Tested across at least one human + one LLM session

Authoritative sources cited

• Andrej Karpathy — LLM Wiki gist (the seed)
• Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020)
• Zettelkasten on Wikipedia — the 70-year ancestor
• Obsidian — Vault concepts
• Anthropic — Effective context engineering for AI agents

What to read next

• Anthropic's Memory + context-engineering articles — production agent perspective
• Sönke Ahrens — How to Take Smart Notes (book) — modern Zettelkasten guide
• Obsidian Dataview docs — programmatic queries against your wiki
• Karpathy's other gists — frequent design patterns for LLM workflows

Internal artefacts

• Wiki index — 11 cited pages
• Slide deck (EN) · Slide deck (TH)
• หลักสูตรภาษาไทย