Skip to main content

​
The Problem

Agents read files. The question is: which files, in what order, with what structure? A flat directory of Markdown files forces the agent to read everything or guess. A well-structured KB lets the agent navigate to exactly what it needs in 1-2 file reads.

​
JIT Retrieval

The approach on this page has a name: JIT Retrieval, also called Explore RAG. It is knowledge retrieval with no vector store, no embeddings, and no indexing pipeline. The knowledge is structured markdown in git, and the agent navigates it with the file tools it already has — glob, grep, read. There is nothing to stand up and nothing to keep in sync with the files. That sounds like a limitation. It is closer to the opposite, for five reasons:
  1. Routing tables are human-authored rerankers. A “read when…” column encodes domain expertise about what is relevant to what. That beats statistical similarity, because it is a judgment about relevance rather than a measurement of resemblance.
  2. Negative knowledge saves more time than positive routing. Recording what is not here prevents the most wasteful searches — the ones that end in nothing after reading half the corpus.
  3. Hierarchical navigation is O(log n). Root index → domain index → file. Adding files does not lengthen the path.
  4. No infrastructure. No embedding model, no vector store, no indexing job, no re-embedding when a document changes. A git commit is the entire update pipeline.
  5. It works to roughly 500 files per KB. Beyond that, federate rather than growing a single corpus.
The quality metric is the right file in three hops or fewer. That is the number to test against, and the reason index files stay short. Maintenance is two-agent: a Curator with read-write access who owns structure and currency, and a Navigator with read-only access who consumes it. Separating them keeps the corpus from being quietly reshaped by whoever last needed an answer.
The honest trade-off: somebody has to build the routing tables. Vector RAG is automatic but dumber; structured routing is manual but dramatically more accurate for domain-specific Q&A. If the corpus is large, general, and nobody will own it, this is the wrong tool.
The rest of this page is how the routing is built.

​
The Agent-Consumption Weighting

Not all documentation types are equally useful to agents. Based on Diataxis (Daniele Procida), we weight the four document types for agent consumption: This inverts the typical human documentation priority. Humans want tutorials first; agents want reference first.

​
Directory Layout

A KB serving both human and agent consumers:

​
The Index Pattern

The index.md at every directory level is the agent’s entry point. It contains a routing table — not content, but pointers:
The “Read when…” column is critical. It tells the agent under what conditions to read the file. This is more useful than a document type label — it encodes priority and relevance.

​
Routing precedence

  • “Always read first” — mandatory context
  • “Task involves X” — conditional on the current task
  • “Only when stuck” — fallback for debugging

​
Progressive Disclosure

The agent reads in layers:
1

Read the root index

~50 lines. The agent sees what domains exist and which are relevant to its task.
2

Read the domain index

~30 lines. The agent sees specific topics and their routing conditions.
3

Read the relevant file

Full content — but only for the 1-3 files that match the task. Not the whole KB.
A well-structured KB turns a 50-file knowledge base into 2-3 file reads. The agent spends tokens on knowledge, not navigation.

​
Two KB Types

The lab uses two distinct KB architectures:

​
Code-Agent KB (task-driven)

For agents that execute coding tasks. Optimized for lookup and action.
  • Root index.md ≤100 lines
  • VOCABULARY.md — controlled vocabulary for consistent terminology
  • Domain directories with per-domain index.md
  • Cheatsheets and structured reference files
  • Update cadence: when frameworks or tools change
  • Agent roles: Curator (read-write maintenance) + Navigator (read-only consumption)

​
Research-Partner KB (question-driven)

For research synthesis and strategic context. Optimized for understanding and connections.
  • CLAUDE.md as session bridge (routing + context)
  • synthesis/ hierarchy with theme index and per-theme docs
  • Immutable source conversations
  • Update cadence: after each research conversation
  • Agent role: session bridge (one agent, dual modes — synthesis intake + Q&A)
Don’t mix them. The same domain can appear in both KB types with different purposes. A code-agent KB about Spring Security has migration recipes. A research-partner KB about Spring Security has strategic analysis of the migration’s impact on the product roadmap.

​
Design Rules

  1. Index files contain pointers, not content. If you’re putting explanation in the index, it belongs in a separate file.
  2. Reference format should be greppable. Consistent headings, predictable structure, machine-parseable tables. The agent’s first retrieval is typically Grep for a keyword, then Read of the matching file.
  3. One topic per file. A file that covers both “how to migrate security” and “why the security API changed” should be split. The agent might need one without the other.
  4. Negative knowledge is explicit. If something is out of scope, say so in the index. “This KB does NOT cover: deployment, monitoring, performance tuning.” This prevents the agent from searching fruitlessly.
  5. KnowledgeRefs are relative paths. In experiment datasets, knowledgeRefs point to files relative to knowledgeBaseDir. Typically 1-5 directory refs per item (usually 2-3). The agent reads the pointed-to index, then drills down.

​
Evidence

​
Code Coverage v1

Variant 3 (flat knowledge base) vs Variant 4 (structured skills) — identical content, different packaging. Variant 4 outperformed Variant 3 in efficiency metrics. The agent using structured skills showed 0% JAR_INSPECT — it stopped needing to inspect dependencies because the knowledge was delivered proactively.

​
SkillsBench

SkillsBench confirmed that structure matters: AgentSkillOS found that hierarchically structured skills outperform flat files even with identical content.

​
Partial Knowledge Paradox

Some knowledge without structure decreases performance (Code Coverage v1, finding #4). An unstructured KB is worse than no KB — the agent wastes tokens navigating and gets confused by contradictory or irrelevant information.

​
Further Reading

For a narrative walkthrough of how these patterns were discovered and applied across 1,772 files and six federated KBs, see Look Ma, No RAG! on the blog.

Knowledge Base Freshness

How knowledge stays true after it’s written — drift, rituals, and the trust principle

Forge

The methodology this KB pattern belongs to, and the /forge-kb command that builds one

Your First Research Agent

JIT Retrieval end to end — five papers, twenty minutes, a corpus you can query