LLM Wiki vs RAG: Two Ways to Build a Knowledge Base

Retrieval-Augmented Generation grounds a model's answer in an external corpus. A retriever finds passages, and the model writes its answer from those passages rather than from its weights alone ([evidence ev-201]). The retriever does not have to be exotic. It can rank by keywords, by vectors, or by following links; the important part is that the answer is grounded in fetched text ([evidence ev-209]).

The appeal is coverage. Point the retriever at a document store and the model can answer questions about material it was never trained on, without fine-tuning.

The common implementation is vector RAG, and it has a fixed shape:

1. Split the source documents into chunks (Text Chunking).
2. Embed each chunk with an Embedding Model, turning text into vectors ([evidence ev-203]).
3. Embed the query the same way and retrieve the nearest vectors ([evidence ev-203]).
4. Optionally rerank the candidates before handing them to the model ([evidence ev-205]).

Every step adds a component to run and a place for quality to leak. Chunk boundaries cut sentences in half. A weak embedding model places unrelated passages close together. A query phrased in the user's words may sit far from the passage that answers it.

An LLM Wiki pre-compiles sources into interlinked markdown pages with an index; retrieval follows explicit links and the index the model reads ([evidence ev-201]). Nothing is cut into anonymous chunks. A source becomes a page, and each page is connected to the concepts and entities it touches.

The Index File is the entry point: a catalog of what exists, rewritten as sources arrive. From there the model opens pages, follows the wiki's internal links, and reads whole sections rather than fragments. There is no similarity threshold to tune because there is no similarity search in the base method.

The table is a starting point, not a verdict. Each column describes a default, and the defaults can be mixed.

The clearest difference is the retrieval mechanism. An embedding model turns text into vectors so that passages with similar meaning sit close together, which is what makes Vector Search work ([evidence ev-203]). That handles paraphrase well: a question worded one way finds a source worded another.

An LLM wiki retrieves by structure instead. The model reads the index, selects the pages that look relevant, and follows links from those pages to related ones. This is closer to how a person uses a reference work: the table of contents and the cross-references do the work.

An LLM wiki is still a form of RAG. What changes is the retrieval method, not the goal ([evidence ev-209]).

The behavioural split matters more than the mechanical one. RAG rediscovers knowledge on every query and does not accumulate; an LLM wiki compiles once, updates as sources arrive, and compounds ([evidence ev-201]).

In a RAG pipeline, the work of understanding a source is thrown away after the answer. The next query re-embeds, re-retrieves, and re-summarizes similar material. In a wiki, the first ingest leaves a page behind. Every later answer leans on that page, and a correction made once fixes it everywhere. Compounding Knowledge is the payoff, and it is the reason a wiki gets more useful as it grows while a chunk store stays roughly as useful as its last query.

Vector search is not the only retriever in a RAG system, and the alternative is older. With no embedding model, retrieval falls back to BM25 keyword search, which ranks passages by term overlap ([evidence ev-203]). BM25 is strong for clause numbers, product names, and proper nouns, exactly the queries where an embedding blurs the distinction between similar-looking strings. Vectors are strong when the user's wording differs from the source ([evidence ev-203]).

With an embedding model in place, keyword and vector search can run together as a hybrid ([evidence ev-204]). Semantic Search catches paraphrase; BM25 catches exact terms. The combination covers both, which is why hybrid retrieval is common once a corpus is large enough to need an embedding model at all.

RAG does not require a cloud service. qmd is a local engine that runs hybrid BM25 and vector search with LLM re-ranking, and it exposes both a command line interface and a Model Context Protocol server ([evidence ev-207]). The MCP server matters because it lets an agent query the index as a tool rather than through custom glue.

Retrieval returns candidates, and candidates are not ordered well. A reranking model re-scores them, sorting the plausible items to the top before the model reads them ([evidence ev-205]). Reranking buys precision at the cost of another model in the loop.

Without a reranker, some tools may not even expose a similarity threshold, so there is no clean way to say that nothing here is relevant enough ([evidence ev-205]). That matters when a question falls outside the corpus: a system that always returns its nearest chunks will always hand the model something to summarize.

RAG fits corpora that change faster than anyone can summarize them, and questions that touch many documents without needing a durable synthesis. It is also the pragmatic choice when the source of truth is already a search index or a document store you cannot restructure. You get grounding without committing to a maintained body of pages.

An LLM wiki fits a body of material you revisit. If the same questions return in new forms, the compilation pays for itself, because the answer to the next question is partly written. It is also the better fit when you need to inspect why a result appeared: pages and links are readable, while a ranked list of chunks is not. The wiki doubles as durable Agent Memory, a store the model reopens across sessions ([evidence ev-201]).

No. No vector database is required to start an LLM wiki; add vector search only when the corpus outgrows what links and an index can navigate, and the two methods compose ([evidence ev-209], [evidence ev-201]). A practical arrangement starts with pages and an index, then layers embeddings and a reranker on top once keyword and link navigation stop keeping up. The wiki remains the compiled layer; the vector index speeds up finding a page in it.

Ask yourself a few questions. Does the corpus change constantly? Lean toward RAG. Do you revisit the same topic and need the answer to stick? Lean toward a wiki. Do you need to see why a result was returned? Pages and links are easier to audit than vectors. Most teams that answer yes to more than one end up running both.

What is a RAG knowledge base?

A RAG knowledge base is a corpus a retriever searches so a model can ground its answer in fetched passages ([evidence ev-201]). The base is the external knowledge; RAG is the pattern of retrieving from it and generating from what was retrieved ([evidence ev-209]).

Is an LLM wiki a RAG?

Yes. An LLM wiki is a form of retrieval-augmented generation; the difference is that retrieval follows links and an index rather than embedding similarity ([evidence ev-209]).

Does an LLM wiki need a vector database?

No. Links and an index are enough to start, and a vector database is added only when the corpus outgrows what they can navigate ([evidence ev-201], [evidence ev-209]).

Which retrieval method is better for proper nouns and clause numbers?

Keyword retrieval such as BM25, because vectors tend to blur similarly written strings ([evidence ev-203]). Use vectors for paraphrase and hybrid search to cover both ([evidence ev-204]).

Why does a wiki compound but a RAG index does not?

A RAG pipeline rediscovers knowledge per query and discards the synthesis, while a wiki compiles once and updates as sources arrive ([evidence ev-201]).

Can I run hybrid retrieval without a cloud service?

Yes. qmd runs hybrid BM25 and vector search with LLM re-ranking locally, with a CLI and an MCP server ([evidence ev-207]).

The XKnow Knowledge Base is a pre-compiled LLM wiki you own: a set of linked SEO and SaaS pages with an index and a changelog, ready for an agent to read and extend. Instead of building the schema and the first pages yourself, you start from a structure that already holds together, and you can layer on embeddings later. See What is an LLM wiki for the model behind it.

• What is an LLM wiki — the definition and the three operations.
• Karpathy's LLM wiki — where the pattern came from.
• add embeddings to a knowledge base — when and how to add vector search.