How to Add Embeddings to a Markdown Knowledge Base (No Code)

A knowledge base is a set of materials plus retrieval settings. The chat model does not read your files; it organises an answer from what the retrieval layer recalled, so recall quality comes first ([evidence ev-203]). Adding an embedding model changes the retrieval layer, not the files.

With keyword retrieval, a query has to share wording with your Markdown notes. With an embedding model, each chunk becomes a vector, and a query can match a passage that means the same thing in different words. That is the difference between BM25 and Vector Search: BM25 rewards term overlap, vector search rewards meaning. Useful setups combine both, so treat this as adding a method rather than replacing one. Semantic Search names what the vector half buys you.

You do not have to start there: setting the embedding model to "Do not use" falls back to BM25 keyword retrieval, so keyword search is a fine starting point until your questions and notes stop sharing vocabulary ([evidence ev-203]).

When you create a knowledge base you pick one of three retrieval choices: no embedding (BM25 only), a cloud embedding model (BM25 plus vector hybrid), or a locally downloaded embedding model (BM25 plus local vector) ([evidence ev-204]). The choice is where vectors are computed, not whether keyword matching survives; BM25 still does part of the work in all three.

• No embedding: nothing to download, nothing leaves the machine, works offline, but matching stays limited to term overlap.
• Cloud embedding model: no local download and strong general-purpose vectors, but every embed call goes to a provider.
• Local embedding model: vectors stay on your machine with no per-call cost, at the price of a one-time download and some local compute.

Pick the first if unsure, the second for quality without local files, and the third if privacy or offline use matters.

The defaults are Top K 6 with a range from 1 to 50, smart chunking on, chunk size 1024 tokens, overlap size 200, and a blank line as the separator ([evidence ev-205]). Two matter most. Chunk size and overlap decide what a single retrieved unit contains; that is Text Chunking, and it determines whether one good answer is split across two chunks that never appear together. Top K decides how many chunks reach the model, and six is a reasonable default to leave alone until you are tuning.

One default is easy to miss: the similarity threshold appears only after a reranking model is configured ([evidence ev-205]). If you cannot find it, you have not configured Reranking yet.

1. Open Knowledge Base from the left navigation.
2. Click Add, then name the base using "object plus purpose" rather than a generic label like "Materials" or "Test" ([evidence ev-204]). The name states what the base holds and what it is for, which keeps retrieval testing honest.
3. Select an embedding model, or choose "Do not use" to stay keyword-only ([evidence ev-204]).
4. Click Create.
5. Open Add Materials and choose File, Note, Directory, or Link ([evidence ev-204]). For a Markdown vault, Directory or File is the usual entry point.
6. Wait for processing, then confirm the materials show as ready. Spot-check the extracted text and chunks for garbled text, missing pages, or ordering errors ([evidence ev-204]). This catches parsing problems before they hide inside a retrieval score.

1. Open Settings, then Local Models.
2. In the Embedding Models section, download a model ([evidence ev-204]).
3. Return to the knowledge base and select the downloaded model as its embedding model ([evidence ev-204]).
4. Let the base build its vector index, then re-run your recall questions.

Enabling an embedding model on a base that ran on BM25 builds a vector index over the existing material ([evidence ev-204]). One warning that can save an afternoon: changing the embedding model later triggers a rebuild, because vectors from different models cannot be mixed ([evidence ev-204]). Treat the choice as a commitment, not a toggle.

Point the Add Materials step at a directory instead of typing notes by hand, let the base chunk the files, and pick the embedding model after the first processing pass so you can compare keyword and hybrid recall on the same material. Keep the source folder as the source of truth; the base is an index over it, not a replacement.

If your notes live in an Obsidian vault, the same logic applies: point at the Markdown and let chunking operate on headings and paragraphs rather than on the app. LLM Wiki material fits well because it is plain Markdown with stable structure, which makes chunk boundaries predictable. For a terminal option, qmd is a local search engine for Markdown files with hybrid BM25 and vector search plus LLM re-ranking, shipped as both a CLI and a Model Context Protocol server ([evidence ev-207]). The CLI serves a human; the MCP server exposes the same search to an agent. To see how this differs from a retrieval-only stack, compare LLM wiki vs RAG.

A knowledge base is not working until you can show it recalls the right source. Test with real questions whose answers you already know, and confirm the correct source appears in the first few results ([evidence ev-206]). Use three to five fixed questions and reuse the same set every time as a regression baseline ([evidence ev-206]). Write questions the way you would ask them, not the way your notes are worded; that mismatch is what vector search is meant to close.

Then change one setting per round, retest with the same questions, and keep a change only if it consistently improves results ([evidence ev-206]). The most common mistake is changing the embedding model and the chunk size in the same round, which leaves you unable to tell which caused the change ([evidence ev-204]). Chunking settings also affect only material added afterward, so existing material must be reindexed to use new chunking ([evidence ev-205]).

Work from parsing downward. A ranking fix cannot repair a chunk that was never parsed correctly.

Read it as a decision order, not a menu.

It is tempting to reach for a stronger chat model when answers disappoint. The chat model only organises the answer from what was recalled, so recall quality comes first ([evidence ev-203]). If the wrong chunks arrive, a better writer still writes from the wrong material. Get the retrieval layer right, prove it with fixed questions, then worry about the model above it. What is an LLM wiki explains why a compiled wiki makes that layer predictable, and Karpathy's LLM wiki shows the idea applied to a personal corpus.

Do I need an embedding model to use a knowledge base?

No. A base can run with no embedding model, falling back to BM25 keyword retrieval ([evidence ev-203]). Add one only when keyword matching misses questions worded differently from your notes.

What is the difference between BM25 and vector search?

BM25 rewards shared terms; vector search rewards shared meaning. A hybrid setup runs both, so an embedded base still uses keyword matching ([evidence ev-204]).

Why can I not find the similarity threshold setting?

It appears only after a reranking model is configured ([evidence ev-205]). Add a reranking model first, then return to retrieval settings.

Do I have to reindex when I change the chunk size?

Yes. Chunking changes affect only material added afterward, so existing material must be reindexed ([evidence ev-205]). Changing the embedding model also triggers a rebuild because vectors from different models cannot be mixed ([evidence ev-204]).

Can I add embeddings to a Markdown folder without writing code?

Yes. Create a knowledge base, choose Directory or File when adding materials, and select an embedding model in the same dialog ([evidence ev-204]). The base indexes the folder; the Markdown stays the source of truth.

When should I add a reranking model?

When the correct chunk is retrieved but ranks too low ([evidence ev-206]). It is optional and reorders the shortlist rather than changing what was retrieved.

The XKnow Knowledge Base at /vaults/seo is a pre-compiled LLM wiki you own: structured Markdown notes with stable titles and predictable chunk boundaries, ready to point a knowledge base at. It works with or without embeddings, so you can start with keyword retrieval and add vectors later without rebuilding your content.

• What is an LLM wiki
• LLM wiki vs RAG
• Karpathy's LLM wiki
• XKnow Knowledge Base