Tested · 2026-10-15

Local AI knowledge base: Q&A over your own docs

Put documents in a local folder and let Claude Code index them so answers come only from your files, with citations. In our run it wrote a Python-stdlib retrieval script `ask.py`. Questions that are not in the docs return “not found in the documents” instead of a guess.

✓3 policy docs: retrieval script written in 67 seconds
✓4 questions answered correctly with source files; a “fitness stipend” question that was not in the docs returned “not found in the documents”
✓Rechecked on 2026-10-15: the same 4 answers. The script itself calls no API

Set up once: point the global config at AICodeWith

Do this once; every use case reuses it. Other tools: integration guides. Where to get a key: Get an API key.

1. Install Claude Code
Follow the official Claude Code install docs. You are done when claude --version prints a version number.
2. Create a key and edit the global config
Sign in, open the dashboard **Key Management** page, and create a key. If you are not sure where that is, see Get an API key.
Open the global config ~/.claude/settings.json and set the Base URL and key under env:
~/.claude/settings.json
```
{
  "env": {
    "ANTHROPIC_BASE_URL": "https://api.aicodewith.ai",
    "ANTHROPIC_AUTH_TOKEN": "sk-YOUR_KEY"
  }
}
```
Do not add /v1 at the end of the Base URL. Reopen the terminal, run claude, and a normal reply means the connection works.

Tested steps

1. Prepare docs and ask it to build a Q&A tool

Create a folder, put Markdown files in docs/, run claude in the folder, and type:

text

The docs/ folder has a few of our company policy documents. Build a local Q&A tool: write a Python script ask.py that needs no third-party API key, reads Markdown under docs/, chunks by paragraph, retrieves the most relevant passages with keyword/BM25, and prints the answer plus source (filename + original sentence). Then use it to answer these questions and write the results to qa-results.md: (replace with your questions). If it is not in the documents, say "not found in the documents". Do not invent answers.

Keep the last sentence about saying “not found in the documents” and not inventing answers. That is what stops the script from guessing.

2. Ask later without Claude Code
After the script exists, you do not need Claude Code. Ask in the terminal:
bash
```
python3 ask.py "How many vacation days after 12 years?"
python3 ask.py -k 3 "Who do I ask for a test phone?"   # top 3 candidate passages
```
Scripts Claude Code writes can differ. The first time, try a few questions whose answers you already know.

Troubleshooting

It says not found even though the docs contain it: Ask it to lower the match threshold or add synonyms, then rerun that question.
The docs are Word or PDF: We only tested Markdown. You can ask Claude Code to convert them first (that step is untested).
401 or request failed: Check that the key is complete and the Base URL is https://api.aicodewith.ai (no /v1). Reopen the terminal after you change it.

FAQ

Are the documents uploaded?

The retrieval script runs locally and calls no API, so questions never leave your computer. Writing the script does send the request and needed excerpts to the Claude Code endpoint you configured.

Will it invent answers?

In our run the script only returned original sentences and sources, or “not found in the documents.”

Do I need a vector database?

No. The tested script uses the Python standard library and keyword scoring. Consider something else when you have many documents and need semantic search.

Want a one-click version? Leave your email

Don't want to set this up yourself? We are evaluating a one-click tool for “local AI knowledge base” and will email you when it is ready.

Set up once: point the global config at AICodeWith

1. Install Claude Code

2. Create a key and edit the global config