General chat tools write plausible code, but they don't know your folder layout, your Laravel or Next.js version, your naming rules or the helper you already wrote last year. If you want suggestions that fit your repository, you can build ai coding assistant tooling of your own: index the codebase, retrieve the right files for each question, give the model your conventions, and check its output with your real tests. This guide walks through a small, working AI programming assistant in Python, and fixes the problems most teams hit on the first attempt.
Problem: Generic AI Tools Don't Know Your Codebase
A language model only sees what you put in the prompt. When a developer pastes one file and asks "add an approval step to purchase orders", the model has to guess how your services, models, events and tests are wired. It fills the gaps with patterns from public code, which is why the answer often looks right and fails on the first run. The fix is not a bigger model; it is better context and a verification loop.
Symptoms
- Suggestions import packages or classes that don't exist in your project, such as a
use App\Services\ApprovalService;line for a service you never wrote. - Code targets the wrong framework version, for example Next.js Pages Router code in an App Router project, or Laravel 8 syntax in a Laravel 11 app.
- Requests fail with
This model's maximum context length is ... tokens. However, your messages resulted in ... tokens(error codecontext_length_exceeded) after someone pastes half the repository. - Bursts of usage return HTTP
429 Too Many Requests. - Generated code ignores team conventions: wrong folder, different naming, missing tests.
- Someone pastes an
.envfile or API key into the chat, and it goes to a third-party API.
Root Cause: Why a Code Generation AI Gives Wrong Answers
- No retrieval. The model never sees the relevant files, so it invents them.
- Too much context. Dumping whole folders blows the token limit and buries the useful lines.
- No project rules. Framework versions, coding standards and "never touch these files" live in people's heads, not in the prompt.
- No verification. Output is accepted on looks instead of being run through linters, type checks and tests.
- No guardrails. Secrets and generated files are indexed and sent along with everything else.
The same issues are covered from the review side in why AI-generated code fails in production.

Step-by-Step Solution: How to Build an AI Coding Assistant
The design below uses PostgreSQL with pgvector for storage, an embeddings API for search and a chat model for answers. Swap in your own provider; the structure stays the same.
Step 1: Decide what the assistant should do
Start narrow. "Answer questions about this repo and propose a patch as a diff" is a good first scope. Leave autonomous commits, database access and deployments for later, if ever. A narrow scope makes it easier to measure whether answers are correct.
Step 2: Choose which files to index
Index source, tests, migrations, configs and docs. Exclude dependencies, build output and anything secret:
# .assistantignore
.env
.env.*
*.pem
*.key
node_modules/
vendor/
.next/
dist/
build/
storage/
public/build/
*.min.js
*.lockStep 3: Chunk the code by structure
Splitting every 1,000 characters cuts functions in half. Split on blank lines between top-level blocks and keep the file path and line numbers with each chunk, so the model can cite where code came from. For a first version, this is enough; tools such as tree-sitter can give you function-level chunks later.
# index_repo.py
import os, pathlib, fnmatch
ROOT = pathlib.Path(".")
IGNORE = [l.strip() for l in open(".assistantignore") if l.strip()]
EXTS = {".php", ".ts", ".tsx", ".js", ".py", ".sql", ".md", ".blade.php"}
MAX_CHARS = 2500
def ignored(path: str) -> bool:
return any(fnmatch.fnmatch(path, p) or path.startswith(p.rstrip("/") + "/") for p in IGNORE)
def chunks(path):
lines = path.read_text(errors="ignore").splitlines()
buf, start = [], 1
for i, line in enumerate(lines, 1):
buf.append(line)
size = sum(len(l) for l in buf)
if (line.strip() == "" and size > 800) or size > MAX_CHARS:
yield start, i, "\n".join(buf)
buf, start = [], i + 1
if buf:
yield start, len(lines), "\n".join(buf)
def files():
for p in ROOT.rglob("*"):
rel = p.as_posix()
if p.is_file() and p.suffix in EXTS and not ignored(rel):
yield pStep 4: Store embeddings in PostgreSQL
Create a table with a vector column whose size matches your embedding model. OpenAI's text-embedding-3-small returns 1,536 dimensions by default. If you have not set up pgvector before, see PostgreSQL pgvector RAG for installation and indexing errors.
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE code_chunks (
id bigserial PRIMARY KEY,
path text NOT NULL,
start_line int NOT NULL,
end_line int NOT NULL,
content text NOT NULL,
sha text NOT NULL,
embedding vector(1536) NOT NULL
);
CREATE INDEX ON code_chunks USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON code_chunks (path);# embed.py
import hashlib, psycopg
from openai import OpenAI
from index_repo import files, chunks
client = OpenAI() # reads OPENAI_API_KEY
EMBED_MODEL = "text-embedding-3-small"
def vec(v): return "[" + ",".join(map(str, v)) + "]"
with psycopg.connect("dbname=assistant") as db:
for f in files():
rel = f.as_posix()
for start, end, text in chunks(f):
sha = hashlib.sha256(text.encode()).hexdigest()
if db.execute("SELECT 1 FROM code_chunks WHERE path=%s AND sha=%s", (rel, sha)).fetchone():
continue # unchanged chunk, skip
emb = client.embeddings.create(model=EMBED_MODEL, input=f"{rel}\n{text}").data[0].embedding
db.execute(
"INSERT INTO code_chunks (path,start_line,end_line,content,sha,embedding) VALUES (%s,%s,%s,%s,%s,%s::vector)",
(rel, start, end, text, sha, vec(emb)))Storing a hash per chunk means re-indexing after a commit only embeds the chunks that changed. Run it from a Git hook or a CI job on the main branch.
Step 5: Retrieve context and keep it under the token limit
# ask.py
import os, sys, psycopg
from openai import OpenAI
from embed import vec, EMBED_MODEL
client = OpenAI()
CHAT_MODEL = os.environ["CHAT_MODEL"] # the chat model you have access to
MAX_CONTEXT_CHARS = 24000 # rough budget, tune for your model
def retrieve(question, k=12):
q = client.embeddings.create(model=EMBED_MODEL, input=question).data[0].embedding
with psycopg.connect("dbname=assistant") as db:
return db.execute(
"SELECT path,start_line,end_line,content FROM code_chunks "
"ORDER BY embedding <=> %s::vector LIMIT %s", (vec(q), k)).fetchall()
def build_context(rows):
out, used = [], 0
for path, s, e, content in rows:
block = f"### {path}:{s}-{e}\n{content}\n"
if used + len(block) > MAX_CONTEXT_CHARS:
break
out.append(block); used += len(block)
return "\n".join(out)Step 6: Give the model your project rules
Keep a short, versioned rules file in the repository. This is what turns a generic model into an assistant that writes code your team would accept. For more on structuring these instructions, see how to write system prompts for LLM apps.
# ASSISTANT_RULES.md
- Backend: Laravel 11, PHP 8.3. Business logic goes in app/Services, not controllers.
- Frontend: Next.js App Router with TypeScript strict mode.
- Database: PostgreSQL. Every schema change needs a migration.
- Every change needs a test (Pest for PHP, Vitest for TS).
- Only use classes, functions and packages that appear in the provided context.
If something is missing, say so instead of inventing it.
- Answer with a unified diff and a short explanation.def ask(question):
rules = open("ASSISTANT_RULES.md").read()
context = build_context(retrieve(question))
resp = client.chat.completions.create(
model=CHAT_MODEL,
temperature=0.2,
messages=[
{"role": "system", "content": "You are a coding assistant for this repository.\n" + rules},
{"role": "user", "content": f"Repository context:\n{context}\n\nTask: {question}"},
],
)
return resp.choices[0].message.content
if __name__ == "__main__":
print(ask(" ".join(sys.argv[1:])))Step 7: Verify every suggestion automatically
Never trust a patch because it reads well. Apply it on a throwaway branch and run the same checks CI runs:
python ask.py "add an approval step to purchase orders" > patch.diff
git switch -c ai/try-approval
git apply --check patch.diff && git apply patch.diff
php artisan test # or: ./vendor/bin/pest
npx tsc --noEmit
npx eslint .
git diff --statIf git apply --check fails with error: patch failed or corrupt patch at line ..., feed the error and the current file back to the model once, then stop and let a developer take over. Endless retry loops waste tokens and rarely converge.
Step 8: Handle rate limits and failures
Wrap API calls with retries and exponential backoff for 429 and 5xx responses, and cap how many requests a single user can make per minute. The details are in LLM API rate limit errors.
Verification: Is Your AI Programming Assistant Actually Helping?
- Run
SELECT count(*), count(DISTINCT path) FROM code_chunks;and confirm the numbers match the files you expect, with nothing fromvendor/or.env. - Write 20 to 30 real questions from past tickets with known answers. Check that the right files appear in the top results of
retrieve()before judging the model at all. - Track how many generated patches pass
git apply --checkand your test suite without edits. - Search the logs for invented names: any class or import in a patch that doesn't exist in the repository is a retrieval or prompt problem.
- Run
EXPLAINon the retrieval query to confirm the HNSW index is used once the table grows.
Prevention: Keep Your Code Generation AI Safe and Accurate
- Keep secrets out of the index. Maintain the ignore file, and scan chunks for patterns like
AKIA,-----BEGIN PRIVATE KEY-----andDB_PASSWORD=before embedding. - Read-only by default. The assistant proposes diffs; humans review and merge. Don't give it production credentials or shell access.
- Treat repo text as untrusted. Comments, README files and issue text can contain instructions aimed at the model. See prompt injection protection.
- Re-index on merge so the assistant never answers from stale code.
- Version the rules file and update it when you upgrade a framework.
- Log questions and outcomes (without secrets) to find weak spots in retrieval.
FAQs
Do I need to fine-tune a model to build an AI coding assistant?
Usually not. Retrieval plus a clear rules file solves most "it doesn't know our code" problems, and it updates as soon as you re-index. Fine-tuning is harder to keep current with a changing codebase.
Can I build an AI programming assistant that runs fully offline?
Yes. You can run an open-weight chat model and embedding model on your own server and keep the same pgvector pipeline. Expect to need a GPU for acceptable speed on larger models, and test answer quality on your own questions.
How is this different from GitHub Copilot or Cursor?
Those tools already index your repository and are a good default. A custom assistant makes sense when you need strict data control, integration with internal systems such as an ERP's documentation or ticket history, or answers shaped by rules specific to your team.
How much code context should I send per request?
Send the smallest set of chunks that answers the question, usually a few files' worth. More context costs more, is slower and often lowers quality because the relevant lines get buried.
How do I stop code generation AI from inventing functions?
Tell it to use only names present in the context and to say when something is missing, improve retrieval so the real code is included, and reject any patch that fails type checks or tests.




Comments
Join the discussion. Replies load on demand, up to 5 levels.
0 threads
No comments yet
Be the first to start the conversation.