Skip to content
|By Muhammad Wilayat Shah|4 views • 4 unique IPs

How to Build an AI Coding Assistant for Your Project

Build an AI coding assistant that knows your codebase: repo indexing, pgvector retrieval, project rules, token budgets and test-based verification.

ShareLinkedInX
How to Build an AI Coding Assistant for Your Project

General chat tools write plausible code, but they don't know your folder layout, your Laravel or Next.js version, your naming rules or the helper you already wrote last year. If you want suggestions that fit your repository, you can build ai coding assistant tooling of your own: index the codebase, retrieve the right files for each question, give the model your conventions, and check its output with your real tests. This guide walks through a small, working AI programming assistant in Python, and fixes the problems most teams hit on the first attempt.

Problem: Generic AI Tools Don't Know Your Codebase

A language model only sees what you put in the prompt. When a developer pastes one file and asks "add an approval step to purchase orders", the model has to guess how your services, models, events and tests are wired. It fills the gaps with patterns from public code, which is why the answer often looks right and fails on the first run. The fix is not a bigger model; it is better context and a verification loop.

Symptoms

  • Suggestions import packages or classes that don't exist in your project, such as a use App\Services\ApprovalService; line for a service you never wrote.
  • Code targets the wrong framework version, for example Next.js Pages Router code in an App Router project, or Laravel 8 syntax in a Laravel 11 app.
  • Requests fail with This model's maximum context length is ... tokens. However, your messages resulted in ... tokens (error code context_length_exceeded) after someone pastes half the repository.
  • Bursts of usage return HTTP 429 Too Many Requests.
  • Generated code ignores team conventions: wrong folder, different naming, missing tests.
  • Someone pastes an .env file or API key into the chat, and it goes to a third-party API.

Root Cause: Why a Code Generation AI Gives Wrong Answers

  • No retrieval. The model never sees the relevant files, so it invents them.
  • Too much context. Dumping whole folders blows the token limit and buries the useful lines.
  • No project rules. Framework versions, coding standards and "never touch these files" live in people's heads, not in the prompt.
  • No verification. Output is accepted on looks instead of being run through linters, type checks and tests.
  • No guardrails. Secrets and generated files are indexed and sent along with everything else.

The same issues are covered from the review side in why AI-generated code fails in production.

build ai coding assistant pipeline: index repository, embed chunks in pgvector, retrieve context, call LLM, verify with tests
The five stages of a repository-aware AI coding assistant.

Step-by-Step Solution: How to Build an AI Coding Assistant

The design below uses PostgreSQL with pgvector for storage, an embeddings API for search and a chat model for answers. Swap in your own provider; the structure stays the same.

Step 1: Decide what the assistant should do

Start narrow. "Answer questions about this repo and propose a patch as a diff" is a good first scope. Leave autonomous commits, database access and deployments for later, if ever. A narrow scope makes it easier to measure whether answers are correct.

Step 2: Choose which files to index

Index source, tests, migrations, configs and docs. Exclude dependencies, build output and anything secret:

Code
# .assistantignore
.env
.env.*
*.pem
*.key
node_modules/
vendor/
.next/
dist/
build/
storage/
public/build/
*.min.js
*.lock

Step 3: Chunk the code by structure

Splitting every 1,000 characters cuts functions in half. Split on blank lines between top-level blocks and keep the file path and line numbers with each chunk, so the model can cite where code came from. For a first version, this is enough; tools such as tree-sitter can give you function-level chunks later.

Code
# index_repo.py
import os, pathlib, fnmatch

ROOT = pathlib.Path(".")
IGNORE = [l.strip() for l in open(".assistantignore") if l.strip()]
EXTS = {".php", ".ts", ".tsx", ".js", ".py", ".sql", ".md", ".blade.php"}
MAX_CHARS = 2500

def ignored(path: str) -> bool:
    return any(fnmatch.fnmatch(path, p) or path.startswith(p.rstrip("/") + "/") for p in IGNORE)

def chunks(path):
    lines = path.read_text(errors="ignore").splitlines()
    buf, start = [], 1
    for i, line in enumerate(lines, 1):
        buf.append(line)
        size = sum(len(l) for l in buf)
        if (line.strip() == "" and size > 800) or size > MAX_CHARS:
            yield start, i, "\n".join(buf)
            buf, start = [], i + 1
    if buf:
        yield start, len(lines), "\n".join(buf)

def files():
    for p in ROOT.rglob("*"):
        rel = p.as_posix()
        if p.is_file() and p.suffix in EXTS and not ignored(rel):
            yield p

Step 4: Store embeddings in PostgreSQL

Create a table with a vector column whose size matches your embedding model. OpenAI's text-embedding-3-small returns 1,536 dimensions by default. If you have not set up pgvector before, see PostgreSQL pgvector RAG for installation and indexing errors.

Code
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE code_chunks (
  id          bigserial PRIMARY KEY,
  path        text NOT NULL,
  start_line  int  NOT NULL,
  end_line    int  NOT NULL,
  content     text NOT NULL,
  sha         text NOT NULL,
  embedding   vector(1536) NOT NULL
);

CREATE INDEX ON code_chunks USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON code_chunks (path);
Code
# embed.py
import hashlib, psycopg
from openai import OpenAI
from index_repo import files, chunks

client = OpenAI()          # reads OPENAI_API_KEY
EMBED_MODEL = "text-embedding-3-small"

def vec(v): return "[" + ",".join(map(str, v)) + "]"

with psycopg.connect("dbname=assistant") as db:
    for f in files():
        rel = f.as_posix()
        for start, end, text in chunks(f):
            sha = hashlib.sha256(text.encode()).hexdigest()
            if db.execute("SELECT 1 FROM code_chunks WHERE path=%s AND sha=%s", (rel, sha)).fetchone():
                continue                     # unchanged chunk, skip
            emb = client.embeddings.create(model=EMBED_MODEL, input=f"{rel}\n{text}").data[0].embedding
            db.execute(
                "INSERT INTO code_chunks (path,start_line,end_line,content,sha,embedding) VALUES (%s,%s,%s,%s,%s,%s::vector)",
                (rel, start, end, text, sha, vec(emb)))

Storing a hash per chunk means re-indexing after a commit only embeds the chunks that changed. Run it from a Git hook or a CI job on the main branch.

Step 5: Retrieve context and keep it under the token limit

Code
# ask.py
import os, sys, psycopg
from openai import OpenAI
from embed import vec, EMBED_MODEL

client = OpenAI()
CHAT_MODEL = os.environ["CHAT_MODEL"]       # the chat model you have access to
MAX_CONTEXT_CHARS = 24000                   # rough budget, tune for your model

def retrieve(question, k=12):
    q = client.embeddings.create(model=EMBED_MODEL, input=question).data[0].embedding
    with psycopg.connect("dbname=assistant") as db:
        return db.execute(
            "SELECT path,start_line,end_line,content FROM code_chunks "
            "ORDER BY embedding <=> %s::vector LIMIT %s", (vec(q), k)).fetchall()

def build_context(rows):
    out, used = [], 0
    for path, s, e, content in rows:
        block = f"### {path}:{s}-{e}\n{content}\n"
        if used + len(block) > MAX_CONTEXT_CHARS:
            break
        out.append(block); used += len(block)
    return "\n".join(out)

Step 6: Give the model your project rules

Keep a short, versioned rules file in the repository. This is what turns a generic model into an assistant that writes code your team would accept. For more on structuring these instructions, see how to write system prompts for LLM apps.

Code
# ASSISTANT_RULES.md
- Backend: Laravel 11, PHP 8.3. Business logic goes in app/Services, not controllers.
- Frontend: Next.js App Router with TypeScript strict mode.
- Database: PostgreSQL. Every schema change needs a migration.
- Every change needs a test (Pest for PHP, Vitest for TS).
- Only use classes, functions and packages that appear in the provided context.
  If something is missing, say so instead of inventing it.
- Answer with a unified diff and a short explanation.
Code
def ask(question):
    rules = open("ASSISTANT_RULES.md").read()
    context = build_context(retrieve(question))
    resp = client.chat.completions.create(
        model=CHAT_MODEL,
        temperature=0.2,
        messages=[
            {"role": "system", "content": "You are a coding assistant for this repository.\n" + rules},
            {"role": "user", "content": f"Repository context:\n{context}\n\nTask: {question}"},
        ],
    )
    return resp.choices[0].message.content

if __name__ == "__main__":
    print(ask(" ".join(sys.argv[1:])))

Step 7: Verify every suggestion automatically

Never trust a patch because it reads well. Apply it on a throwaway branch and run the same checks CI runs:

Code
python ask.py "add an approval step to purchase orders" > patch.diff
git switch -c ai/try-approval
git apply --check patch.diff && git apply patch.diff
php artisan test          # or: ./vendor/bin/pest
npx tsc --noEmit
npx eslint .
git diff --stat

If git apply --check fails with error: patch failed or corrupt patch at line ..., feed the error and the current file back to the model once, then stop and let a developer take over. Endless retry loops waste tokens and rarely converge.

Step 8: Handle rate limits and failures

Wrap API calls with retries and exponential backoff for 429 and 5xx responses, and cap how many requests a single user can make per minute. The details are in LLM API rate limit errors.

Verification: Is Your AI Programming Assistant Actually Helping?

  • Run SELECT count(*), count(DISTINCT path) FROM code_chunks; and confirm the numbers match the files you expect, with nothing from vendor/ or .env.
  • Write 20 to 30 real questions from past tickets with known answers. Check that the right files appear in the top results of retrieve() before judging the model at all.
  • Track how many generated patches pass git apply --check and your test suite without edits.
  • Search the logs for invented names: any class or import in a patch that doesn't exist in the repository is a retrieval or prompt problem.
  • Run EXPLAIN on the retrieval query to confirm the HNSW index is used once the table grows.

Prevention: Keep Your Code Generation AI Safe and Accurate

  • Keep secrets out of the index. Maintain the ignore file, and scan chunks for patterns like AKIA, -----BEGIN PRIVATE KEY----- and DB_PASSWORD= before embedding.
  • Read-only by default. The assistant proposes diffs; humans review and merge. Don't give it production credentials or shell access.
  • Treat repo text as untrusted. Comments, README files and issue text can contain instructions aimed at the model. See prompt injection protection.
  • Re-index on merge so the assistant never answers from stale code.
  • Version the rules file and update it when you upgrade a framework.
  • Log questions and outcomes (without secrets) to find weak spots in retrieval.

FAQs

Do I need to fine-tune a model to build an AI coding assistant?

Usually not. Retrieval plus a clear rules file solves most "it doesn't know our code" problems, and it updates as soon as you re-index. Fine-tuning is harder to keep current with a changing codebase.

Can I build an AI programming assistant that runs fully offline?

Yes. You can run an open-weight chat model and embedding model on your own server and keep the same pgvector pipeline. Expect to need a GPU for acceptable speed on larger models, and test answer quality on your own questions.

How is this different from GitHub Copilot or Cursor?

Those tools already index your repository and are a good default. A custom assistant makes sense when you need strict data control, integration with internal systems such as an ERP's documentation or ticket history, or answers shaped by rules specific to your team.

How much code context should I send per request?

Send the smallest set of chunks that answers the question, usually a few files' worth. More context costs more, is slower and often lowers quality because the relevant lines get buried.

How do I stop code generation AI from inventing functions?

Tell it to use only names present in the context and to say when something is missing, improve retrieval so the real code is included, and reject any patch that fails type checks or tests.

ShareLinkedInX
Back to blog

Keep reading

Newsletter

Occasional notes on Laravel, React, and shipping production systems — no spam.

Comments

Join the discussion. Replies load on demand, up to 5 levels.

0 threads

Loading editor…
Reviewed before publishing

No comments yet

Be the first to start the conversation.