The first time an AI editor suggests the exact function signature you needed — one that lives three files away in a utility module you half-forgot existed — it feels like magic. Then it happens again. And again.
It's not magic. It's not luck. And it's definitely not just autocomplete with a fancier model.
This post is the no-hand-waving answer to: how does it actually know that?
The Old World: Single-File Thinking
Classic IntelliSense worked on Abstract Syntax Trees. You type:
const user = new User();
user.
// ^ IDE parses User class → offers .id, .email, .save()
Useful. But it only answered: "given what I can see in this one file, what tokens are syntactically valid next?"
It couldn't answer: "the *parseUser() helper in utils/auth.ts has a null-email edge case — your test in __tests__/auth.spec.ts already covers it, so don't re-invent it."*
That's not a subtle difference. That's the difference between a dictionary and a colleague.
The first ML-based tools (TabNine early models, Kite) tried to go further — training on millions of GitHub repos to predict likely next tokens. But they had the same blind spot:
# TabNine circa 2020 — knows that catch blocks often contain:
try:
result = fetch_user(user_id)
except Exception as e:
# → suggests: logger.error(e) / raise / return None
# Based on: "what do most codebases do here?"
# NOT based on: "what does THIS codebase do here?"
The model was a well-read stranger. Knowledgeable about programming in general. Completely ignorant about your code in particular.
The Shift: What Goes Into the Context Window
The core change in modern AI editors is deceptively simple: the model sees more of your codebase at once. But "more" isn't just quantity — it's a qualitative shift in what reasoning becomes possible.
Here's how the context window has grown:
| Era | Window | What fits |
|---|---|---|
| Codex / early Copilot (2021) | 4k tokens | ~1 file |
| GPT-4 Turbo (2023) | 128k tokens | ~30–40 files |
| Claude 3.5 / Gemini 1.5 (2024) | 200k tokens | ~entire small project |
When you trigger a suggestion today, the context window assembled for the model typically looks like this:
[CONTEXT ASSEMBLED FOR: "write a sendWelcomeEmail function"]
1. Current file (full): api/users.ts
2. Recently visited: services/email.ts, db/models/user.ts
3. Imported by current file: utils/auth.ts, config/constants.ts
4. Type definitions: types/User.d.ts, types/Email.d.ts
5. Related tests: __tests__/users.spec.ts
6. Config: tsconfig.json, package.json
7. RAG-retrieved chunks: [see next section]
The model never sees your whole repo. It sees a curated slice — assembled by a retrieval pipeline that runs before the model ever processes your query.
RAG: The Invisible Brain Before the Brain
Retrieval-Augmented Generation is the engine behind every "how did it know that?" moment. Here's exactly what happens.
Step 1 — Indexing: Your Codebase Becomes Vectors
When you open a project, the editor quietly builds a semantic index. Every function, class, type, and docstring gets converted into a vector embedding — a list of numbers representing its meaning, not its syntax.
// Source code chunk:
function validateEmail(email: string): boolean {
return /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email);
}
// After embedding model processes it:
→ [0.23, -0.87, 0.44, 0.91, -0.12, 0.67, ...]
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
// 768 numbers encoding "email validation logic"
// Another chunk, completely different file:
it('should reject emails without @ symbol', () => {
expect(validateEmail('notanemail')).toBe(false);
});
→ [0.21, -0.83, 0.48, 0.88, -0.09, 0.71, ...]
// Similar vector = semantically related
Chunks with similar meaning cluster near each other in vector space — even if they share zero literal words.
The chunking isn't by line count. It uses Tree-sitter (a structural parser) to split at meaningful code boundaries: one function per chunk, one class per chunk. This ensures each chunk is a semantically complete unit, not an arbitrary slice of text.
Step 2 — Retrieval: Semantic Search at Query Time
When you write a comment or ask a question, your query gets embedded the same way:
Your query: "write a function that sends a welcome email to new users"
→ Query vector: [0.19, -0.79, 0.51, 0.85, -0.14, 0.63, ...]
Vector DB finds nearest neighbours:
✓ validateEmail() — distance: 0.12 (email logic)
✓ UserProfile interface — distance: 0.18 (user data shape)
✓ emailService.sendGrid() — distance: 0.21 (email sending)
✓ NEW_USER_TEMPLATE const — distance: 0.24 (email templates)
✓ user.spec.ts fixture — distance: 0.31 (test patterns)
None of those results contain the words "welcome" or "send." The retrieval found them because they're conceptually related — not textually matched.
This is what separates modern AI editors from grep. Grep finds literal matches. RAG finds conceptual neighbours.
Most editors also run a BM25 lexical search in parallel (a fast keyword search) and merge both result sets — so you get the best of semantic understanding and exact-name matching.
Step 3 — Reranking: Quality Filtering
Raw retrieval returns the 50-100 most similar chunks. A reranker model then re-scores them by reading the query and each chunk together:
Bi-encoder retrieval (fast, less accurate):
→ Scores chunks independently against query vector
Cross-encoder reranking (slow, highly accurate):
→ Reads query + chunk together
→ Models their actual interaction
→ Re-ranks the candidate set
Final top-10 chunks injected into context window ✓
The model sees only the reranked top results. It never knows a 50-candidate shortlist was assembled and filtered before it got involved.
Tree-sitter: The Structural Skeleton
Tree-sitter is an incremental, error-tolerant parser that maintains a live syntax tree of your code — updated character-by-character as you type, even when your code has syntax errors.
// You type this (incomplete, invalid syntax):
function processOrder(order: Order) {
const items = order.li
// ^ cursor here, code is broken
// Tree-sitter still produces:
FunctionDeclaration
name: "processOrder"
parameters:
Parameter { name: "order", type: "Order" }
body:
VariableDeclaration
name: "items"
initializer: MemberExpression (incomplete)
object: Identifier "order"
property: Identifier "li" [ERROR]
Even broken code gets a useful tree. This does three critical things for AI editors:
1. Precise chunk boundaries for RAG. Instead of splitting on line 50, line 100, line 150... the RAG pipeline splits at FunctionDeclaration ends. Every embedded chunk is a complete, coherent unit.
2. Scope and symbol resolution.
const userId = 'admin'; // scope: module
function getUser(userId: string) { // scope: function (shadows outer)
return db.find(userId); // which userId? Tree-sitter knows.
}
The AI gets precise scope information, not guesses.
3. Surgical edits. When applying a suggested change, the editor uses Tree-sitter node ranges to replace exactly lines 14-28, columns 2-47 — not fragile line-number approximations.
The Semantic Graph: Thinking in Relationships
Beyond RAG, the best editors maintain a live semantic graph — a map of every relationship between every symbol in your project.
Nodes: functions, classes, types, constants, interfaces
Edges: calls, imports, implements, uses, tests
Example subgraph:
checkout()
├─ calls → validateCart()
│ ├─ calls → getInventory()
│ └─ uses → CartItem (type)
├─ calls → processPayment()
│ └─ calls → stripe.charge()
├─ uses → Order (type)
│ └─ uses → LineItem (type)
└─ tested by → checkout.spec.ts
├─ uses fixture → mockCart
└─ uses fixture → mockPayment
This graph enables reasoning that no single file can provide.
Impact analysis before you change anything:
// You're about to change:
interface User {
id: number;
email: string;
// adding:
accountId: string; // ← new field
}
// Semantic graph instantly knows:
// → 23 files reference this interface
// → 4 will have type errors (missing accountId)
// → 2 tests use User fixtures that need updating
// → 1 DB migration needs to add the column
// → GraphQL schema needs a new field
Without the graph, the AI infers this from whatever's in the context window. With the graph, it knows it precisely.
Parametric vs. Retrieved Knowledge
The model has two completely different sources of "knowledge" — and confusing them explains most AI editor failures.
Parametric knowledge — baked into weights during training:
// Ask: "how does Promise.allSettled differ from Promise.all?"
// Model answers from parametric knowledge — reliable, no retrieval needed
Promise.all([p1, p2, p3])
// → Rejects immediately if ANY promise rejects
// → Returns values array only on full success
Promise.allSettled([p1, p2, p3])
// → Waits for ALL promises to finish
// → Returns [{status, value/reason}, ...] always
Retrieved knowledge — read fresh from your codebase every request:
// Ask: "write a new admin route using our middleware pattern"
// Model reads YOUR code to understand YOUR conventions:
// Retrieved chunk from api/users.ts:
router.get('/users', authenticate, authorize('admin'), async (req, res) => {
const users = await UserService.getAll();
res.json({ data: users, meta: buildMeta(req) });
});
// Retrieved chunk from api/orders.ts:
router.get('/orders', authenticate, authorize('admin'), async (req, res) => {
const orders = await OrderService.getAll(req.query);
res.json({ data: orders, meta: buildMeta(req) });
});
// Now generates for your new route — matching YOUR patterns:
router.get('/products', authenticate, authorize('admin'), async (req, res) => {
const products = await ProductService.getAll(req.query);
res.json({ data: products, meta: buildMeta(req) });
});
// ✓ Same middleware order
// ✓ Same response shape
// ✓ Same meta helper
// ✓ Same query-forwarding pattern
The dangerous gap: asking about a library where parametric knowledge is version 3.x but your project uses version 4.x. The model will confidently generate wrong code. Always check suggestions involving third-party libraries against your actual installed version.
Agentic Loops: When the AI Iterates Like You Do
Static suggestions work for small tasks. Complex tasks need feedback. Agentic mode closes the loop:
Task: "Add input validation to the POST /orders endpoint"
Loop iteration 1:
AI reads: api/orders.ts, types/Order.ts, existing validators
AI writes: validation middleware using zod schema
AI runs: npm test -- orders
AI reads: "FAIL: missing validation for lineItems array"
Loop iteration 2:
AI reads: types/LineItem.ts (retrieved by semantic search)
AI writes: adds lineItems schema to zod validator
AI runs: npm test -- orders
AI reads: "FAIL: test expects 422 status, got 400"
Loop iteration 3:
AI reads: api/users.ts (how validation errors are returned elsewhere)
AI writes: changes error status to match project convention (422)
AI runs: npm test -- orders
AI reads: "PASS: 8 tests passed"
AI done ✓
Each iteration, the model re-reads the codebase with fresh context informed by what it just learned. It's not lucky — it's using the same feedback loop you use, powered by everything we've described: retrieval to find related code, Tree-sitter to apply precise edits, the semantic graph to understand what else might be affected.
Where the Seams Show
None of this is perfect. Each failure mode maps to a specific architectural layer.
Retrieval miss: The AI ignores code you know exists.
Cause: RAG didn't retrieve the right chunks.
Fix: @mention the file explicitly in your prompt.
"In the style of @api/users.ts, write a..."
Stale index: The AI reasons about code you deleted last week.
Cause: Background indexing lagged after a large refactor.
Fix: Trigger a manual reindex. Start a fresh chat session.
Parametric version mismatch: Confident but wrong API usage.
// AI generates (using parametric knowledge of Mongoose v5):
User.findOne({ email }).exec().then(...)
// Your project uses Mongoose v8 (async/await preferred):
await User.findOne({ email }) // ← no .exec() needed
Cause: Parametric knowledge doesn't match your installed version.
Fix: Include your package.json or the actual function signature
in your prompt context.
Context window saturation: The AI "forgets" things in long sessions.
Cause: Window is full. Old context gets dropped.
Fix: Start a new conversation for each new task.
Don't rely on the AI retaining earlier context indefinitely.
The Gap Has Closed Farther Than Most Developers Realise
The developer who tried Copilot in 2022, found it useful for boilerplate, and formed the opinion "AI coding = fancy tab complete" is working with a two-year-old mental model.
The current generation — used well — doesn't feel like a faster way to type. It feels like a collaborator who has read your entire codebase, understands your architecture, follows your conventions, and can reason about your specific system rather than code in general.
That's not because models got smarter (though they did). It's because of the full stack:
Your query
↓
RAG pipeline — finds relevant code across your entire repo
↓
Tree-sitter — provides precise structural grounding
↓
Semantic graph — maps relationships between symbols
↓
Long context window — holds enough to reason coherently
↓
Language model — combines parametric + retrieved knowledge
↓
Agentic loop — iterates until tests pass
↓
Your editor
Every layer compounds. Remove any one of them and the experience degrades sharply.
The autocomplete era is genuinely over.
Originally published on ZyVOP

