Mental Models

These are the eight load-bearing ideas of agentic engineering — not the most famous ideas, the ones everything else rests on. Each comes with why it matters, a concrete case, and the misconception that most often wrecks it.

The model is a stateless function

Every call to a language model is a pure function: tokens in, tokens out, nothing retained. The model that answers your second message has no memory of your first — the application re-sends the whole conversation every time, and the model re-reads it from scratch. Anything that looks like memory, personality, or learning within a session is reconstructed on every single call from whatever text the application chooses to send.

Why it’s load-bearing: once you internalize this, “what does the model know right now?” stops being mysterious. It knows exactly what’s in the current request — no more, no less. Every memory feature, every “the agent forgot what I told it” bug, every context-management technique in this hub is downstream of this one fact.

Example: a chat app’s “conversation” is an illusion the app maintains by replaying the full transcript on each turn. When the transcript gets too long to replay, the app must summarize or trim it — and whatever gets trimmed is, for the model, gone as if it never happened.

Misconception: “the model remembers our conversation.” It doesn’t. The application remembers, and pays — in tokens, on every call — to remind the model.

An agent is a loop with an exit condition

Take the stateless function, put it in a while loop, and let its output trigger real actions: the model proposes an action, ordinary software executes it, the result is appended to the transcript, and the model is called again — until it declares the task done, hits a budget, or fails. That loop, plus the harness code around it, is the entire architecture of every agent you’ve heard of.

Why it’s load-bearing: it demystifies the field. Debugging an agent means reading the loop’s iterations. Improving an agent means improving what the loop feeds the model or what actions it offers. There is no hidden intelligence layer to appeal to — just the loop.

Example: a coding agent fixing a failing test runs this exact cycle: read the test file → run the tests → read the failure output → edit the source → run the tests again → exit when green. Each arrow is one loop iteration.

Misconception: “agents are a new kind of AI.” They’re the same model you’d call once, called repeatedly, with its own outputs and their consequences folded back in. The magic is in the folding, and the folding is code you write.

Errors compound over the horizon

If each step succeeds 99% of the time, a 100-step task succeeds 37% of the time. Reliability that feels excellent per step is catastrophic per task, because errors multiply and — worse — early mistakes become part of the context every later step builds on.

Why it’s load-bearing: it explains most of the field’s architecture: why practitioners decompose tasks, verify at boundaries, keep autonomous stretches short, and obsess over per-step quality. Any design that ignores compounding works in demos and fails in production, since demos are short and production is long.

Example: an agent misreads a database schema at step 3 of a migration task. Steps 4 through 30 are all internally consistent, competently executed — and built on the wrong schema. The failure surfaces at step 30, but it happened at step 3.

Misconception: “the next model generation will fix reliability.” Better models raise the per-step rate, but the exponent — task length — rises just as fast, because we hand better models longer tasks. Verification and recoverability are permanent parts of the job, not stopgaps.

Context is a budget, not a warehouse

The model’s context window is finite, and — more importantly — its attention degrades as the window fills. Facts buried in the middle of a 100,000-token transcript get missed; irrelevant material actively distracts. Treat tokens as money: every one you spend on noise is one not spent on signal, and hoarding doesn’t work because the currency inflates as the pile grows.

Why it’s load-bearing: it converts “put everything in, just in case” from a safe default into a recognized failure mode, and motivates the whole toolkit of retrieval, summarization, and splitting work across fresh windows.

Example: an agent given an entire repository up front performs worse than the same agent given a file-search tool — the second agent reads only what the task needs, and its context stays clean enough to reason over.

Misconception: “million-token windows make context management obsolete.” Bigger windows raise the ceiling but not the quality of attention. Long-context degradation is measurable in every model generation so far; the budget got bigger, but it’s still a budget.

Tools are the model’s user interface

The model never sees your tool’s implementation — only its name, its description, its parameters, and what it returns. That thin surface is the model’s entire understanding of what it can do, which makes tool definitions interface design in the full sense: ambiguity confuses, good error messages teach, and the reader whose intuitions matter is the model.

Why it’s load-bearing: when an agent misuses a tool, the instinct is to blame the model. The productive move is almost always to fix the interface — and once you see tools as UI, you know how: clearer naming, tighter descriptions, errors that say what to try instead.

Example: a search tool that returns [] on no results leaves the agent flailing. The same tool returning “No matches for ‘authetication’ — did you mean ‘authentication’? Try a broader query” turns the failure into a next step. Same implementation, different interface, different agent.

Misconception: “more tools make a more capable agent.” Every tool adds surface area for confusion, and overlapping tools force choices the model can only guess at. A few powerful, clearly-bounded tools beat a drawer of specific ones.

Nothing works until an eval says so

Agent outputs vary run to run, so three good demos prove almost nothing — you happened to sample three successes. The only trustworthy signal is a set of real test tasks, run repeatedly, graded consistently. Practitioners call these evals, and mature teams treat them the way software teams treat test suites: the thing you run before believing any change helped.

Why it’s load-bearing: without evals, agent development is guessing. Prompt changes, tool tweaks, and model upgrades all feel better or worse; the feeling is noise. The eval is what turns iteration into hill-climbing instead of wandering.

Example: a team “improves” a prompt after it fixes a failure they saw in a demo. The eval suite shows the overall pass rate dropped from 74% to 66% — the fix helped one task and quietly broke a category of others. Without the eval, that regression ships.

Misconception: “evals are like unit tests — write them, get green, done.” Evals are statistical: results need multiple runs, scores have error bars, and the suite must keep growing with real-world failures or the agent overfits to it.

Autonomy is a dial, not a binary

Between “script that calls a model once” and “agent that plans its own work” lies a spectrum. A workflow fixes the steps in code and uses the model inside each step; an agent lets the model choose the steps. More autonomy buys flexibility on open-ended tasks and costs predictability, debuggability, and money. The engineering skill is picking the lowest setting that does the job.

Why it’s load-bearing: the most common architecture mistake in the field is reaching for full autonomy because it’s impressive, when a workflow would be cheaper, faster, and more reliable. Knowing the dial exists is what prevents it.

Example: extracting fields from invoices is a workflow — same steps every time, model doing extraction inside each one. Debugging a flaky integration test is an agent problem — the steps genuinely can’t be known in advance.

Misconception: “real agentic systems are fully autonomous.” Production systems are almost all hybrids: workflow backbones with agentic sections where the task is genuinely open-ended, and human approval gates where actions are irreversible.

Build for the model six months from now

Models improve on a fast, steady clock, and every improvement redraws the line between “the scaffolding must handle this” and “the model just does this.” Scaffolding that compensates for a model weakness — elaborate prompt chains, hand-coded plan validators — becomes dead weight at the next upgrade. Scaffolding that supplies things no model can conjure — context, tools, verification, measurement — compounds instead.

Why it’s load-bearing: it’s the field’s investment thesis. Teams that pour effort into workaround scaffolding rewrite it every model generation; teams that pour effort into plumbing get a free capability jump every generation, delivered through the same pipes.

Example: a team builds a multi-stage pipeline that forces the model to plan, critique its plan, and re-plan — needed because the model planned poorly. A model upgrade later, the pipeline underperforms just letting the new model work, and gets deleted. The eval suite that proved it should be deleted survives, and is now the team’s most valuable asset.

Misconception: “so just wait for better models.” The opposite: tools, evals, and context plumbing are exactly what let you exploit each new model the week it ships. The advice isn’t to build less — it’s to build the durable half.