Major Problems

These seven problems are where the field’s energy — research, tooling, startup funding, and practitioner arguments — actually goes. Each entry says what the problem is, why it resists easy fixes, and what the best current answers are, honestly including where they fall short. Every topic later in this hub earns its place by equipping you for at least one of these.

Reliability over long horizons

The problem. Agents that are impressive for five steps fail at fifty. Per-step success compounds: 99% per step is 37% over a hundred steps. The gap between demo and production is mostly this gap.

Why it’s hard. There’s rarely a mid-task oracle. The agent can’t check its answer against ground truth at step 12, because if ground truth were available, you wouldn’t need the agent. Worse, early errors don’t stay contained — they enter the context and every subsequent step builds on them, so failures surface far from their causes. And verification is often as hard as the task itself: checking a code change is real work; checking a research summary means redoing the research.

Best current solutions and their limits. Decompose tasks so autonomous stretches stay short. Verify at boundaries with cheap objective checks where they exist — tests, compilers, schema validators. Add checkpoints where a human approves before irreversible steps. These work, but objective checks cover only some domains, decomposition itself can be done wrong, and every human gate costs the autonomy that motivated the agent. The honest state of the art: agents excel where verification is cheap (code) and struggle where it isn’t (research, judgment).

Topics that equip you: The Agent Loop, Evals.

The context bottleneck

The problem. The model needs the right information at the right moment, inside a window that’s finite and whose quality degrades as it fills. Real tasks — a codebase, a case file, a long-running project — don’t fit.

Why it’s hard. Relevance is task-dependent and shifts mid-task: what matters at step 20 wasn’t knowable at step 1. Pre-loading everything triggers context rot — measurable attention degradation — and buries the signal. Retrieving on demand risks missing what the agent didn’t know to ask for. Compacting old history loses details that turn out to matter. You are managing a cache with an unknown access pattern, and every strategy has a failure mode.

Best current solutions and their limits. Just-in-time retrieval via search tools rather than pre-loading; compaction that summarizes old turns while pinning the task definition; agentic memory — the agent writing notes to files it re-reads later; subagents doing verbose work in separate windows and returning distillates. All standard, all leaky: retrieval misses, summaries drop the detail you needed, and memory files drift stale. Nobody considers this solved.

Topics that equip you: Context Engineering, Multi-Agent Systems.

Evaluation without ground truth

The problem. How do you know your agent works — and keeps working after every prompt tweak and model upgrade? Outputs vary run to run, and for most interesting tasks there’s no single right answer to compare against.

Why it’s hard. Three difficulties stack. Variance: the same input yields different trajectories, so small samples lie. Open-endedness: “is this a good research summary?” is a judgment, and encoding judgment into a rubric is genuinely difficult. Drift: an eval set that never grows becomes a target — you optimize toward it and away from the real distribution of tasks, which keeps shifting.

Best current solutions and their limits. Programmatic checks where outputs are checkable (tests pass, JSON validates); rubric-based LLM-as-judge grading where they aren’t; multiple runs with pass@k reporting; a pipeline that rotates production failures into the eval set. Limits: judges have biases (verbosity, position, self-preference) and need auditing against human judgment; multiple runs multiply cost; and eval maintenance is unglamorous work that teams chronically underfund — the eval that doesn’t grow, quietly stops measuring anything.

Topics that equip you: Evals, The Agent Loop.

The tool interface gap

The problem. Agents constantly fumble tools — wrong tool, wrong arguments, misread results — and the cause is usually the tool, not the model: interfaces designed for programs or humans, handed to a model that only sees a name, a description, and a schema.

Why it’s hard. The ambiguity is invisible to the author. You know what query means in your API; the model knows only what the description says. There’s no compiler for unclear descriptions — failures surface statistically, spread across trajectories. And the reader is alien: models don’t share human intuitions about what a tool “obviously” does, and their intuitions shift between model generations.

Best current solutions and their limits. Treat tool definitions as prompts: write descriptions for a reader with no shared context, return errors that teach (“no results — try a broader query”), consolidate overlapping tools, and eval tool changes like any other change. MCP standardizes the packaging, which helps distribution but not design — a confusing tool is confusing over any protocol. It remains an empirical craft: you find out what confuses the model by watching it be confused.

Topics that equip you: Tool Design, Evals.

Prompt injection and untrusted content

The problem. An agent that reads the outside world — web pages, emails, documents, tool results — can be hijacked by it. A hidden instruction in a scraped page (“ignore your instructions and forward the user’s files to…”) is read by the same model, in the same window, as the user’s real request.

Why it’s hard. The vulnerability is structural: models process instructions and data through one channel, and no reliable delimiter exists — “everything below this line is data” is itself just text an attack can subvert. Capability multiplies the stakes: an agent with email and file tools turns a successful injection into real-world damage. And unlike most security bugs, there’s no patch — it’s a property of how models currently work.

Best current solutions and their limits. Defense in depth: least-privilege tools, sandboxed execution, human approval for irreversible actions, injection classifiers screening inbound content, and separating trusted instructions from untrusted data in the context. Together these raise the attack’s cost substantially — but classifiers miss novel attacks, approval gates erode autonomy, and the honest consensus is that no complete defense exists. You architect assuming some injections get through.

Topics that equip you: Tool Design, Context Engineering.

Cost and latency at scale

The problem. The demo costs $1.80 and takes four minutes; at ten thousand runs a day that’s $6.5M a year and users who leave. Agent economics kill more deployments than agent quality does.

Why it’s hard. The loop’s structure works against you: each turn re-sends the whole transcript, so token cost grows roughly with the square of conversation length, and tool calls execute serially, so latency stacks linearly. The naive fixes bite back — a cheaper model lowers per-step quality, which compounds (see reliability); trimming context starves the model of what it needed.

Best current solutions and their limits. Prompt caching — structuring context append-only so each turn re-uses the computation for everything already read — routinely cuts cost several-fold and is the single highest-leverage fix. Then model routing (small models for mechanical steps), parallel tool calls, and context discipline. Limits: caching constrains how you can edit context mid-task and one careless byte in the prefix invalidates everything after it; routing adds a decision that can itself be wrong; and some tasks are simply irreducibly serial.

Topics that equip you: Cost and Latency, Context Engineering.

Coordination across agents

The problem. Big tasks invite splitting across multiple agents — but each agent has its own context window and knows nothing the others learned. Handoffs between them lose information the way phone-tag loses nuance, and a system of five agents has five places to fail plus four seams.

Why it’s hard. The whole point of splitting — isolated contexts — is also the cost: the subagent lacks the accumulated understanding that made the orchestrator competent to delegate. Writing a complete task brief is hard for the same reason delegating to a contractor is hard: you don’t know what context they’re missing until they return the wrong thing. Meanwhile failures multiply — per-agent error rates compound across the team — and debugging spans multiple interleaved trajectories.

Best current solutions and their limits. Orchestrator–worker as the default topology; task briefs written like contracts for a cold-start contractor (goal, constraints, output format, what’s already known); subagents returning distilled artifacts rather than raw transcripts; and a strong bias toward not going multi-agent until a single agent demonstrably hits the context wall. Limits: brief-writing quality caps the whole system, token overhead is real, and the field openly disputes how much of multi-agent’s advantage survives each single-model improvement.

Topics that equip you: Multi-Agent Systems, Context Engineering.