This website uses cookies

Read our Privacy policy and Terms of use for more information.

In 2025, Andrej Karpathy named what engineers working with AI agents had been circling around for two years: the problem is not the prompt. The problem is the entire context the model receives, and whether that context is designed to make the task solvable. Tobi Lutke compressed it further: context engineering is "the art of providing all the context for the task to be plausibly solvable by the LLM."

That re-frame is correct and important. It is also the opening of a more precise vocabulary, not the complete one.

Naming the problem as "context engineering" tells you where to look. It does not yet name what breaks inside that context, why it breaks, or what architectural pattern prevents each failure mode from recurring. That vocabulary lives one level below the frame, in the components the frame correctly identifies as the unit of design.

This issue maps six concrete production failure modes to their names in the precision vocabulary. If you are doing context engineering without this vocabulary, you have the frame but not the diagnostic.

Method Deep-Dive: six problems, six names

When engineers say they are "doing context engineering," they are managing six distinct problems simultaneously. Each one has a precise name.

Problem 1: You do not know what the model is actually receiving.

You write a system prompt. You add tool definitions. You retrieve documents. You have a conversation history. The model receives all of it at once, as a single undifferentiated input, and you have no direct view of the combined result.

The Control Surface is the term for this totality: every token the model receives at the moment of generation. System prompt, conversation history, tool schemas, retrieved context, injected examples, all of it. Context engineering is, at its core, the discipline of designing the Control Surface deliberately rather than letting it accumulate by accident. The failure mode when you do not: context pollution. Retrieved documents contradict the system prompt. Tool schemas consume budget you needed for examples. The model resolves contradictions by guessing, and produces inconsistent outputs you cannot trace to a single cause.

Problem 2: You cannot fit everything into the window.

Context windows have grown. The failure mode has shifted, not disappeared: from "cannot fit" to "attends poorly to content near the edges" and "costs scale with token count across agentic loops."

Context Window Budgeting is the practice of treating the context window as a resource with a hard budget: assigning token allocations to each Control Surface component before starting a task, and enforcing those allocations rather than letting them drift. Without a budget, a multi-step agent loop that works on step 1 accumulates context until the window overflows on step 7, mid-task, with no fallback behaviour defined.

Problem 3: Your constraints conflict with each other.

You write a prompt with twelve constraints. Eleven are fine. One is mutually exclusive with another. You do not find out until the model starts producing output that satisfies neither.

The Constraint Satisfaction Problem is the framing tool: before deploying a constraint set, check every constraint pair for mutual exclusivity. The Constraint Architecture is the practice of organizing those constraints into a coherent system (what must be true, what must never happen, what the model should prefer when mandatory constraints are satisfied). Context engineering without Constraint Architecture produces context that is rich in information and under-specified in what to do with it.

Problem 4: You have not defined what the model should do when it cannot answer.

A model with no fallback behaviour resolves uncertainty with confident output. It does not surface ambiguity. It produces the most plausible-sounding answer it can construct, with no signal that it is uncertain.

The Fallback Cascade is a tiered policy: at each level of scope ambiguity, the agent has an explicit instruction (refuse, narrow to in-scope portion, confirm, or execute). Without one, context engineering produces a model that knows a great deal about the task and guesses on the cases that exceed what the context makes answerable.

Problem 5: You have not specified what requires human review before execution.

Context engineering makes agents capable. Capable agents take actions. Some of those actions are irreversible: deleting files, sending messages, committing code, calling external APIs that write state.

The Circuit Breaker is the prompt-level gate: certain operations require an explicit unlock phrase before the agent proceeds. It is not a guardrail against bad output. It is a gate against irreversible action. This is the component the context engineering frame correctly sets up but does not name, because it is not about what information the model has. It is about what the model is permitted to do with that information.

Problem 6: You do not have a structured test for whether your context design is correct.

Context engineering without validation is optimism. You design the context, run the task, the output looks right, and you ship. Then it fails on inputs that vary slightly from the ones you tested, because the context design had implicit assumptions you did not document.

The Ground Truth Contract is that documentation: a written specification, scoped to a specific prompt, that commits to what correct output looks like before testing begins. The Three-Constraint Rule is the diagnostic check for why a context design fails: every prompt that produces unreliable output is missing a complete specification of Intent, Context, or Guardrails. Missing any one of the three produces a specific and predictable failure pattern.

Constraint Case Study: context pollution in a document retrieval agent

A team builds a retrieval agent. The system prompt specifies role, task, and response scope. Retrieved documents supply context. The system works.

Over four months, the team adds tool schemas for three integrations, expands the system prompt to handle new use cases, and increases the example count. Each change is individually correct. Collectively, the Control Surface has grown to the point where the system prompt's core response-scope constraint is buried in a window where attention to early tokens is attenuated by everything that followed.

The agent starts producing responses outside the stated scope. The team adds a stronger constraint to the system prompt. The behaviour persists. The instinct is to blame prompt wording. The actual problem is architectural: the constraint is competing with accumulated context that was never budgeted.

A Control Surface audit at any point in those four months would have made the problem visible:

CONTROL SURFACE AUDIT
Date: [date]    Agent: Document Retrieval v3

COMPONENT               TOKENS      NOTES
System prompt           1,840       Core scope constraint at token 1,200+
Tool schemas            3,100       Three integrations; schemas verbose
Retrieved context       6,000       Unbounded; no per-query cap
Conversation history    4,200       No turn limit defined
Injected examples       1,600       Added incrementally over 4 months
User message            ~80         Runtime variable

TOTAL                  16,820 / 32,000 available

FLAGS
  System prompt scope constraint unreachable at typical attention depth
  Retrieved context volume unbounded: 6,000 tokens average; spikes to 12,000+
  No truncation policy defined for conversation history overflow
  Token allocation was never specified; components grew by accretion

REMEDIATION
  Move scope constraint to token positions 1-200 in system prompt
  Cap retrieved context at 3,500 tokens per query
  Define conversation history limit: 8 turns max, then summarise
  Set budget: 2,000 / 3,500 / 4,000 / 1,600 / variable

The context pollution was not a failure of instruction. It was a failure to design the Control Surface as a system rather than as a collection of individually correct additions.

Vocabulary Anchor: Control Surface

The Control Surface is every token the model receives at the moment of generation: system prompt, conversation history, tool schemas, retrieved context, user message, and injected examples combined. The Control Surface is the unit of design in context engineering. A prompt is one component of the Control Surface. Context engineering is the practice of designing the full surface deliberately.

In use: "The agent's inconsistent output traced to an uncontrolled Control Surface: two retrieved documents contradicted a system-prompt constraint, and the model was resolving the conflict by de-prioritizing the constraint. The fix was a retrieval pre-filter, not a prompt rewrite."

Where it does not apply: the Control Surface describes what the model receives synchronously at generation time. It does not describe the model's training-time knowledge, the state of external tools between turns, or outputs from prior generations in an agentic loop. Failures that appear consistent across sessions with entirely different Control Surfaces are more likely training-prior failures (Semantic Drift Vector) than Control Surface failures. Those require a different diagnostic.

Architecture Brief: the frame and the vocabulary are two layers, not two positions

Context engineering is a frame. It establishes that the unit of design is not the prompt but the full context the model receives. That reframe is correct and now broadly adopted.

What the frame opens up is the next layer: the vocabulary for each component of that context, the failure mode that emerges from each component, and the architectural pattern that prevents each failure mode. The frame tells you where to look. This vocabulary tells you what you are looking at and what to name it when it breaks. Naming the failure mode precisely is what enables the right fix. "The context was wrong" and "the Control Surface had an uncapped retrieved-context component at token position 9,000" are descriptions of the same event. Only one of them tells you what to change.

Next week: the Validation Suite, introduced in Issue 4. A Ground Truth Contract specifies correctness at a moment in time. Keeping it enforced as the model version changes and the domain's conventions evolve requires an automated check that runs without your intervention. The check compares live outputs to the reference set, surfaces deviations, and routes failures to a review queue. The most common mistake in building one is treating it as a test suite for the prompt rather than a contract-enforcement layer for the deployment. That distinction determines what you catch and what slips through.

Closing Calibration

One thing to try this week. Take your current production agent and list every component the model receives at the moment of generation: system prompt, conversation history, retrieved documents, tool schemas, injected examples. That list is your Control Surface as it currently exists, not as you imagined it when you wrote the prompt. Estimate the token count of each component. Compare the total to your context window budget.

If you have not set a budget, that is the first architectural gap. If you have set a budget but no one enforces it, that is the second. The context pollution scenario above took four months to become a production problem. The Control Surface audit that would have caught it takes twenty minutes.

The Constraint

Reply

Avatar

or to participate

Keep Reading