Context Engineering: The Skill That Replaced Prompt Engineering

A term that arrived quietly and then took over

In June 2025, two people with large followings in software said almost the same thing within a week of each other. Tobi Lutke, the chief executive of Shopify, wrote that he liked "the term 'context engineering' over prompt engineering", because it "describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM". Andrej Karpathy, formerly of OpenAI and Tesla, agreed, adding that in "every industrial-strength LLM app, context engineering is the delicate art and science of filling the context window with just the right information for the next step".

Simon Willison, who collected both remarks on his blog on 27 June 2025, predicted the name would stick, and it has. He had spent years defending "prompt engineering" as a real discipline and losing the argument, because most people heard it as a description of typing cleverly into a chatbot. The new term carries its own definition.

It was not a new coinage even then. Willison had quoted Riley Goodside using it in January 2023, splitting the job into "'context engineering', selecting and preparing relevant context for a task, and 'prompt programming', writing clear instructions". What changed in 2025 was that the second half stopped being the hard part.

What the two terms actually mean

Prompt engineering is the practice of writing and organising the instruction you give a model. Context engineering is the practice of deciding what the model can see at the moment it answers.

Anthropic's engineering team drew the line explicitly in September 2025. Prompt engineering covers "methods for writing and organizing LLM instructions for optimal outcomes". Context engineering, by contrast, "refers to the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference, including all the other information that may land there outside of the prompts". The distinction they draw is between a discrete task and a continuous one: "In contrast to the discrete task of writing a prompt, context engineering is iterative and the curation phase happens each time we decide what to pass to the model."

The engineer Philipp Schmid put it more bluntly in June 2025, in a definition that has been widely reused since: context engineering is "the discipline of designing and building dynamic systems that provides the right information and tools, in the right format, at the right time, to give a LLM everything it needs to accomplish a task". His diagnosis is the sentence that made the piece travel: "Most agent failures are not model failures anymore, they are context failures."

What lands in a context window in a working system is a lot more than a prompt. Schmid lists seven components: the system prompt or instructions, the user's own message, the conversation so far, long-term memory, retrieved documents, the descriptions of every tool the model can call, and the structure the output is expected to take. Every one of those is a decision, and every one of them costs something.

By July 2025 the field had a formal survey. "A Survey of Context Engineering for Large Language Models", led by Lingrui Mei and colleagues, analysed more than 1,400 research papers and described context engineering as "a formal discipline that transcends simple prompt design", concerned with the "systematic optimization of information payloads for LLMs".

Why more context is not better context

The intuitive assumption is that a bigger context window is straightforwardly better, and that the right move is therefore to put everything in and let the model sort it out. The research says otherwise, and this is the single most useful thing for a non-technical executive to understand about the subject.

In July 2025 the vector database company Chroma published a technical report called "Context Rot: How Increasing Input Tokens Impacts LLM Performance". Kelly Hong, Anton Troynikov and Jeff Huber evaluated 18 models, including the Claude, GPT, Gemini and Qwen families, and reached a blunt conclusion: "models do not use their context uniformly; instead, their performance grows increasingly unreliable as input length grows."

Several of their findings are counterintuitive enough to be worth stating individually. Performance degraded faster when the question was less semantically similar to the information being looked for, which is to say the harder the retrieval, the worse long context handled it. A single irrelevant but plausible distractor was enough to push performance below baseline, and more distractors made it worse. And, oddly, every one of the 18 models performed better when the surrounding text was shuffled into nonsense than when it was left as a coherent document.

Chroma is a company that sells retrieval infrastructure, and so has a commercial interest in the conclusion that long context is not a substitute for retrieval. The methodology and the code are published, which is the reason to take it seriously anyway. Anthropic subsequently adopted both the finding and the name, describing context rot as the observation that "as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases", and framing the underlying cause as an "attention budget": every additional token depletes it, because the transformer architecture requires each token to attend to every other one.

Independent work points the same way from a different angle. In May 2025, Philippe Laban and colleagues at Microsoft Research and Salesforce published "LLMs Get Lost In Multi-Turn Conversation", based on more than 200,000 simulated conversations. They measured an average 39% performance drop between single-turn and multi-turn versions of the same six tasks, across every leading open and closed model they tested. Their explanation is not that the models got less capable but that they got less reliable: "when LLMs take a wrong turn in a conversation, they get lost and do not recover."

Drew Breunig's taxonomy from June 2025 is the most useful shorthand for how this shows up in practice. He names four failure modes: context poisoning, "when a hallucination or other error makes it into the context, where it is repeatedly referenced"; context distraction, when the context grows so long the model over-focuses on it and neglects what it learned in training; context confusion, when superfluous content is used to generate a poor answer; and context clash, when newly accumulated information contradicts what is already there.

The finding that should concern regulated firms

In May 2026, Sam Martin and Fabien Roger published research on what happens when a model is used not to do the work but to watch it. Their paper, "Classifier Context Rot: Monitor Performance Degrades with Context Length", tested frontier models acting as classifiers over long transcripts of activity, looking for dangerous actions.

The result is that the watcher degrades along with everything else. Models missed subtle malicious actions between two and thirty times more often after 800,000 tokens of ordinary benign activity than they did in short contexts. On their benchmark, one model's recall fell from 98.6% to 88% once 800,000 tokens of benign content preceded the problem. For obvious attacks, which ought to be the easy case, recall fell from 99.7% to 69%. Detection was worst when the problem sat in the middle of the transcript rather than at the end. False positive rates stayed below 2% throughout, so the failure is not noise: the monitor simply stops noticing.

Anyone designing an automated control that reviews a long stream of activity should read that carefully. A monitoring model tested on a short sample and then deployed against a full day's transaction log has not been tested under the conditions it will actually work in.

What the discipline actually consists of

The techniques that have settled out are less exotic than the name suggests. Anthropic's engineering team describes four.

Compaction means taking a conversation approaching the limit of the window, summarising it, and restarting with the summary. It is the most common technique and it has an obvious failure mode, which Anthropic states plainly: "Overly aggressive compaction can result in the loss of subtle but critical context whose importance only becomes apparent later."

Structured note-taking means the system writes notes to a file outside the context window and reads them back when needed, so that state survives the window being cleared.

Just-in-time retrieval means holding lightweight identifiers, file paths, stored queries, links, and loading the actual content only at the moment it is needed, rather than pre-loading everything that might turn out to be relevant.

Sub-agents means giving a focused task to a separate model instance with a clean window, which explores at length and returns only a condensed summary. Anthropic quantifies the compression: a sub-agent might use tens of thousands of tokens and return one to two thousand.

Anthropic reports internal evaluation figures for the combination of memory and context editing: a 39% improvement over baseline, 29% for context editing on its own, and an 84% reduction in token consumption on a hundred-turn web search evaluation. These are the vendor's own numbers on the vendor's own evaluation set, not an independent benchmark, and should be read as such.

Two results worth knowing before you spend money on this

The first is that the sophisticated option is not reliably the better one. In August 2025, Tobias Lindenbauer and colleagues published "The Complexity Trap", comparing intelligent LLM summarisation of an agent's history against the crude alternative of simply masking out old tool outputs. On SWE-bench Verified with one large model, masking scored 54.8% and summarisation 53.8%, at a cost of $0.61 per instance against a raw baseline of $1.29. The cheap technique matched the expensive one. Summarisation also lengthened the average run by 13 to 15%, which explains part of the gap.

The second is that the experts disagree about sub-agents. Anthropic recommends them. Cognition, which builds the coding agent Devin, published a piece in June 2025 titled "Don't Build Multi-Agents" arguing the opposite: that parallel sub-agents cannot see each other's work, act on unstated assumptions, and produce incompatible output, and that as things stood, agents collaborating with one another "only results in fragile systems". Both are credible practitioners with production systems. The disagreement is genuine and unresolved, which is a reasonable signal that the field is younger than the confidence of its vocabulary suggests.

There is also a cost dimension that gets overlooked. Anthropic's own measurement of its research system found that agents typically use around four times the tokens of a chat interaction, and multi-agent systems around fifteen times. Context is not free, and the decision to give a model more of it is a spending decision as well as a design one.

What this means if you are buying rather than building

Three things follow for an organisation that is deploying AI rather than developing it.

The first is that "we have a million token context window" is not an answer to "how do you make sure it finds the right information". The evidence is that the second question gets harder as the window gets bigger, not easier. Ask what is being put into the window and why, and what is deliberately being left out.

The second is that a demonstration on a short example tells you very little about behaviour on a long one. If the system will run against a full case file, a full day of transactions or a full contract set, it needs to be tested against those, at that length, with the relevant item buried in the middle rather than sitting at the end.

The third is that this is where the skill now sits. The valuable thing is no longer knowing how to phrase a request. It is knowing what the model needs to see, what it must not see, and how to keep that true over hours of work rather than a single exchange. That is a design discipline, and it looks a great deal more like information architecture than like writing.

References

Origin and definition of the term

Simon Willison - Riley Goodside on prompt engineering (23 January 2023) https://simonwillison.net/2023/Jan/23/

Simon Willison - Context engineering (27 June 2025), collecting the Tobi Lutke and Andrej Karpathy remarks https://simonwillison.net/2025/jun/27/context-engineering/

Philipp Schmid - The New Skill in AI is Not Prompting, It's Context Engineering (30 June 2025) https://www.philschmid.de/context-engineering

Lingrui Mei et al. - A Survey of Context Engineering for Large Language Models, arXiv:2507.13334 (17 July 2025) https://arxiv.org/abs/2507.13334

What context engineering consists of

Anthropic - Effective context engineering for AI agents (29 September 2025) https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

Anthropic - Managing context on the Claude Developer Platform (29 September 2025), source of the 39%, 29% and 84% internal evaluation figures https://claude.com/blog/context-management

Anthropic - How we built our multi-agent research system (13 June 2025), source of the 4x and 15x token figures https://www.anthropic.com/engineering/multi-agent-research-system

Evidence on context rot and long-context degradation

Kelly Hong, Anton Troynikov and Jeff Huber - Context Rot: How Increasing Input Tokens Impacts LLM Performance, Chroma technical report (14 July 2025) https://www.trychroma.com/research/context-rot

Sam Martin and Fabien Roger - Classifier Context Rot: Monitor Performance Degrades with Context Length, arXiv:2605.12366 (12 May 2026) https://arxiv.org/abs/2605.12366

Philippe Laban, Hiroaki Hayashi, Yingbo Zhou and Jennifer Neville - LLMs Get Lost In Multi-Turn Conversation, arXiv:2505.06120 (9 May 2025) https://arxiv.org/abs/2505.06120

Drew Breunig - How Long Contexts Fail (22 June 2025) https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html

Comparative results and dissent

Tobias Lindenbauer, Igor Slinko, Ludwig Felder, Egor Bogomolov and Yaroslav Zharov - The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management, arXiv:2508.21433 (29 August 2025) https://arxiv.org/abs/2508.21433

Walden Yan - Don't Build Multi-Agents, Cognition (12 June 2025) https://cognition.com/blog/dont-build-multi-agents

Image

Image by makabera on Pixabay (Pixabay image ID 8372701), used under the Pixabay Content License, which permits free use without attribution. Credit given as a courtesy.