A powerful model receives the task “fix the checkout.” It edits the right file, announces victory, and leaves without running a test. The checkout remains broken, but now it has confidence.
It is tempting to conclude that the model lacks intelligence. Sometimes it does. Often, however, it lacks the environment that turns capability into reliable work: suitable tools, completion criteria, limits, persistent state, tests, and feedback.
That surrounding system is the harness. Harness Engineering is the discipline of designing it.
What is a harness, anyway?
In everyday English, a harness can restrain, connect, or support. For agents, the useful metaphor is a system that connects and controls the available power.
The model decides and generates. The harness organizes everything around it:
- which tools may be used and with which permissions;
- how the agent discovers the repository and starts the environment;
- which context enters each stage;
- how progress and decisions survive another session;
- which tests prove that the task is complete;
- when to retry, stop, or request human judgment;
- which logs, metrics, and traces make failures investigable.
It is not merely a larger prompt. In fact, a giant manual is often a poor harness disguised as documentation.
The same intelligence, two outcomes
Consider two agents using the same model.
The first receives unrestricted terminal access and the instruction “make it work.” The second enters an isolated worktree, reads short domain instructions, gets a bug reproduction, uses scoped tools, runs focused tests, and finishes only when the acceptance criterion can be verified.
The second agent did not gain neurons. It gained rails and instruments.
That is the central lesson of OpenAI's case study, Harness engineering: leveraging Codex in an agent-first world. A team reports building an internal product with roughly one million agent-generated lines of code, about 1,500 pull requests, and an estimated one tenth of the manual development time.
Those figures are impressive, but they need the right label: they come from OpenAI, for a specific product, using infrastructure designed for Codex. They are not an independent benchmark or a universal productivity promise. The case is most valuable for what the team had to build around the model.
Legibility before autonomy
An agent does not observe a system like a team that has worked on it for five years. If the architecture exists only in someone's head, the agent receives a puzzle without the picture on the box.
The OpenAI team made the application legible through isolated environments, browser access, searchable logs, metrics, traces, structural linters, and mechanically checked architecture rules. Instead of writing “keep the architecture clean,” it encoded permitted dependency directions and quality invariants.
Guidance requests good behavior. A verifiable constraint makes behavior observable.
Documentation still matters, but it should operate as a map. Global instructions point to specific rules; local files describe only the relevant domain. When everything has maximum priority, the agent does what people do: chooses something and hopes for the best.
The harness also carries memory
Long tasks outlive a context window. Anthropic explored this in Effective harnesses for long-running agents. The experiment separated an initializer agent, responsible for preparing the environment and requirements, from later sessions that advanced one feature at a time.
State survived through simple artifacts: a structured feature list, a progress file, and Git history. Each session began by checking the environment and ended by leaving code and notes in a clean state for the next one.
Without that support, the article observed two recurring failures: trying to do everything at once until context runs out, and looking at a partially completed project and declaring it finished. Any similarity to a home renovation is purely coincidental.
The lesson is not to create progress.txt everywhere. It is to make state external, verifiable, and resumable. Depending on the risk, this may be a state machine, an execution database, commits, checkpoints, or immutable events.
Six layers of a useful harness
| Layer | Question it must answer |
|---|---|
| environment | where can the agent work without affecting what it should not? |
| context | how does it find the right information without loading the planet? |
| tools | which actions exist, with clear schemas and errors? |
| control | which permissions, limits, confirmations, and stop conditions apply? |
| verification | what evidence demonstrates success or failure? |
| observability | how can we reconstruct what happened and what it cost? |
The model moves through these layers in a loop: observe, decide, act, receive feedback, and update state. When something fails, the harness should return usable evidence — not only error: something went wrong, the digital equivalent of shouting “it broke” and hanging up.
Feedback loops without an infinite roulette wheel
Agents improve when they can test their actions. A compiler, unit test, or screenshot provides a more concrete signal than “looks right.” But loops need budgets and exits.
Define a maximum number of attempts, required progress between attempts, failures eligible for automatic correction, actions requiring human approval, terminal conditions, and evidence to preserve. Otherwise, “reflect and try again” can become a sophisticated machine for repeating the same mistake with different wording.
Permissions must also be proportional. Reading files, editing a branch, and publishing to production are not the same level of authority. A harness can start restricted, expand only when necessary, and record every external effect. Autonomy is a ladder, not a switch.
A model benchmark does not measure the whole system
Agent results depend on the model, tools, prompt, environment, and evaluator. METR measures task-completion horizons using more than 100 problems, mainly in software, ML, and security. The organization itself warns that performance varies with the exact agent configuration and that well-specified, automatically evaluated tasks are cleaner than much real economic work.
Choosing the leader of a ranking therefore does not finish the architecture. Measure the product: tasks completed with evidence, regressions introduced, cost and time per accepted result, human interventions, permission failures, unintended effects, and recovery after failure.
Test harness versions with the same model too. Sometimes a clearer tool or acceptance test produces a larger gain than an expensive model upgrade. Without a controlled experiment, the credit goes to whoever has the brightest logo.
The main point
Harness Engineering shifts the question from “which model writes better?” to “which system lets the model work well, safely, and with proof?” It is an engineering change, not a vocabulary change.
The model remains the engine. Engines do not choose the route, install brakes, inspect the dashboard, or decide who gets the key. A good harness gets that power to the destination — preferably without the broken checkout in the back seat.
References
- OpenAI. Harness engineering: leveraging Codex in an agent-first world, 2026.
- Anthropic. Effective harnesses for long-running agents, 2025.
- Anthropic. Building a C compiler with a team of parallel Claudes, 2026.
- METR. Task-Completion Time Horizons of Frontier AI Models, updated May 2026.