AI Harness: the layer that turns an LLM into a useful agent

Ask an LLM: “Fix the bug, run the tests and open a PR”. On its own, it can’t. A language model takes text in and gives text back. It doesn’t open files, doesn’t run tests, doesn’t remember what it did ten minutes ago and doesn’t know what it is allowed to do.

What turns that model into an agent that can carry out the request is the software layer around it: the harness. Think of the LLM as the brain. The harness is everything that lets that brain act in the real world, safely.

What an AI harness is#

Three terms show up together and are often confused:

• LLM: the model. It interprets instructions and generates text.
• Agent: the LLM using tools, over several steps, to reach a goal.
• Harness: the structure around the model that makes this possible and safe. It decides what goes into the context, which tools exist, when to stop, what needs approval and how to check the result.

A harness is not a long prompt or a specific framework. It can be a hundred-line script or a whole platform. Anthropic, for example, describes the SDK that runs Claude in a loop as a general-purpose agent harness (Anthropic, 2025).

To see a harness that coordinates several agents, read Six Thinking Hats in the Age of LLMs.

The five pieces of a harness#

1. Context and memory#

An agent needs more than the current request. The harness organizes three kinds of information:

State works like a structured scratchpad. For the PR request, it could look like this:

status = "testing"
files_changed = 3
deployment = "staging"

With this, the agent doesn’t need to reread the whole conversation to know where it stands.

2. Tools and execution#

An LLM only generates text. It becomes an agent when it can act, and the harness is what hands it the tools: Python functions, APIs, databases, GitHub, a browser, cloud services and MCP servers, the open protocol that connects models to tools and data sources.

For the PR request, the work becomes a loop:

Read the code, understand it, edit, run the tests, read the result and fix. Running this loop is one of the harness’s most important jobs.

3. Flow and limits#

Beyond execution, the harness decides how the work moves forward. It plans the steps, runs them one at a time or several in parallel, and defines when to stop.

4. Security, permissions and privacy#

This is where the harness becomes essential, especially in companies. Picture a coding agent with access to the production database, customer data, repositories, cloud credentials and internal APIs. Nobody wants to give it unrestricted access to all of that. The harness controls three things:

• Sandbox: the agent runs in an isolated environment, with limited access to files and the network.
• Permissions: what the agent can do on its own and what needs approval.
• Privacy: which data can be sent to the model.

For the PR request, the permissions could look like this:

5. Evaluation#

One of the biggest challenges with agents is knowing whether the work is good. The harness can combine three layers, from cheapest to most expensive:

An LLM judge helps, but it doesn’t replace the other layers. It has known biases: it tends to prefer the first answer shown, longer answers and text written by its own model (Zheng et al., 2023). Compare its scores with human ratings before trusting them.

How the pieces connect#

Follow the arrows in the image:

1. The user hands the request to the harness.
2. The harness assembles what the model needs: instructions, context, available tools and permissions.
3. The model chooses the next action, such as reading a file or calling an API. In practice, it only requests the call. The harness is what executes it in the real world, after checking permissions.
4. The result comes back, and the harness updates the state, checks the limits and decides whether the loop continues.
5. When the task is done or a limit is reached, the harness evaluates the result and responds to the user.

An analogy: the new hire#

Think of someone who just joined a company. The LLM is their intelligence. The harness is the workplace around them:

The person can be brilliant. Even so, it is the environment around them that determines how safely and efficiently that intelligence gets used.

Checklist: questions to design your harness#

An LLM alone can’t answer any of these questions. The harness has to answer them for it:

• What can the agent access?
• Which tools can it use?
• Where does the task state live?
• When should it try again?
• When should it stop?
• How do we know the result is correct?
• Which actions require human approval?
• What data must never be sent to the model?

Conclusion#

AI systems are moving out of the chat window and starting to act: they edit code, query databases, open PRs. In that shift, choosing the model still matters, but it is no longer the only decision. Context, tools, memory, flow, security, permissions and evaluation are engineering choices too, and they are what separate a demo from a reliable system.

The model provides the intelligence. The harness provides the environment that intelligence works in.

References#

1. Anthropic (2024). Building effective agents. Link
2. Young, J. / Anthropic (2025). Effective harnesses for long-running agents. Link
3. OWASP (2025). LLM01:2025 Prompt Injection. OWASP Top 10 for LLM Applications. Link
4. Zheng, L. et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS 2023 Datasets and Benchmarks. arXiv:2306.05685