AI Agents in Production: Autonomy Is an Architectural Decision

AI agents in production: autonomy is an architectural decision.

In my last article, I wrote about context orchestration and MCP.

About delivering the right context, the right way, to the right tool.

But there is a question that comes right after, and almost nobody asks it out loud:

Once the agent has context and tools, what is it allowed to do on its own?

That is the most important question of the agent era.

And it is not a technical question.

It is an architectural one.

The autonomous agent hype

The current narrative is seductive:

• Agents that resolve tickets on their own
• Agents that deploy code
• Agents that negotiate with suppliers
• Agents that operate financial systems
• Agents that talk to other agents

In the demo, everything works.

The agent receives a goal, plans, calls tools, adjusts the plan, and delivers the result.

Impressive.

But in production, the question changes:

• What happens when the agent gets it wrong?
• Who approved that action?
• Can it be undone?
• How much did that 47-step run cost?
• Why did it call that API three times?

If your architecture cannot answer those questions, you do not have an agent.

You have an operational risk with a chat interface.

An agent is not a feature. It is a distributed system with autonomy.

An agent in production brings together almost every classic distributed systems problem:

• Calls to external services that can fail
• State that must be kept across steps
• Operations that must be idempotent
• Concurrency between runs
• Timeouts, retries, and compensations

And it adds a new one:

The flow is not deterministic. The next step is decided by a probabilistic model.

That changes everything.

In a traditional system, the architect designs the flow.

In an agent system, the architect designs the boundaries of the flow.

That is the difference between software engineering and AI architecture.

Levels of autonomy

When I work with teams adopting agents, the first thing I do is remove the word “autonomous” from the conversation.

And replace it with a scale.

Level	The agent...	Example
0 — Assistant	Suggests, a human executes	Draft reply for support
1 — Copilot	Executes reversible actions with approval	Opens a PR for review
2 — Supervised	Acts alone within limits, a human audits	Classifies and routes tickets
3 — Bounded autonomy	Takes real-impact actions within a budget	Refunds up to a set amount
4 — Autonomous	Decides and acts without direct supervision	Rarely justifiable today

The goal is not to reach level 4.

The goal is that every action, of every agent, has a defined level.

And that level must live in the architecture.

Not in the prompt.

Why the prompt is not a boundary

A mistake I see all the time:

“I put in the system prompt that it cannot delete records.”

That is not control.

It is a suggestion.

A prompt is a probabilistic instruction. It can be ignored, misread, or overridden by malicious content coming from a document, an email, or a web page.

Real boundaries live outside the model:

• Tool permissions, not agent intentions
• Minimal credential scopes
• Allow-lists of actions per context
• Parameter validation before execution
• Human approval as a step in the flow, not a polite request

If the agent cannot delete records because the tool does not expose that operation, you have a boundary.

If it should not delete them because the prompt asked nicely, you have a hope.

Four architectural decisions before putting an agent in production

1. Classify actions by reversibility and impact

Every tool the agent can reach should be classified:

• Read — query data
• Reversible write — create a draft, open a PR, schedule
• Irreversible write — send an email, pay, delete, publish

Irreversible actions should not go through without an explicit confirmation step.

It is that simple.

2. Give the agent a budget

Agents without limits on steps, time, and cost are an invoice waiting to happen.

Define per run:

• Maximum number of steps
• Maximum time
• Maximum token cost
• Maximum calls per tool

When the budget runs out, the agent stops and escalates to a human.

This ties directly to what I wrote about AI cost in production: token price is just the beginning. The real cost lives in system behavior.

3. Treat every run as an auditable transaction

Every agent run should produce a record with:

• The goal it received
• The context it used
• The plan it generated
• Every tool called, with parameters and results
• Decisions made and why
• Cost and duration

Without it, you cannot explain what happened.

And if you cannot explain it, you cannot trust it.

4. Design the failure path before the happy path

The question is not “can the agent solve it?”.

The question is “what happens when it cannot?”.

• Does it stop or keep trying?
• Who does it escalate to?
• Is partial state left consistent?
• Is there compensation for actions already taken?

A good agent is not one that never fails.

It is one that fails predictably.

Multi-agent: more agents, more architecture

Many teams, when they hit the limits of a single agent, respond by creating more agents.

A planner. An executor. A reviewer. An agent that coordinates the others.

Sometimes it makes sense.

But every additional agent multiplies:

• Points of failure
• Cost
• Latency
• The difficulty of tracing a decision

Before adding an agent, ask:

Does this need another agent, or a well-designed deterministic step?

Most of the time, a deterministic flow that calls a model at the right points is cheaper, faster, and more reliable than a society of agents talking to each other.

The architect's role in the agent era

The AI architect is not the person who picks the trendiest agent framework.

It is the person who answers:

• Where the model decides and where the system decides
• Which actions require a human in the loop
• Which limits are guaranteed by infrastructure
• How every run is traced and audited
• How the system behaves when the model is wrong

That is what turns a demo agent into an operational agent.

Final reflection

Autonomy is not a feature you switch on in the model.

It is a trust budget you grant to the system.

And that budget should grow only as fast as your ability to observe, limit, and correct what the agent does.

Companies that treat autonomy as configuration will collect incidents.

Companies that treat autonomy as an architectural decision will scale agents safely.

In the next article, I will go deeper into the foundation that supports all of this: observability in AI systems.

Because there is no autonomy without traceability.