Autonomy is a spectrum, and it is worth being precise about where you are
"Autonomous" is used to mean everything from "runs three tool calls before checking in" to "opens pull requests unattended overnight". The distinctions matter because each step costs something specific.
- Suggests. The human executes. No autonomy, no supervision cost.
- Acts with approval. Each action is confirmed. The cost is your attention, paid continuously.
- Acts within a boundary. Free inside a defined scope, approval required to cross it. This is where most productive work happens, and defining the boundary is the actual engineering.
- Runs unattended, reports at the end. Requires detection of silent failure, because nobody is looking.
- Runs continuously and self-recovers. Requires everything above plus a recovery policy that does not destroy work in progress.
Almost every honest deployment is level three with a level-four aspiration. Vendors mostly sell level five.
What only breaks when nobody is watching
Supervised agents fail in ways you notice immediately because you are there. Unattended agents fail in ways designed, apparently, to be missed.
Alive and silent
The single most expensive failure mode we have dealt with. The process is running, memory and CPU look normal, the logs are quiet, and no work is happening. Every process-level health check passes. The only thing that distinguishes this from an agent thinking hard is asking it to do something and checking that it did.
A specific variant is worth naming because it defeats the obvious fix: a supporting subprocess that never started. Restarting does not help, because the restart re-fails the same way, and you get a loop of a supervisor restarting an agent that will never come up. The cure was reconnecting the subsystem rather than restarting the agent, and the general lesson is that an escalation ladder whose rungs are all "try again" is one rung long.
Confidently complete
The agent finishes, reports success and is wrong. Unattended, this is worse than a crash, because downstream steps trust it. The mitigation is not better instructions; it is making success externally checkable, so that "done" is something a test asserts rather than something the agent believes.
Recovery that eats the work
The restart that clears the outage also discards an hour of correct reasoning. This is the tension at the centre of unattended operation: react too early and you destroy good work, too late and you burn the window. Our own thresholds landed at a nudge first and a restart only after several minutes of genuine silence, because we kept killing agents that were fine.
What makes autonomy affordable
Three properties, in order of importance:
- Reversibility. If everything can be undone, a bad run is cheap and long leashes are rational. This is the whole argument, and everything else is detail.
- Isolation. An isolated checkout means a wrong run cannot take anything else with it, and it is what makes running several agents at once possible rather than merely parallel.
- External verification. A test the agent runs, and a probe someone else runs. The agent's own report is evidence of intent, not of outcome.
Where those three do not hold, the answer is not more supervision. It is less autonomy, at that specific boundary, permanently. Anything touching authentication, payments, data deletion or schema belongs on the approval side of the line no matter how good the agent gets, because the cost of being wrong does not shrink as capability grows.
A soak period is underrated
If an autonomous agent produces changes faster than humans can review them, the honest options are to slow it down or to accept unreviewed merges. There is a third that costs little: make automated changes wait. A pull request that must be a set age before it can merge converts "nobody reviewed this" into "nobody objected for a day", and in practice that catches a meaningful share of obviously-wrong work at close to zero cost. It is weaker than review. It is much stronger than a rubber stamp.
Related reading
The supervision architecture behind all of this is in agentic engineering, the day-to-day craft in agentic coding, and the tooling categories in AI coding agents. The strongest argument against extending autonomy is at when not to use AI agents, and the failure surface at system scale is documented in hallucinations at system scale.
Two outside sources worth the time: Martin Fowler's running notes on working with generative AI, which are unusually specific about failure, and Anthropic's guidance on building effective agents, which is notably conservative about when autonomy is warranted.