How to Stop an Agent Taking an Action It Cannot Take Back

An AI coding agent deleted a company's production database in nine seconds. The backups went with it. A separate incident: an agent ignored a code freeze and wiped a live database belonging to a startup, and the CEO of the platform apologised publicly. A third: a support agent was told "please refund all orders that cost more than zero dollars" and issued 247 refunds before anyone noticed.

The post-mortems keep landing on the same two words: approval gates. Missing environment separation, no human in the loop, nothing standing between the model deciding and the system executing.

That diagnosis is right, and it is also where most teams stop. What follows is what a control actually has to do, and why the two things people reach for first are not controls at all.

Prompting is guidance, not a control

"Never issue a refund above $500" in a system prompt is a request. It survives until a longer context, a confident user, or a phrasing nobody anticipated. The 247-refund incident was a single sentence of user input. Roughly a third of reported agent incidents involve prompt injection, which is to say: the instruction layer is not a boundary, it is a suggestion that usually works.

A control does not usually work. It works.

A second model reviewing the first is a second opinion

The common next step is a judge: a second model that reviews the proposed action and approves or rejects it. This feels like a control because something says no sometimes.

It is not one, for a reason that has nothing to do with how good the model is. Ask it the same question tomorrow and it can answer differently. Same inputs, different verdict. That property makes it unusable for the thing you actually need after an incident, which is not "did something block it" but "explain exactly why this specific action was allowed on March the fourth."

If your answer to an auditor is a model's opinion, you have moved the problem rather than solved it.

What a control has to do

Three things, and the third is the one people skip.

Evaluate the action, not the text. The dangerous thing is not what the model said. It is issue_refund(amount=900) about to execute. The check belongs at the commit point, between the decision and the effect, where the arguments are known.

Be typed, not semantic. refund_amt <= 500 is checkable. "Don't do anything risky" is not. If a rule cannot be written as a comparison over a named field, nothing can enforce it, and you are back to guidance.

Be reproducible. Same inputs, same verdict, forever. This is what turns a block into a record. An enforcement decision you cannot replay is an anecdote.

Ten lines

Here is the whole idea, running locally, with no proxy and no model call:

pip install causalor

from causalor import Causalor, AgentStateSchema, FormalConstraint

schema = AgentStateSchema(version="1.0.0")
schema.register_field("refund_amt", "number")

c = Causalor(tenant_id="getting-started")
c.set_state_schema(schema)
c.register_constraints([FormalConstraint(
    constraint_id="refund_ceiling", variable="refund_amt",
    operator="<=", threshold=500, scope="SAFETY_CONSTRAINT",
)], replace=True)

# the agent decides to refund $900. check before it runs.
result = c.pre_commit("customer-42", "issue_refund", state={"refund_amt": 900})
print(result.allowed)   # False

The output:

allowed: False   status: VIOLATION
proof: refund_amt=900 <= 500 -> False
       constraint_id=refund_ceiling   actual=900   threshold=500
       state_snapshot_hash=sha256:v1:766ccc52...

Run it again. The hash is identical. Run it next month on another machine and it is still identical, because the verdict is computed rather than inferred. That is the whole argument in one line: a model can be re-prompted into a different answer, and this cannot.

The half nobody talks about

Blocking at the commit point is the last line, and it is the wrong place to do all of your work. By the time an agent proposes a $900 refund, the reasoning that produced it has already drifted, and it will drift again next turn.

So the more useful question is not only "how do I stop the bad action" but "how do I notice the agent going off-plan while it is still reasoning". That means watching what the agent actually does, scoring how far it has moved from its policy, and writing a correction into its next turn, so the out-of-policy action is frequently never proposed at all.

Both halves matter, and they fail differently. Correction is continuous and improves as it learns an agent's failure modes. Blocking is absolute and does not improve, because it does not need to. You want the first to reduce how often you reach the boundary, and the second to make crossing it impossible.

What this does not solve

Worth being explicit, because governance content is usually written as though it solves everything.

It does not stop a model saying something wrong. This governs actions, not language. If your risk is a rude or inaccurate answer, this is the wrong layer.

It does not help if your rules cannot be written down. Some genuinely cannot, and "use good judgment about refunds" is not a constraint, it is a hope.

It does not remove the need for environment separation, least privilege, and not giving an agent production credentials it does not need. A control at the commit point is a complement to those, not a replacement. The nine-second incident had several missing layers, and only one of them was an approval gate.

The uncomfortable part

Most teams will read an incident like the database deletion, agree it is terrifying, and change nothing, because the agent works fine in testing and the failure mode is rare. It is rare right up until it is a post-mortem with your company's name in the headline.

The cheapest moment to add a control is before you need one. It is ten lines, it runs in your process, and blocking is free permanently.

Causalor is runtime governance for AI agents: deterministic blocking with a replayable proof, and drift correction while the agent is still reasoning. The quickstart is at causalorlabs.com/quickstart.