How Do You Stop Agents from Doing Dangerous Things?

Part 4 of 6 — “Governing AI Agents in the Enterprise: A Practical Architecture Guide”

This article covers preventive governance: the controls that stop an agent from taking an unauthorized, dangerous, or cascading-failure action — before it happens, not after. It covers four questions: policy enforcement, agent identity, execution sandboxing, and reliability engineering.

4.1 - How do we stop an agent from taking a dangerous or non-compliant action in the first place -- not just detect it afterward?

The pattern: Runtime policy enforcement at the action boundary.

Evaluation and telemetry are largely detectivecontrols — they tell you something went wrong, often after it did. For high-consequence actions you need a preventivecontrol: a deterministic policy engine that sits between the agent’s decision and the real-world action, and that can permit, deny, narrow, or escalate for approval before the action executes.

Critically, this enforcement must be rule-based and deterministic — you do not want a second probabilistic model deciding whether the first one is allowed to act. Policy decisions need to be fast (well under a millisecond), auditable, and predictable.

A tool like the Microsoft Agent Governance Toolkit (AGT) provides a policy engine where policies are written declaratively — in YAML, OPA Rego, or Cedar — and evaluated against every tool invocation. The engine integrates with an agent framework’s callback hooks, so every tool call is checked against policy before it executes. A policy can outright deny an action, require human approval for it, rate-limit it, or constrain its arguments. Each decision is recorded in the audit log.

A declarative policy — versioned in git, readable without engineering context:

rules:
  - id: block-destructive-sql
    match: { tool: db_query }
    deny_if: { arg.query: { contains_any: [DROP, DELETE, TRUNCATE] } }
  - id: approve-external-pii
    match: { tool: send_external }
    require_approval_if: { arg.payload: { contains_pii: true } }
  - id: rate-limit-search
    match: { tool: web_search }
    rate_limit: { per_agent: 30, window_s: 60 }

The callback that evaluates every action against these rules before the tool runs:

from agt.policy import PolicyEngine  # API is representative — verify AGT's current signatures
engine = PolicyEngine.from_file("policy.yaml")

def before_tool_callback(tool, args, ctx):
    decision = engine.evaluate(tool=tool.name, args=args, principal=ctx.agent_id)
    if decision.effect == "deny":
        raise PermissionError(decision.reason)      # blocked before the tool runs
    if decision.effect == "require_approval":
        return request_human_approval(tool, args, decision)
    return None                                     # allow

agent = Agent(tools=[...], before_tool_callback=before_tool_callback)

Real-world examples

4.2 - We have a swarm of agents and sub-agents. How do we know which agent did what, and on whose authority?

The pattern: Cryptographic agent identity and inter-agent trust.

In a multi-agent system, “the agent did it” is not an accountable statement — there are many agents, they call one another, and they act on behalf of different users and systems. Each agent needs a verifiable, non-spoofable identity, and inter-agent calls need mutual authentication, so that every action can be attributed to a specific agent acting under a specific authority.

The AGT identity module gives each agent a cryptographic identity — a decentralized identifier signed with strong keys — and supports mutual TLS between agents. An inter-agent trust protocol lets a calling agent and a called agent verify each other before exchanging work. Identities have a lifecycle: they can be issued, suspended, and revoked, including cascade revocation if a parent identity is compromised.

This identity stack is cloud-agnostic — it does not depend on any cloud provider’s directory — so it works in multi-cloud, hybrid, and sovereign deployments.

In practice, each agent presents a signed SVID (SPIFFE Verifiable Identity Document) or W3C DID; the callee verifies it over mutual TLS before accepting work. SPIFFE/SPIRE handles workload identity; W3C DIDs extend this to cross-boundary and cross-organization scenarios — the concrete technologies behind AGT’s identity module.

Real-world examples

4.3 - Our agents execute code and call real systems. When one goes wrong, how do we contain the blast radius?

The pattern: Execution sandboxing — isolate what runs.

A policy engine decides whether an action is allowed. It does not control what damage that action can do once it is running. If an agent can execute code — through a code-interpreter tool, a shell tool, a Python runtime, or a third-party tool that itself runs code — then a malicious or hallucinated payload that the policy permitted can still read files, open network connections, or consume resources it should not. You need execution isolation.

The AGT runtime module provides layered execution sandboxing — graduated isolation rings — for code-executing tools. Code runs with constrained file-system, network, and resource access, so even a policy-permitted action cannot reach beyond its intended boundary.

This module is mandatory from day one for any agent with a code-executing tool. It is not a scale-up option, because policy enforcement alone does not contain runtime behavior.

Any tool that runs model-written code needs isolation from day one. A minimal runnable example:

from e2b_code_interpreter import Sandbox  # or gVisor / Firejail / Docker with dropped caps

def run_generated_code(code: str, dataset_path: str) -> str:
    with Sandbox() as sbox:                    # network off, ephemeral FS, resource caps
        sbox.files.write("/data.csv", open(dataset_path, "rb").read())
        execution = sbox.run_code(code)        # cannot reach host FS or internal network
        return execution.text

Real-world examples

4.4 - One stuck or looping agent threatens to take down the whole workflow. How do we stop cascading failure and hit an emergency stop?

The pattern: Reliability engineering for agents — circuit breakers, SLOs, and a kill switch.

In a multi-agent workflow, one failing, looping, or degraded agent can fan out and exhaust budgets, rate limits, and downstream capacity across the entire system. Agents need the same reliability controls mature distributed systems have: circuit breakers that isolate a failing dependency, service-level objectives that quantify acceptable failure, and — non-negotiably — a kill switch that can halt an agent or fleet immediately.

The kill switch is both an engineering control and increasingly a regulatory expectation.

The AGT reliability module provides circuit breakers between agents, SLO and error-budget tracking, chaos-testing support, and an emergency kill switch. Wired into the agent runner and the transitions between sub-agents, it prevents one bad actor in the graph from cascading into a system-wide outage.

For any multi-agent application, this module is core — not optional.

A circuit breaker and kill-switch check using standard libraries:

import pybreaker

market_data_breaker = pybreaker.CircuitBreaker(fail_max=5, reset_timeout=30)

@market_data_breaker                     # trips after 5 failures; dependents fail fast
def market_data(symbol): ...

def before_tool_callback(tool, args, ctx):   # kill switch checked at every action
    if kill_switch.is_engaged(scope=ctx.fleet):
        raise RuntimeError("Automation halted by kill switch")

Real-world examples

The distinction that matters

Questions 5 through 8 all live in the governance layer, but they address different threat surfaces:

All four are needed. They are not substitutes for one another. The next article covers the accountability side of governance — how to prove that all of this was in place when a regulator or auditor asks.

Appendix: Standards-based policy engine

For teams already running Open Policy Agent or preferring a portable, standards-based engine, the §4.1 rules translate directly to Rego:

package agent.policy

deny[msg] {
    input.tool == "db_query"
    re_match(`(?i)\b(DROP|DELETE|TRUNCATE)\b`, input.args.query)
    msg := "destructive SQL blocked"
}

Evaluate with opa eval or the OPA Go/Python SDK. Cedar (Amazon) is an alternative engine with similar expressive power; AGT’s own policy engine supports both.