Oriental Nippon Development
Back to blog
Technology

Governed Autonomy for AI Agents: Let the Model Think, Not Act

A regulated utility lets AI agents diagnose production incidents but never act on their own. Here is the control-plane pattern behind it, and how it could be adapted for hospitals and enterprise platforms.

Written byHassan Mamoon

It is 06:40 on a hospital ward. The interface that carries lab results into the ward system has slowed down, the lab system is quietly retrying, and a queue is growing. No alarm has fired, because none of those signals is bad enough on its own. In forty minutes, doctors start the morning round without the latest blood results.

An AI agent could spot this pattern in seconds. It could also restart the interface. The real design question is whether it should be allowed to, and who decides.

Purnang Borah's InfoQ article, Governed Autonomous Intelligence for Mission-Critical Utility, gives one of the clearest answers so far. It describes an agentic operations platform running in a regulated electricity utility, where a wrong action can take down bill payments or touch systems involved in delivering power. The pattern transfers almost unchanged to healthcare, finance and B2B platforms. This post explains how it works and how it could be adapted outside the energy sector.

The idea in one paragraph

Split the platform into a side that thinks and a side that acts, and put a wall between them. AI agents live entirely on the thinking side: they correlate signals, read runbooks, estimate business impact and propose a fix. Every proposal has to pass a rulebook that people wrote, version and review, and then a single gateway that is the only route into production. The model can propose anything. It can only do what the rules allow, with a person's approval wherever the rules require one. Autonomy is then switched on one kind of action at a time, based on evidence, and can be switched off just as easily.

The model can propose anything. It can execute only what the policy gate authorizes. Authority never lives inside the model.

The problem is authority, not intelligence

Most conversations about AI in operations focus on whether the model is smart enough. That is the wrong question for systems where mistakes are expensive.

The article draws a useful distinction: "A wrong summary is inconvenient. A wrong production action can affect a customer-facing service." In a utility that might mean billing or regulatory reporting. In a hospital it might mean a medication or results system going dark during a ward round. No model is right often enough to be trusted with those actions on its own, and "the model was confident" is not something you can show an auditor.

So the utility did not try to make the model trustworthy. It made the model's trustworthiness matter less.

Four parts and one door

The pattern borrows the idea of a control plane from networking and cloud infrastructure: one layer decides what should happen, a separate layer carries it out.

PartIts jobCan it change production?
Agents (the reasoning plane)Correlate signals, retrieve runbooks and incident history, estimate impact, propose one actionNever
Action catalogA versioned list of every permitted action, its risk class and its approval ruleNo, it is configuration
Policy gateDeterministic code that decides whether a proposal is allowed and what approval it needsNo, it only decides
Tool gatewayRe-checks everything, borrows a narrowly scoped credential, performs the action, records the resultYes, and nothing else can

The gateway only counts as a boundary if nothing can go around it. In the utility's design that means no production credentials anywhere in the agents' runtime, deny-by-default identity policies, tightly restricted network paths, and credentials exposed only to the one tool that needs them. The article's phrase for this is "one door to production, with every control attached to that door."

A wrong root cause becomes a bad proposal, not a bad action

This is the most important consequence of the design. In a conventional "autonomous" design, a hallucinated root cause turns straight into a bad change in production. Behind a policy gate, it turns into a bad proposal. The gate can reject it, a person can review it, or the gateway can refuse to authorise it.

The model's mistakes still matter. Their cost is just capped by how the system is built, rather than by how good this month's model happens to be.

Narrow agents beat one clever agent

The utility does not use one large agent with access to everything. It uses an orchestrator and a handful of narrow agents:

  • Orchestrator: receives each event, decides what analysis is needed and assembles the findings. "It coordinates but does not execute."
  • Application Reliability and Platform Intelligence agents: compare failures, latency and dependency behaviour with known patterns and history.
  • Business Impact agent: turns technical state into consequences: billing at risk, a reporting deadline, a reliability commitment.
  • Remediation agent: the only one allowed to propose a production action, and even then only a proposal.

Small agents are easier to test, constrain and explain. Two other details matter just as much, and both are easy to skip.

Knowledge hygiene. Retrieval will happily return an outdated runbook. So every runbook has a named owner and a last-review date, both are returned with every passage, overdue runbooks are flagged to their owner, and when an operator rejects a recommendation because the runbook was wrong, the rejection goes back to that owner.

Facts come from code, not the model. Which system raised an alert, whether two IDs refer to the same service, and which business service depends on it are established by ordinary parsers and the service catalog. The model helps interpret an unfamiliar log line. It does not decide what the basic facts are. Applied to the ward example above, a normalised event could look like this:

{
  "event_id": "evt-ward-0640",
  "timestamp": "2026-11-18T06:40:12Z",
  "source": "interface-monitor",
  "service": "lab-results-interface",
  "environment": "prod",
  "event_type": "latency_degradation",
  "technical_severity": "warning",
  "dependency_ids": ["lab-system", "results-queue"],
  "business_service": {
    "name": "Ward results for morning round",
    "criticality": "high",
    "mapping_source": "service-catalog"
  }
}

Note mapping_source: the clinical importance of this interface was looked up, not guessed.

Walkthrough: from weak signals to a verified fix

Here is the utility's own example, followed step by step, in the hospital setting from the opening.

  1. Three weak signals arrive (slow interface, more retries, growing queue) and are normalised with their source evidence kept.
  2. The orchestrator sees they share a dependency chain and routes them to the platform and application agents, while the business-impact agent links the chain to the morning ward round.
  3. Retrieval pulls the current runbook for this failure and the most similar past incidents.
  4. The agents produce a hypothesis, not a summary: the likely failure mode, the evidence for it, what else is affected, and the documented options. Weak or contradictory evidence is stated, not smoothed over.
  5. The runbook allows two steps. Extra diagnostics are low risk and run automatically. Restarting the interface is mapped to human approval, and the remediation agent cannot downgrade that because it feels sure.
  6. The on-call engineer approves. The gateway checks the approval reference and policy again at the moment of execution, uses an identity that can restart only that one service, and captures the response.
  7. Success is measured on the patient side of the system. The workflow checks that latency, retries and the queue actually recovered, and escalates if they did not.

"API returned 200" is not the same as "the system is healthy."

Risk lives in configuration, not in the prompt

Every action in the catalog belongs to a risk class, and the class is decided by people in advance, never by the model about its own proposal.

Risk classUtility examplesEquivalents in hospital ITRule
LowRead-only diagnostics, health checks, extra telemetryReading interface queues and logs, running test messagesMay run automatically, fully logged
MediumRestarting an approved stateless service, scaling within limitsRestarting the results interface, clearing a stuck print queueApproval according to policy
HighConfiguration changes, failover, sensitive data operationsFailing over the patient record system, changing access rulesExplicit approval, two people where policy requires

Each catalog entry is concrete enough to test without an LLM: the tool, the environment, the allowed scope, the risk class, the approval type, a timeout, the conditions that must hold before and after, and a link to the rollback. The agent can fill in parameters such as which service, but policy decides whether those parameters fall inside the allowed envelope.

Prediction is advice, not permission

The platform also predicts: it looks for combinations of weak signals that have historically come before a failure, and reports what may fail, when, how, with what confidence and with what business impact. The design decision that matters is what prediction is not allowed to do.

A confidence score is not an authorization token.

Predictions enter exactly the same governed path as real incidents. The article is also honest that false positives are expected, and that overselling prediction is one of the fastest ways to lose operators' trust. This is a familiar failure mode in operations: a team that is repeatedly paged for nothing soon stops reading the alerts.

The autonomy ladder

Here the full article is at its most practical. Autonomy expanded in phases, one action class at a time. No class moved up because the model seemed confident. Each step needed sign-off from the system owner and change-management reviewers, based on specific evidence:

StepAction classHow it runsEvidence needed to move up
1EverythingRead-only: observe, correlate, predict, recommendOperators consistently accept recommendations, and rejection reasons have been reviewed and fixed
2Diagnostics and health checksFrom approval-only to automaticNo diagnostic ran outside its scope, and evidence records are complete
3Reversible changes such as stateless restartsIntroduced as approval-onlyPost-action checks confirm recovery, and rollback has been tested
4Selected low-impact reversible changesFrom approval-only to automatic, within set scope and time windowsApproved consistently without edits, recovery confirmed, rollback works, no policy denials or out-of-scope requests
5High-risk changes such as configuration and failoverAlways explicit approvalNever eligible for automation

Two details are worth noting. First, the evidence for each step is operational, not a model benchmark: acceptance rates, complete audit records, tested rollbacks. Second, step 5 is a permanent ceiling, decided up front, which removes a whole category of arguments later.

And because each step is just a mode on a catalog entry, revoking autonomy is a configuration change, not a redeployment. If an automated restart ever misbehaves, that single action class drops back to approval-only within minutes while everything else keeps running. It is the same principle behind a parallel run and the draft, don't send approach to AI output: earn trust in the open before letting anything act alone.

In code: the gate is deliberately boring

The policy gate should be the least interesting code in the system, with no model call anywhere near it. Here is the core of one in TypeScript:

type Mode = "auto" | "approval" | "dual-approval" | "disabled";

type CatalogEntry = {
  actionId: string;            // e.g. "restart-service"
  environment: "prod" | "staging";
  allowedTargets: string[];
  risk: "low" | "medium" | "high"; // set by people, never by the model
  mode: Mode;                  // change this to grant or revoke autonomy
  approvalTtlMinutes: number;
};

type Proposal = { actionId: string; environment: string; target: string; reason: string };
type Approval = { approvers: string[]; approvedAt: Date };

type Decision =
  | { outcome: "execute"; entry: CatalogEntry }
  | { outcome: "needs-approval"; approvers: number }
  | { outcome: "deny"; why: string };

export function decide(catalog: CatalogEntry[], p: Proposal, approval?: Approval, now = new Date()): Decision {
  const entry = catalog.find((e) => e.actionId === p.actionId && e.environment === p.environment);
  if (!entry) return { outcome: "deny", why: "Not in the catalog" };          // deny by default
  if (entry.mode === "disabled") return { outcome: "deny", why: "Action switched off" };
  if (!entry.allowedTargets.includes(p.target)) return { outcome: "deny", why: "Target out of scope" };
  if (entry.mode === "auto") return { outcome: "execute", entry };

  const needed = entry.mode === "dual-approval" ? 2 : 1;
  const fresh = approval && (now.getTime() - approval.approvedAt.getTime()) / 60_000 <= entry.approvalTtlMinutes;
  const enough = approval && new Set(approval.approvers).size >= needed;

  return fresh && enough ? { outcome: "execute", entry } : { outcome: "needs-approval", approvers: needed };
}

What it leaves to the gateway: minting a short-lived credential for that one target, validating arguments against the tool's schema, calling the tool, checking health afterwards and writing the audit record. You do not have to build all of that yourself. The utility's reference implementation uses Amazon Bedrock AgentCore, whose Policy feature compiles rules to Cedar and enforces them at the gateway rather than in agent code, with the Model Context Protocol describing the tools. The article lists Azure API Management and Google's Agent Gateway as the equivalent doors on other clouds.

The article is careful to add a caveat: typed tools and a strict gate limit what a manipulated agent can do, but they do not stop it being manipulated. Prompt injection still needs its own layers, from treating everything the agent ingests as untrusted data to content screening at the gateway. The OWASP Top 10 for LLM Applications lists prompt injection and "excessive agency" as separate risks for good reason.

Evidence, not a compliance badge

Because every action passes through one door, the platform can rebuild the whole story of any change: the signal that started it, the evidence retrieved, the recommendation, the policy applied, the human decision, the operation, the identity used and the verified result.

The article is precise about what that does and does not achieve. The utility maps its evidence to the NERC CIP standards that apply to its in-scope grid systems, for change management, access, protected information and incident response. But not every system a utility runs is in CIP scope, and "the platform itself does not create compliance." The rule is to map evidence to whatever regime governs each specific system.

For a hospital, that means mapping the same records to clinical safety, data protection and change-control obligations system by system, rather than declaring the whole estate "AI governed". The article also makes a point that matters even more in healthcare: minimise what the agents ingest. Logs and tickets are full of personal and health data the agent does not need to diagnose a slow interface. Classify and filter it before it ever reaches retrieval.

The agents themselves are production software too: versioned prompts and agent definitions, reviewed changes, evaluation on past incidents before release, and the ability to roll back.

Beyond utilities

DomainWhat the agents doCan run automaticallyAlways needs a person
Hospital IT and clinical data feedsCorrelate interface, queue and data-feed problems against runbooksDiagnostics, test messages, health checksRestarting clinical interfaces, failing over patient records
Data retention and compliance jobsPrepare a redaction run, check its scope, assemble the evidenceDry runs and reportsExecuting redaction against production data
Enterprise and B2B platformsTriage tickets, spot failing integrations, draft fixesRead-only checks, scaling within limitsConfiguration changes, anything touching customer data or billing

Questions to ask before giving an agent production access

Whether you are evaluating an agentic operations tool or building one, these questions expose most of the weak points quickly:

  1. Where are the production credentials? If the answer is "the agent has them", stop there.
  2. Who decides an action's risk class? It should be a versioned catalog that people review, not the model.
  3. Is there exactly one path to production? Can anything call a target system without going through the gateway?
  4. Are approvals checked again at execution time, or does the gateway trust that an approval happened upstream?
  5. How is success measured? "The command ran" is not an answer. "The service recovered" is.
  6. What evidence moves an action up the ladder, and who signs it off?
  7. How fast can you revoke autonomy for one action without redeploying anything?

The article closes with the best one-line summary of the whole approach:

The goal is not maximum autonomy, but evidence-based autonomy that can be safely governed and revoked.

In healthcare, finance and every other system where mistakes have real victims, that is the only kind of autonomy worth building.

Further reading

  1. Governed Autonomous Intelligence for Mission-Critical Utility, Purnang Borah, InfoQ
  2. Policy in Amazon Bedrock AgentCore: core concepts, AWS
  3. Model Context Protocol, Anthropic and contributors
  4. OWASP Top 10 for LLM Applications 2025, OWASP
  5. CIP-010-4: Configuration Change Management, NERC
  6. AI Risk Management Framework, NIST
  7. ISO/IEC 42001:2023, AI management systems, ISO

Related case study

A Platform of 20 Task-Specific AI Agents for Long-Tail Knowledge Work