Can You Trust AI to Resolve Production Incidents?

Not blindly, and you shouldn't have to. How much you can trust AI in incident response depends far less on the model than on the architecture around it: read-only by default, findings that link to evidence you can check yourself, write actions behind human approval, and an audit trail covering all of it.

The word "resolve" in the title is doing a lot of work, so let's split it up front. An agent can trigger and run an investigation on its own. Remediation happens with human approval.

What is the objection really about?

Liability and autonomy. Hardly anyone still argues that a model can't read logs or correlate metrics. The distrust starts where the agent acts. Who answers for the action it took? Who decided an incident was over while customers were still affected? If nobody can reconstruct why the system did what it did, you have a problem that no accuracy benchmark will fix.

The surveys show the split clearly. In PagerDuty's AI Resilience Survey (September 2025), 81% of executives said they trust AI agents to take action on the company's behalf during a crisis. PagerDuty's 2026 State of AI-First Operations report (March 2026) then asked about expected outcomes, and 59% of IT decision-makers expected AI to improve downtime and recovery performance by more than 20%, while only 37% of developers agreed. Two surveys with two different questions, but a consistent pattern. The closer someone sits to production, the less they trust the agent.

We think the developers have it right. An unsupervised text generator that can close incidents and change production is a bad idea, and better models don't make it a good one. But that describes a specific design: full autonomy with no accountability. Nobody is forcing you to build or buy that design.

If you run infrastructure in the public sector or a regulated industry, auditability and clear accountability are procurement requirements anyway. The most useful way to read the objection is as a spec.

Will AI cause outages?

Yes. It already does. In the same PagerDuty survey from September 2025, 84% of companies reported at least one AI-related outage. Gartner expects worse for the systems that matter most, predicting that by 2028, misconfigured AI in cyber-physical systems will shut down national critical infrastructure in a G20 country (Gartner press release, February 2026).

What's striking is that nobody is stopping. The Catchpoint SRE Report 2026 (January 2026) found that more than half of organizations plan to run agentic AI in production within twelve months, while only 13% feel very or extremely confident in their ability to assess and monitor the reliability of AI components. Adoption is running ahead of trust. The analysts expect governance to close that gap, with Gartner projecting that by 2029, 70% of organizations will require explainable AI for agentic SRE decisions, up from just 5% today (Gartner, Cool Vendors in AI for SRE and Observability, October 2025).

The consequence for buyers is simple. Be suspicious of any vendor, us included, who promises their agent won't cause problems, because nobody building on language models can make that promise honestly. The promises worth evaluating are about blast radius, and about who stays in charge of anything that changes production.

What does trustworthy architecture look like?

This is the checklist we'd apply to any AI incident-response system, including our own:

• Read-only by default. Investigation runs on read credentials; nothing restarts, scales, or reconfigures. An agent that can't write can't cause an outage while diagnosing one.
• Investigation separated from remediation. Different phases, different privileges. A wrong hypothesis can't turn directly into a wrong action.
• Evidence-linked findings. Every claim links to the query, log line, or metric it came from, so a fluent-but-wrong analysis has nowhere to hide.
• Writes behind human approval. A named human approves each mutation, and the approval is recorded. That's the answer to the liability question from the top of this post: someone is always accountable, and you can show who.
• Containment outside the agent. Hard loop caps, operator timeouts, and an abort switch that doesn't ask the agent first, because a runaway loop shouldn't be stopped by the same judgment that caused it.

None of this is hypothetical for us. Hyground runs read-only by default with a full audit trail, and when an alert fires, the investigation starts on its own and ends in a report where every query and reasoning step is visible. The OWASP Top 10 for Agentic Applications for 2026, released in December 2025, puts names on the failure modes this architecture defends against. We've mapped ourselves against the agentic list and the LLM Top 10, including where our defenses fall short.

Gated autonomy is the industry norm

A fair suspicion at this point: maybe "human approval required" is what vendors say while their agent isn't good enough yet, and the guardrails quietly come off later.

What we actually see is that the teams furthest ahead run the tightest gates. On the Google SRE Prodcast (July 2025), Google engineer Swapnil Haria described the agents his team builds: "we don't allow them to make any kind of world modification," and for anything that writes, "we try to get human permission before it does anything." STRATUS (arXiv:2506.02009), a multi-agent SRE system from UIUC and IBM researchers accepted at NeurIPS 2025, formalizes a safety property called Transactional No-Regression, which roughly means agents only take actions the system can roll back. STRIDE (arXiv:2512.02228, December 2025, no relation to the threat-modeling mnemonic) treats full autonomy as something a task has to earn, and recommends the least autonomous setup that handles the job.

If anything, the relationship runs the other way around. The more capable the agent, the more valuable the gate, because a capable agent finds more things it could do.

Hallucinations don't go away

They also matter more here than in most applications, because an on-call engineer at 3 a.m. is in a bad position to spot a well-written wrong answer. Microsoft researchers built PACE-LM (2023), a calibrated-confidence system for LLM-based root cause analysis, precisely because a model's incident analysis can be what the paper calls a "well-disguised hallucination". Confident, fluent prose over a wrong conclusion. Probabilistic failure is also really awkward to debug, since running the same investigation twice can produce two different answers. Sampling is part of how these models work.

The practical rule for anyone using these tools: verify the citation, not the prose. Reading the analysis tells you little, because fluency is what the model is best at. Checking whether the log line it cites exists, and says what the agent claims, takes seconds. That's why evidence-linked findings are on the checklist above, and it's fair to say the whole industry is still working on making them reliable, ourselves included.

What can you safely hand over today?

Triage, context gathering, hypothesis generation, and the investigation write-up. Judgment calls and write actions stay with humans. The line between the two is easier to draw than it sounds. Would you let a junior engineer do it under supervision, with no consequences for the team if they got it wrong? Then an agent can do it with explicit approval. Does it need judgment? Then a human should keep the agency.

In practice the split is comfortable. The agent covers the work it can do best: querying every relevant source at 3 a.m., correlating deploy history with error rates, writing up what it found and how. The human keeps the work where accountability matters, like declaring the incident over, approving the fix, and owning the call. Teammate design, not replacement design.

Whether these agents deliver enough value to be worth running is a separate question, and we've written an honest assessment of whether AI SRE agents are useful or just hype. On trust, our answer hasn't moved since the first line of this post. You shouldn't have to extend blind trust to an AI agent, and with the right architecture you never do. The agent shows its evidence, a human approves anything that changes production, and every step is on the record. That's a system you can start using before you fully trust it.