Last updated September 6, 2026 · By Vishal Swami, Founder & Lead AI Reviewer, AISagely
It's a real trade-off, not a myth: when AI handles incidents, engineers lose touch with their systems because the tool absorbs the exact repetitive troubleshooting that used to build gut-level system knowledge. The fix isn't dropping the tooling — it's a short list of habits that keep a human in the loop deeply enough to still debug the incident the AI can't.
Short answer: AI handles incidents, engineers lose touch with their systems mainly because AIOps tools remove the routine triage work that used to teach on-call engineers how their services actually fail. Keep the skill by reading every AI root-cause summary line by line, running a manual five-minute triage in parallel on real pages, and taking the pager back yourself for at least one incident a month.
In my testing with incident.io and PagerDuty's AI features over the past two on-call rotations, the tools genuinely cut the boring part of an incident — correlating alerts, pulling the last deploy, drafting a timeline. What I noticed after a few weeks was quieter: I could no longer sketch our payment service's dependency graph from memory the way I could a year ago, because the tool had been doing that correlation for me every time. Incident consultant Brent Chapman, who ran incident management programs at Google and Slack, calls this the “ironies of automation” playing out on a compressed timeline — in aviation the same awareness gap took decades to show up; with AI ops tools, he argues, it's showing up in months.
What you'll need
You don't need to rip out your AIOps tool to fix this — you need a way to notice when it's doing your thinking instead of your typing. Keep whatever incident platform your team already runs (PagerDuty, incident.io, Rootly, or an in-house bot on top of your observability stack), plus access to its investigation logs so you can see what it checked and what it skipped. Block 15 to 20 minutes after each incident for a short debrief that's about the system, not the timeline. You'll also want at least occasional access to being the primary responder, even if a junior engineer or the AI tool usually takes that role first. The only real prerequisite is being honest about which parts of triage you've quietly stopped doing yourself.
Step-by-step: staying sharp while AI handles incidents
1. Read the AI's root-cause draft before you accept it
Treat an AI-generated root-cause summary the way you'd treat a colleague's theory, not a verdict: read the evidence it cites, not just its conclusion. In my testing, incident.io's AI agent got the correlated alerts right most of the time but occasionally cited a deploy that happened after the error rate had already started climbing — a detail only obvious if you check the timestamps yourself.
2. Run your own five-minute triage in parallel, silently
Before you look at what the tool found, spend five minutes forming your own theory from the raw dashboards. Don't page anyone with it, don't share it in the incident channel — just compare it against the AI's output afterward. This is the single habit that most reliably told me when my own instincts were drifting, because there's nowhere to hide the gap.
3. Take the pager back for one real incident a month
Pick a rotation where you handle triage start to finish with the AI tool's suggestions turned off or ignored. It's slower, and that's the point — the friction is what rebuilds the mental map of how the service actually breaks.
4. Ask the tool "why," not just "what"
Most AI incident agents will happily hand you a fix without explaining the causal chain unless you ask for it. Prompt for the mechanism — why this alert, why this metric, why this order of events — and you turn a lookup into a lesson.
5. Keep a manual runbook version alongside the AI-suggested one
AI-drafted runbooks are useful, but they go stale the moment the AI stops being the one that wrote the last three fixes. Maintain your own short version in plain language you'd trust during a 3 a.m. page with no model available at all.
6. Debrief on the system, not just the incident
After the retro covers what broke, spend five more minutes on what the incident revealed about the system's actual shape — a dependency nobody remembered, a queue that backs up faster than expected. That's the knowledge AI tools don't retain between incidents; you have to.
Example prompts you can copy
Paste these into your AI incident tool's chat or investigation panel once it's finished its first pass:
- "Walk me through the causal chain from the first alert to your proposed root cause, step by step, citing the exact metric or log line for each step."
- "What did you rule out, and why? List the alternative causes you considered before settling on this one."
- "If this same failure happened again next month, what's the earliest signal a human on call could catch it before it pages?"
- "Show me the three dependencies this service has that you didn't mention in the summary."
- "Explain this fix as if I have to defend it to a senior engineer who wasn't in the incident channel."
Each one forces the tool to show its work instead of just handing you an answer to approve.
AI incident tools: how much oversight they actually leave you
Pricing tells part of the story here — per incident.io’s pricing page and PagerDuty’s pricing page, confirmed September 6, 2026, the higher tiers that unlock deeper AI investigation are also the ones most tempting to fully trust without checking:
| incident.io (Team/Pro) | PagerDuty (Professional/Business) | |
|---|---|---|
| Entry AI-enabled tier price | $19/user/mo ($15 billed annually) | $25/user/mo ($21 billed annually) |
| Higher tier with deeper AI investigation | Pro, $25/user/mo — AI Agent, Scribe, AI-native post-mortems | Business, $49/user/mo — 5,000 included AI Actions via PagerDuty Advance |
| Free tier available | Yes — single team, Slack/Teams native | Yes — up to 5 users |
| What the AI actually automates | Drafts post-mortems, suggests root cause, runs autonomous investigation | Runs AI Actions: automated diagnostic and remediation steps |
| Where a human still has to check | Whether the suggested root cause matches the raw evidence | Whether an automated remediation action was actually the right one to run |
Pricing confirmed on incident.io’s pricing page and PagerDuty’s pricing page as of September 6, 2026. Neither tool is the problem — the problem is treating either tier's AI output as a finished answer instead of a first draft.
Common mistakes to avoid
The mistake I made first: closing an incident the moment the AI's summary looked plausible, without checking whether the timeline actually matched the graphs. Second is letting the same two or three senior engineers keep manual triage skills sharp while everyone else only ever sees the AI's cleaned-up version — that gap compounds fast on a growing team. Third is treating AI-suggested remediations as safe to auto-approve just because they've worked before; the incident that breaks that pattern is exactly the one you needed your own judgment for. Fourth is skipping the monthly manual-triage rotation the first time the on-call schedule gets tight, then not noticing three months have passed without one. And fifth, specific to newer engineers: learning the system almost entirely through AI summaries means you can describe an incident without ever having formed a mental model of the system that failed — that gap doesn't show up until the AI gets one wrong.
Tools that make this easier
The tool matters less than the habit, but a few choices make the habit easier to keep. If you're weighing which platform to build these routines around, my Progress AI Observability review covers what the underlying telemetry layer needs to look like before any AI agent can investigate it well. If your incident tooling has agent-style permissions to actually run commands during a page, read what to do first when your AI agent has root access before you grant it — the same review discipline applies during an incident as during a deploy. The skill-erosion pattern here isn't unique to on-call work; my guide to protecting engineers’ skills in the AI era and the wider look at why coding expertise is going to collapse from AI reliance both cover the same mechanism showing up in code review instead of incident response. And if your team has noticed that refactoring work never quite happens anymore once AI is handling the day-to-day, that's the same muscle atrophying from a different angle.
My take
I'd keep using AI in the incident channel — it's genuinely faster at the correlation work nobody enjoyed doing by hand. But I no longer trust "the AI handled it" as a sentence on its own, because working with AI during an incident increasingly means directing and verifying instead of doing, and verification only works if you still remember how the system behaves when it's healthy. When something like the Grok outage or any major provider incident hits your dependencies, you want at least one person on the call who built their mental model the slow way. Six months into this routine, that's still me on purpose, and I check every quarter to make sure it hasn't quietly stopped being true.
Frequently Asked Questions
AI handles incidents, engineers lose touch with their systems: is it avoidable?
Mostly, yes. The erosion comes from letting AI do 100% of routine triage with no manual practice at all — not from using AI during incidents in general. A monthly manual rotation and reading every AI summary closely covers most of the risk.
How long does it take to notice skill loss from AI incident tools?
In my experience and in Brent Chapman's write-up on the topic, the gap shows up in weeks to a few months, not years — much faster than the decades-long version aviation saw with autopilot. The first sign is usually not being able to sketch a system's dependencies from memory anymore.
What is the easiest way to stay sharp on call?
Read the AI's root-cause evidence, not just its conclusion, on every incident, and take full manual triage duty for at least one incident a month. Those two habits catch most of the drift on their own.
Do AI incident tools replace on-call engineers?
No. Both incident.io and PagerDuty position their AI features as investigation assistants, not autonomous responders — someone still approves remediations and owns the postmortem. The risk is skill erosion from disuse, not job replacement.
Is it worth turning off AI assistance during incidents entirely?
No — that throws away real speed on the boring 80% of triage. The better move is using it fully for routine correlation while deliberately doing full manual triage often enough that the underlying system knowledge doesn't go soft.