Who this questionnaire is for
Operational teams, AI platform owners, incident response leads, SREs, and governance teams reviewing live systems or post-incident behaviour.
What it assesses
Operational risk in deployed AI systems — including drift detection, incident handling, rollback authority, escalation paths, evidence retention, and post-incident learning.
How it helps
This questionnaire functions as a live operational review tool. It supports decisions to continue, pause, rollback, or re-approve AI systems after changes, incidents, or observed drift. Outputs are designed to be recorded, shared, and referenced as part of operational governance.
Best used when
- Reviewing incidents or near-misses
- Assessing readiness after model or data changes
- Determining whether systems can safely remain in production
AI Change, Drift & Incident Review
Scores automatically as you click. The top fields are optional — only the questions affect scoring.
Status: Not scored
Coverage: 0%
Score: —
Decision: —
Optional fields: these improve your governance record, but are not required to score.
1) Change Classification & Re-approval Triggers
Is there a clear classification of the change (minor vs material), with a documented decision and owner?
Change gatingMaterial changes should trigger re-evaluation and possibly re-approval.
YesChange class, owner, and rationale documented.
PartialInformal classification or missing rationale/owner.
NoNo clear classification or governance gating.
Are re-approval triggers defined (and actually used) for the system?
Release controlExamples: new model, new retrieval domains, new tools/actions, new user groups.
YesTriggers exist and were applied for this change.
PartialTriggers exist but inconsistently applied.
NoNo defined re-approval triggers.
2) Drift Monitoring & Post-Deployment Signals
Do you measure drift with concrete signals (quality, safety, grounding, latency/cost) on an ongoing basis?
MonitoringDrift includes retrieval/policy/prompt drift and user behaviour drift — not only model drift.
YesDashboards/alerts exist and are reviewed regularly.
PartialSome signals tracked; cadence inconsistent.
NoNo meaningful drift monitoring in production.
Are thresholds defined for “stop / rollback / investigate” and connected to operational actions?
ActionabilityMonitoring without action thresholds is not governance.
YesThresholds trigger playbooks/alerts with owners.
PartialSome thresholds; actions/owners unclear.
NoNo operational thresholds for intervention.
3) Incident Response & Escalation
Is there an incident playbook specific to this AI system, including escalation paths and authority?
IRInclude who can pause the system and how evidence is preserved.
YesSystem-specific playbook + escalation paths exist.
PartialGeneral playbook exists; AI specifics incomplete.
NoNo usable incident playbook for this system.
After incidents/near-misses, do you complete a post-incident review with tracked follow-through?
Learning loopA governance system that cannot learn from incidents will repeat them.
YesReviews are standard; actions tracked to completion.
PartialReviews happen; follow-through inconsistent.
NoNo consistent post-incident review process.
4) Evidence, Logging & Auditability
Can you reconstruct what the system did for a given event (inputs, context, outputs, tool actions)?
EvidenceLogging should be privacy-aware and access controlled.
YesEvent reconstruction is possible with logs + access controls.
PartialSome logs exist; reconstruction incomplete.
NoNo reliable reconstruction capability.
If the system uses retrieval (RAG), do you capture retrieval evidence (sources, top-k, scores) per response?
RAGGoverned RAG requires evidence of what was retrieved.
YesRetrieval evidence is captured and reviewable.
PartialSome retrieval logging; missing detail/checks.
NoNo retrieval evidence in practice.
5) Access, Permissions & Operational Boundaries
Are access permissions and tool/action boundaries documented and enforced (least privilege)?
PermissionsOver-permissioning is a common governance failure.
YesLeast privilege enforced; access reviewed periodically.
PartialSome controls; reviews/boundaries incomplete.
NoPermissions/boundaries unclear or not enforced.
Are operational limits documented (rate limits, cost caps, latency SLOs) with escalation when exceeded?
Ops limitsOperational limits prevent silent failure modes.
YesLimits monitored with escalation paths.
PartialSome limits; monitoring/escalation incomplete.
NoNo operational limits or escalation.
6) Decision & Required Actions
Can you justify “continue / monitor / re-approve / pause / rollback” with documented owners?
DecisionA governance review should end in an explicit decision and ownership.
YesDecision documented with owners and deadlines.
PartialDecision implied; owners/deadlines unclear.
NoNo clear operational decision or ownership.
Are remediation actions tracked (ticketed) and verified before the next major release/change?
Follow-throughGovernance fails when remediation is not executed.
YesActions ticketed, assigned, and verified.
PartialTracked informally; verification inconsistent.
NoNo tracking/verification of remediation.
