Who this questionnaire is for AI builders, ML/engineering teams, knowledge/platform owners, and governance stakeholders shipping retrieval-augmented generation (RAG) or evidence-based assistants.
What it assesses Whether your system can reliably ground outputs in permitted sources: retrieval quality, source allowlists, freshness/metadata controls, citation correctness, contradiction handling, refusal/partial answering, and exportable audit trails.
How it helps Prevents the two classic RAG failures: confident nonsense and unclear provenance. Results show whether you can (1) retrieve the right evidence, (2) cite it correctly, and (3) refuse when evidence is missing — with logs that stand up to audit and incident review.
This assessment checks whether your Retrieval-Augmented Generation system is governable: sources are controlled, retrieval is logged, answers are consistent with evidence, and failures are detectable.
Status:Not scoredCoverage:0%Score:—Decision:—
Your role (optional)
— Select role —
Engineering / Platform
ML / Applied AI
Security
Risk / Legal / Compliance
Product / Operations
Research
Other
Company or project identifier (optional)
Your score
0
out of 45
Answered 0/15
Posture
—
Answer the questions to see your posture.
Section A — Source Control
1) Do you have an explicit allowed-source policy (domains/corpora) that the system enforces?
No — open web / undefined sources
Partial — documented but not enforced
Mostly — enforced in many paths
Enforced — allowlist + audits + exceptions logged
2) Are document owners, rights, and retention rules known for every corpus?
No
Partial
Mostly
Enforced — mapped + reviewed
3) Can you control freshness (update cadence, re-indexing rules, and stale-doc handling)?
No — unknown freshness
Partial — ad-hoc updates
Mostly — cadence exists
Enforced — freshness policy + monitoring
4) Are ingestion steps deterministic and versioned (chunking, cleaning, metadata, embeddings model/version)?
No
Partial
Mostly
Enforced — versioned pipeline + change control
5) Can you prevent retrieval from unsafe corpora (PII, restricted, unreviewed) by design?
No
Partial — policy only
Mostly — filtered in many paths
Enforced — tagging + access controls + audits
Section B — Retrieval Quality & Evidence
6) Do you log retrieval evidence per response (top-k IDs, scores, timestamps, corpus/version)?
No logs
Partial — some logs
Mostly — logged for most responses
Enforced — always logged + exportable schema
7) Do you measure retrieval performance (coverage, precision, empty-retrieval rate, drift) over time?
No
Occasional checks
Regular metrics
Enforced — monitored + alert ownership
8) Do you handle empty/low-confidence retrieval safely (refuse, ask clarifying questions, or return partial with sources)?
No — still answers
Sometimes
Usually
Enforced — deterministic behaviour + tests
9) Do you have defenses against prompt injection via retrieved content (filters, tool restrictions, policy gates)?
No
Partial
Mostly
Enforced — tests + monitoring
10) Do you provide user-facing citations or evidence trails that are auditable (not cosmetic)?
No
Cosmetic citations
Mostly auditable
Enforced — citations map to logged evidence
Section C — Consistency, Testing & Operations
11) Do you run groundedness checks (answer must match retrieved evidence) and block contradictions?
No
Partial — manual reviews
Mostly — some automated checks
Enforced — automated + logged + tested
12) Do you maintain a regression suite for RAG failure paths (stale docs, empty retrieval, conflicting sources)?
No
Ad-hoc
Regular tests exist
Enforced — CI gates + coverage tracked
13) Can you reconstruct “what happened” after an incident (inputs, retrieval, versions, tool actions)?
No
Partial reconstruction
Mostly reconstructable
Enforced — deterministic reconstruction
14) Do you have alert ownership for retrieval outages, index drift, and quality drops?
No
Some alerts
Owned alerts
Enforced — SLOs + on-call + drills
15) Are changes to corpora, prompts, embedding models, and retrieval settings controlled and approved?
No
Some tracking
Mostly controlled
Enforced — change logs + approvals + rollback
Tip: A RAG system is only “trustworthy” if you can show the evidence and prove the answer is consistent with it.
Posture
—
—
Decision
—
—
Recommended next steps
Top risks
Built for grounded systems: controlled sources, logged retrieval evidence, consistency checks, and operational reconstruction.
(function(){
function init(){
const root = document.getElementById(‘oyez-rag-readiness’);
if(!root) return false;
if(root.__oyezInit) return true;
root.__oyezInit = true;
const TOTAL=15;
const bar = root.querySelector(‘#oyezRagBar’);
const statusEl = root.querySelector(‘#oyezRagStatus’);
const coverageEl = root.querySelector(‘#oyezRagCoverage’);
const scorePillEl = root.querySelector(‘#oyezRagScorePill’);
const decisionPillEl = root.querySelector(‘#oyezRagDecision’);
const scoreBigEl = root.querySelector(‘#oyezRagScoreBig’);
const answeredEl = root.querySelector(‘#oyezRagAnswered’);
const postureTopEl = root.querySelector(‘#oyezRagPosture’);
const postureTopNoteEl = root.querySelector(‘#oyezRagPostureNote’);
const roleEl = root.querySelector(‘#oyezRagRole’);
const orgEl = root.querySelector(‘#oyezRagOrg’);
const calcBtn = root.querySelector(‘#oyezRagCalc’);
const copyBtn = root.querySelector(‘#oyezRagCopy’);
const emailBtn = root.querySelector(‘#oyezRagEmail’);
const resetBtn = root.querySelector(‘#oyezRagReset’);
const results = root.querySelector(‘#oyezRagResults’);
const postureOut = root.querySelector(‘#oyezRagPostureOut’);
const postureNoteOut = root.querySelector(‘#oyezRagPostureNoteOut’);
const decisionOut = root.querySelector(‘#oyezRagDecisionOut’);
const decisionDesc = root.querySelector(‘#oyezRagDecisionDesc’);
const nextEl = root.querySelector(‘#oyezRagNext’);
const risksEl = root.querySelector(‘#oyezRagRisks’);
function updateSelected(){
root.querySelectorAll(‘.opt’).forEach(opt=>{
const r = opt.querySelector(‘input[type=”radio”]’);
opt.classList.toggle(‘selected’, !!(r && r.checked));
});
}
function answeredCount(){
let n=0;
for(let i=1;i<=TOTAL;i++){
if(root.querySelector(`input[name="q${i}r"]:checked`)) n++;
}
return n;
}
function score(){
let s=0;
for(let i=1;i<=TOTAL;i++){
const picked = root.querySelector(`input[name="q${i}r"]:checked`);
if(picked) s += parseInt(picked.value,10);
}
return s;
}
function postureFromScore(s){
if(s<=15) return 'Ungrounded';
if(s<=30) return 'Partly grounded';
return 'Grounded';
}
function pack(posture){
const packs={
"Ungrounded":{
note:"High concern. The system can produce confident answers without trustworthy evidence, and you may not be able to reconstruct failures.",
decision:"Do not scale (add enforceable grounding)",
decisionDesc:"Treat this as unsafe-to-scale until source controls, retrieval logging, and consistency gates are enforced and tested.",
risks:[
"Answers can be produced without reliable retrieval (empty/low-confidence retrieval).",
"Citations may be cosmetic and not linked to logged evidence.",
"Injection or stale docs can silently steer outputs."
],
next:[
"Enforce allowed sources and block unsafe corpora by design.",
"Log retrieval evidence per response (top-k IDs, scores, timestamps, corpus/version).",
"Add groundedness checks that refuse/return partial when evidence is weak or contradictory."
]
},
"Partly grounded":{
note:"Workable posture. Main risk: enforcement gaps and weak regression testing allow silent drift and inconsistent evidence trails.",
decision:"Proceed with gates + regression suite",
decisionDesc:"You can proceed, but only with testable gates, failure-path regression coverage, and owned alerts for retrieval health.",
risks:[
"Controls exist but aren’t consistently enforced across all paths.",
"Monitoring exists without alert ownership or SLOs.",
"Change control for corpora/embeddings/prompts may be incomplete."
],
next:[
"Build a RAG regression suite (stale docs, conflicts, empty retrieval, injection, tool failure).",
"Separate retrieval metrics vs reasoning metrics and track both over time.",
"Add change logs + approvals for corpora, embedding versions, and retrieval settings."
]
},
"Grounded":{
note:"Positioned to scale responsibly. Main risk: governance becoming static while corpora, retrieval settings, and threat models evolve.",
decision:"Scale responsibly + continuous evaluation",
decisionDesc:"Scale is appropriate — keep evaluations continuous, run periodic control reviews, and keep evidence trails exportable by default.",
risks:[
"Corpus updates can change behaviour without explicit review.",
"Threats evolve (injection, poisoning) faster than tests unless updated.",
"Audit readiness degrades without retention/access reviews."
],
next:[
"Operationalise continuous evaluation before/after changes and at fixed intervals.",
"Run quarterly reviews of source allowlists, retention, and access control boundaries.",
"Keep evidence trails exportable and tied to user-facing citations."
]
}
};
return packs[posture];
}
function compute(){
updateSelected();
const ans = answeredCount();
const cov = Math.round((ans/TOTAL)*100);
const s = score();
bar.style.width = cov + '%';
statusEl.textContent = ans===0 ? 'Not scored' : (ans{ const li=document.createElement(‘li’); li.textContent=x; nextEl.appendChild(li); });
risksEl.innerHTML=”;
p.risks.forEach(x=>{ const li=document.createElement(‘li’); li.textContent=x; risksEl.appendChild(li); });
results.style.display=’block’;
copyBtn.disabled=false; emailBtn.disabled=false;
return {ans,cov,s,posture,pack:p,role:roleEl.value||’—’,org:orgEl.value||’—’};
}
function summaryText(state){
const lines=[];
lines.push(‘OYEZ — AI RAG & Grounding Readiness’);
lines.push(‘———————————-‘);
lines.push(`Role: ${state.role}`);
lines.push(`Company / Project: ${state.org}`);
lines.push(”);
lines.push(`Status: ${state.ans===TOTAL ? ‘Completed’ : ‘In progress’}`);
lines.push(`Coverage: ${state.cov}% (${state.ans}/${TOTAL})`);
lines.push(`Score: ${state.s}/45`);
lines.push(`Posture: ${state.posture}`);
lines.push(`Decision: ${state.pack.decision}`);
lines.push(”);
lines.push(state.pack.note);
lines.push(”);
lines.push(‘Top risks:’);
state.pack.risks.forEach((r,i)=>lines.push(`${i+1}. ${r}`));
lines.push(”);
lines.push(‘Recommended next steps:’);
state.pack.next.forEach((n,i)=>lines.push(`${i+1}. ${n}`));
return lines.join(‘\n’);
}
root.addEventListener(‘change’, (e)=>{
if(e.target && e.target.matches(‘input[type=”radio”]’)) compute();
});
calcBtn.addEventListener(‘click’, (e)=>{ e.preventDefault(); compute(); });
copyBtn.addEventListener(‘click’, async (e)=>{
e.preventDefault();
const state = compute();
if(!state) return;
const text = summaryText(state);
try{
await navigator.clipboard.writeText(text);
copyBtn.textContent=’Copied’;
setTimeout(()=>copyBtn.textContent=’Copy Results’, 900);
}catch(err){
const ta=document.createElement(‘textarea’);
ta.value=text; document.body.appendChild(ta);
ta.select(); document.execCommand(‘copy’);
document.body.removeChild(ta);
copyBtn.textContent=’Copied’;
setTimeout(()=>copyBtn.textContent=’Copy Results’, 900);
}
});
emailBtn.addEventListener(‘click’, (e)=>{
e.preventDefault();
const state = compute();
if(!state) return;
const subject = encodeURIComponent(`Oyez — RAG Readiness (${state.posture}, ${state.s}/45)`);
const body = encodeURIComponent(summaryText(state));
window.location.href = `mailto:?subject=${subject}&body=${body}`;
});
resetBtn.addEventListener(‘click’, (e)=>{
e.preventDefault();
root.querySelectorAll(‘input[type=”radio”]’).forEach(r=>r.checked=false);
roleEl.value=”; orgEl.value=”;
compute();
root.scrollIntoView({behavior:’smooth’, block:’start’});
});
compute();
return true;
}
if(!init()){
let tries=0;
const t=setInterval(()=>{
tries++;
if(init() || tries>=50) clearInterval(t);
}, 100);
}
})();
