Who this questionnaire is for Procurement, security, legal/compliance, risk, IT, and product leaders evaluating AI vendors, tools, model providers, or platforms.
What it assesses Whether a vendor can meet real operational and governance requirements: evidence/log exportability, auditability, access controls, data handling, incident support, evaluation discipline, failure-path behaviour, and contractual clarity on risks.
How it helps Stops “demo-driven buying.” Results translate requirements into concrete proof points (what the vendor must show, log, export, and support) so you don’t discover gaps after rollout — when failures are expensive and public.
Best used when
Selecting a vendor/model/provider
Renewing or expanding contracts
Introducing AI into regulated/high-stakes workflows
This assessment helps teams avoid demo-driven purchasing. It focuses on evidence exportability, controllability, evaluation discipline, and audit/incident readiness.
Status:Not scoredCoverage:0%Score:—Decision:—
Your role (optional)
— Select role —
Procurement / Finance
Engineering / Platform
ML / Applied AI
Security
Risk / Legal / Compliance
Product / Operations
Other
Company or project identifier (optional)
Your score
0
out of 45
Answered 0/15
Posture
—
Answer the questions to see your posture.
Section A — Evidence & Exportability
1) Can the tool export raw logs (inputs, retrieval context, tool calls, outputs) in a structured format?
No — no export
Partial — limited/export on request
Mostly — exportable logs exist
Enforced — exportable + documented schema
2) Can you reproduce results later (versioned model/prompt/data + timestamps)?
No
Partial — some versions tracked
Mostly — reproducible in most cases
Enforced — reliably reproducible
3) Are costs and rate limits predictable at scale (quotas, per-seat/per-token clarity, alerts)?
Unknown / unpredictable
Rough estimates only
Predictable with monitoring
Enforced — predictable + enforceable controls
4) Can you migrate away without losing critical data (configs, evals, embeddings, logs)?
No — hard lock-in
Painful / partial
Mostly possible
Enforced — designed for portability
5) Can the vendor support audits (retention, access logs, incident disclosures, evidence trail)?
No
Informal / unclear
Documented support exists
Enforced — contracted + demonstrated
Section B — Security & Control Surface
6) Are permissions least-privilege with roles, audit logs, and periodic reviews?
No controls
Basic roles
Roles + audit logs
Enforced — RBAC + reviews + exportable audits
7) Can you enforce groundedness (allowlists, citations, refusal gates) inside the product?
No
Partial configuration
Strong configuration
Enforced — testable + logged gates
8) Is security posture clear (data handling, training use, retention, encryption) and kept current?
No clarity
Basic statements
Detailed documentation
Enforced — contracted + verified
9) Can you define “high-stakes” workflows with human review and escalation paths?
No
Informal handling
Supported workflows
Enforced — audited workflows
10) Can you tune behaviour (policies/configs) with versioning and rollback?
Black box
Limited tuning
Most knobs exist
Enforced — full control + rollback
Section C — Evaluation & Contract Discipline
11) Do you require failure-path demos (empty retrieval, contradictions, injection, tool errors) before purchase?
No
Occasionally
Usually
Enforced — acceptance criteria + sign-off
12) Can you run structured evaluations (not demos) and compare runs over time?
No
Informal
Structured evaluations exist
Enforced — regression tracking
13) Do contracts include incident expectations (SLAs, breach notification, support, postmortems)?
No
Minimal
Defined
Enforced — strong + tested in practice
14) Do contracts include evidence expectations (export logs, retention, audit support) rather than “trust us”?
No
Partial
Mostly — documented clauses
Enforced — contractual + measurable
15) Do you have an exit plan before signing (data export, replacement timeline, contingency)?
No
Rough idea
Documented plan
Enforced — contracted + rehearsed
Tip: If you can’t export evidence, you can’t govern or defend decisions later — and you can’t learn from incidents.
Posture
—
—
Decision
—
—
Recommended next steps
Top risks
Built for procurement discipline: exportable evidence, controllable configurations, failure-path demos, and contractual auditability.
(function(){
function init(){
const root = document.getElementById(‘oyez-vendor-readiness’);
if(!root) return false;
if(root.__oyezInit) return true;
root.__oyezInit = true;
const MAX=45, TOTAL=15;
const bar = root.querySelector(‘#oyezVendorBar’);
const statusEl = root.querySelector(‘#oyezVendorStatus’);
const coverageEl = root.querySelector(‘#oyezVendorCoverage’);
const scorePillEl = root.querySelector(‘#oyezVendorScorePill’);
const decisionPillEl = root.querySelector(‘#oyezVendorDecision’);
const scoreBigEl = root.querySelector(‘#oyezVendorScoreBig’);
const answeredEl = root.querySelector(‘#oyezVendorAnswered’);
const postureTopEl = root.querySelector(‘#oyezVendorPosture’);
const postureTopNoteEl = root.querySelector(‘#oyezVendorPostureNote’);
const roleEl = root.querySelector(‘#oyezVendorRole’);
const orgEl = root.querySelector(‘#oyezVendorOrg’);
const calcBtn = root.querySelector(‘#oyezVendorCalc’);
const copyBtn = root.querySelector(‘#oyezVendorCopy’);
const emailBtn = root.querySelector(‘#oyezVendorEmail’);
const resetBtn = root.querySelector(‘#oyezVendorReset’);
const results = root.querySelector(‘#oyezVendorResults’);
const postureOut = root.querySelector(‘#oyezVendorPostureOut’);
const postureNoteOut = root.querySelector(‘#oyezVendorPostureNoteOut’);
const decisionOut = root.querySelector(‘#oyezVendorDecisionOut’);
const decisionDesc = root.querySelector(‘#oyezVendorDecisionDesc’);
const nextEl = root.querySelector(‘#oyezVendorNext’);
const risksEl = root.querySelector(‘#oyezVendorRisks’);
function updateSelected(){
root.querySelectorAll(‘.opt’).forEach(opt=>{
const r = opt.querySelector(‘input[type=”radio”]’);
opt.classList.toggle(‘selected’, !!(r && r.checked));
});
}
function answeredCount(){
let n=0;
for(let i=1;i<=TOTAL;i++){
if(root.querySelector(`input[name="q${i}v"]:checked`)) n++;
}
return n;
}
function score(){
let s=0;
for(let i=1;i<=TOTAL;i++){
const picked = root.querySelector(`input[name="q${i}v"]:checked`);
if(picked) s += parseInt(picked.value,10);
}
return s;
}
function postureFromScore(s){
if(s<=15) return 'Demo-led';
if(s<=30) return 'Controlled';
return 'Governable';
}
function pack(posture){
const packs={
"Demo-led":{
note:"High risk of buying a black box you can’t govern later. Prioritise exportability, evidence, and failure-path discipline before signing.",
decision:"Do not buy (raise requirements)",
decisionDesc:"Treat this as a no-go until evidence export, auditability, and failure-path demos meet defined acceptance criteria.",
risks:[
"Demos substitute for evaluation; failure cases arrive after rollout.",
"Limited logs/exports prevent audit, debugging, and incident reconstruction.",
"Vendor lock-in accumulates via configs, embeddings, and non-portable evidence."
],
next:[
"Require exportable logs + a documented schema (inputs, retrieval, tool calls, outputs).",
"Run failure-path demos with acceptance criteria (empty retrieval, injection, tool errors).",
"Add contract requirements: retention, audit support, incident disclosures, and a clear exit plan."
]
},
"Controlled":{
note:"Workable posture. Main risk: enforcement gaps (auditability, gates, portability) that show up at scale.",
decision:"Buy with conditions (contract + gates)",
decisionDesc:"Proceed only if gaps are closed contractually and the product enforces evidence, controls, and exportability.",
risks:[
"Controls exist but aren’t consistently enforced across teams/vendors.",
"High-stakes workflows may be policy-only rather than enforced.",
"Portability may still be costly without a rehearsed exit plan."
],
next:[
"Make groundedness enforceable: allowlists, citations, refusal gates, audit logs.",
"Require structured evaluations + regression tracking as part of rollout.",
"Document and rehearse an exit plan before contract signature."
]
},
"Governable":{
note:"Strong procurement posture. Main risk: governance drifting as the vendor/product evolves.",
decision:"Buy / standardise (with quarterly review)",
decisionDesc:"Adopt as a standard only if you keep evaluation continuous and run periodic reviews of retention, access, and incident support.",
risks:[
"Vendor changes can silently shift behaviour without version pinning.",
"Audit readiness degrades without periodic retention/access reviews.",
"Evidence trails fragment across tools unless standardised."
],
next:[
"Standardise evidence schemas across vendors and internal systems.",
"Operationalise quarterly reviews: access, retention, incident drills, eval cadence.",
"Keep portability active: routine exports, comparison runs, contingency planning."
]
}
};
return packs[posture];
}
function compute(){
updateSelected();
const ans = answeredCount();
const cov = Math.round((ans/TOTAL)*100);
const s = score();
bar.style.width = cov + '%';
statusEl.textContent = ans===0 ? 'Not scored' : (ans{ const li=document.createElement(‘li’); li.textContent=x; nextEl.appendChild(li); });
risksEl.innerHTML=”;
p.risks.forEach(x=>{ const li=document.createElement(‘li’); li.textContent=x; risksEl.appendChild(li); });
results.style.display=’block’;
copyBtn.disabled=false; emailBtn.disabled=false;
return {ans,cov,s,posture,pack:p,role:roleEl.value||’—’,org:orgEl.value||’—’};
}
function summaryText(state){
const lines=[];
lines.push(‘OYEZ — AI Vendor & Procurement Readiness’);
lines.push(‘—————————————‘);
lines.push(`Role: ${state.role}`);
lines.push(`Company / Project: ${state.org}`);
lines.push(”);
lines.push(`Status: ${state.ans===TOTAL ? ‘Completed’ : ‘In progress’}`);
lines.push(`Coverage: ${state.cov}% (${state.ans}/${TOTAL})`);
lines.push(`Score: ${state.s}/45`);
lines.push(`Posture: ${state.posture}`);
lines.push(`Decision: ${state.pack.decision}`);
lines.push(”);
lines.push(state.pack.note);
lines.push(”);
lines.push(‘Top risks:’);
state.pack.risks.forEach((r,i)=>lines.push(`${i+1}. ${r}`));
lines.push(”);
lines.push(‘Recommended next steps:’);
state.pack.next.forEach((n,i)=>lines.push(`${i+1}. ${n}`));
return lines.join(‘\n’);
}
root.addEventListener(‘change’, (e)=>{
if(e.target && e.target.matches(‘input[type=”radio”]’)) compute();
});
calcBtn.addEventListener(‘click’, (e)=>{ e.preventDefault(); compute(); });
copyBtn.addEventListener(‘click’, async (e)=>{
e.preventDefault();
const state = compute();
if(!state) return;
const text = summaryText(state);
try{
await navigator.clipboard.writeText(text);
copyBtn.textContent=’Copied’;
setTimeout(()=>copyBtn.textContent=’Copy Results’, 900);
}catch(err){
const ta=document.createElement(‘textarea’);
ta.value=text; document.body.appendChild(ta);
ta.select(); document.execCommand(‘copy’);
document.body.removeChild(ta);
copyBtn.textContent=’Copied’;
setTimeout(()=>copyBtn.textContent=’Copy Results’, 900);
}
});
emailBtn.addEventListener(‘click’, (e)=>{
e.preventDefault();
const state = compute();
if(!state) return;
const subject = encodeURIComponent(`Oyez — Vendor Readiness (${state.posture}, ${state.s}/45)`);
const body = encodeURIComponent(summaryText(state));
window.location.href = `mailto:?subject=${subject}&body=${body}`;
});
resetBtn.addEventListener(‘click’, (e)=>{
e.preventDefault();
root.querySelectorAll(‘input[type=”radio”]’).forEach(r=>r.checked=false);
roleEl.value=”; orgEl.value=”;
compute();
root.scrollIntoView({behavior:’smooth’, block:’start’});
});
compute();
return true;
}
if(!init()){
let tries=0;
const t=setInterval(()=>{
tries++;
if(init() || tries>=50) clearInterval(t);
}, 100);
}
})();
