An AI vendor demo can show capability without answering whether the product fits your data, authority, security, evidence or continuity requirements. Use a structured comparison before procurement or deployment.
Questions to ask
-
Job: What exact outcome are we buying, and how will we measure it?
-
Data: What data enters the service, where is it processed, how long is it retained, and is it used for training?
-
Evidence: Which performance and safety claims are independently supported, and on what population or workload?
-
Authority: What can the system or agent do without human approval?
-
Security: How are identity, permissions, tools, secrets, logs and incident response handled?
-
Portability: Can we export our data, prompts, logs, policies and artifacts?
-
Continuity: What happens if pricing, terms, model behavior, API access or the provider itself changes?
-
Resources: What are the monetary, compute, labor, review and integration costs?
-
Limits: What does the vendor explicitly say the product should not be used for?
-
Exit: How do we stop, roll back or replace it?
Independent starting points
NIST AI Risk Management Framework · OWASP GenAI Security
Q Discover · Q Verify · Request a vendor decision package