How do you assess AI vendor risk?

Send a structured question set before contract and score each answer as answered, partial, or refused. Cover the supply chain including all sub-processors, underlying model providers and inference hosts; where inference is physically performed and under whose law; how data is handled, retained and used for training; and what evaluation evidence you receive on your own data rather than the vendor’s benchmark. A pattern of refusals is itself a finding.

What should you ask an AI vendor before signing a contract?

Ask them to list all sub-processors including underlying model providers and inference hosts, state where inference is physically performed, confirm whether your data is used for training or retained after termination, provide evaluation results on your data and languages, and set out what happens on exit, specifically whether you leave with your data and decision records intact.

AI Vendor Assurance Question Set

Last updated: 2026-08-14

The AI vendor assurance question set is sent before contract rather than after, and each answer is scored as answered, partial, or refused. It covers the supply chain including sub-processors and underlying model providers, where inference is physically performed, how data is handled and retained, and what evaluation evidence the buyer receives.

Where a system is bought rather than built, assurance replaces inspection. The contract and the questions asked before signing are the only governance instruments that still work once the system is in place.

The template

Send this before contract, not after. Score each answer as answered, partial, or refused.

Supply chain 1. List all sub-processors, including underlying model providers and inference hosts. 2. Where is inference physically performed, by jurisdiction? 3. Which sub-processors can access customer content, and under what controls?

Data handling 4. Is customer content used for training, fine tuning, or evaluation? State the default and how to opt out. 5. What is the retention period for prompts, completions, and logs? 6. Is content isolated per tenant at rest and in transit? Describe the mechanism. 7. Which of your staff can read customer content, under what approval, and is that access logged and reviewable by us?

Change management 8. What is your notice period for a change to the underlying model? 9. What is your deprecation policy and minimum support window for a pinned version? 10. Do we have the contractual right to re-test after a model change?

Security controls 11. Provide the results of your most recent independent security assessment and the remediation status of open findings. 12. Describe your controls against prompt injection, and state which residual risks you consider open. 13. What permissions does the product require in our environment, and what is the minimum viable set?

Incident and exit 14. What is your incident notification commitment, in hours, and what triggers it? 15. On termination, how is our content deleted, on what timeline, and how is deletion evidenced?

How to use it

Send the set before contract. After signature the leverage to obtain answers is gone.
Score each answer as answered, partial, or refused, and treat a pattern of refusals as a finding in itself rather than a gap to chase.
Insist on evaluation results against your own data and languages, not the vendor’s published benchmark.
Carry the answers into the model register entry so the assurance position is visible at every later review.

This template comes from the AI Security and Assurance: A Practical Playbook, which sets out the reasoning behind it, the rules of thumb that govern its use, and the signals that tell you the approach is failing. The full playbook is also available as Markdown. See all AI governance templates.