When you deploy AI from a vendor, you inherit part of their AI risk. This guide covers the questions to ask, the documents to request, and the red flags to watch for when evaluating AI suppliers under the EU AI Act and NIST AI RMF.

Why Third-Party AI Risk Matters More Than You Think

Most organisations deploying AI are not building models from scratch. They are using AI from vendors — cloud API providers, SaaS platforms with embedded AI, open-source models, or fine-tuned foundation models from commercial providers. In every case, the organisation deploying the AI inherits a portion of the risk that comes with it.

Under the EU AI Act, the deployer of a high-risk AI system has obligations regardless of whether they built it. You cannot outsource your compliance obligations to your AI vendor. You are responsible for ensuring the system meets Article 8-15 requirements, even if the system was entirely built by a third party.

Under the NIST AI RMF, third-party AI risk is explicitly part of the GOVERN function (GOVERN 6.1 and 6.2 cover AI risk management in the supply chain) and the MANAGE function.

The Three Layers of Third-Party AI Risk

Layer Examples Your exposure
Foundation model layer Claude, GPT-4, Gemini, Llama — the base model your application is built on Training data bias, unknown capability limitations, model behaviour changes between versions, provider compliance status under EU AI Act Chapter V (GPAI).
Platform and infrastructure layer MLOps platforms (Weights and Biases, MLflow), vector databases, AI APIs, cloud AI services (AWS Bedrock, Azure AI) Data residency, access controls, security certifications, SLA for availability and performance.
Application layer AI AI-powered SaaS tools your organisation uses: AI hiring tools, AI customer support, AI content generation, AI analytics platforms High-risk AI system compliance if the use case falls in Annex III. Provider's documentation quality. Alignment with your own AI policies.

What to Request from AI Vendors: The Document Set

For foundation model providers (API-level)

  • Model card or technical report: training data overview, intended uses, known limitations, evaluation results.
  • EU AI Act GPAI compliance statement: for models with substantial compute, evidence of Chapter V compliance (technical documentation, copyright compliance, training data summary).
  • Acceptable use policy: what use cases the model is and is not designed for.
  • Version change management policy: how the provider notifies customers of model updates that may affect performance or behaviour.
  • Data processing agreement covering inference data: who can see your inputs, how long they are retained.

For AI-powered SaaS platforms (application layer)

  • EU AI Act compliance status: if the product's use case falls in Annex III, has the provider completed a conformity assessment and registered in the EU AI database?
  • Technical documentation summary (Article 11): or at minimum the Article 13 instructions for use.
  • Performance documentation including disaggregated metrics by relevant demographic groups.
  • SOC 2 Type II or ISO 27001 certification (security baseline).
  • Incident response and notification SLA: if the AI system produces a serious incident, what is the provider's notification timeline?
  • Sub-processor list: who else has access to data processed by the AI system?

For open-source or self-hosted AI models

  • Model card and associated research papers.
  • License terms: commercial use, attribution, fine-tuning, and redistribution rights.
  • Provenance of training data: what datasets were used and are there known bias or contamination issues?
  • Community issue tracker: known failure modes reported by the community.
  • Last maintenance update: is the model actively maintained or abandoned?

The Due Diligence Questionnaire: Key Questions

Category Questions to ask the vendor
EU AI Act classification Have you classified this product under the EU AI Act? Is it high-risk under Annex III? If so, has the conformity assessment been completed and what is your EU AI database registration number?
Fairness and bias What fairness assessments have been completed? What protected attributes were tested? What were the results? Are disaggregated performance metrics available for our specific use case?
Data governance Where does training data come from? Is there a known data bias history? What data does the platform process at inference time, and where is it stored?
Human oversight What human oversight does the system provide or require? Can oversight settings be configured? What are the system's out-of-scope use restrictions?
Incident history Have there been any serious incidents with this AI system? If so, what happened and what was remediated?
Version management How are model updates managed? Do updates require revalidation? What is the notification period before a model change goes live?
Subprocessor and data residency Who are the key sub-processors? Can data residency be restricted to specific regions? What certifications cover the infrastructure?

Red Flags in Vendor Responses

  • Cannot provide disaggregated performance metrics. Any vendor offering AI in a high-risk domain who cannot provide group-level performance data either has not tested properly or is hiding unfavourable results.
  • Vague or no response to EU AI Act classification questions. Reputable AI vendors selling into EU markets have addressed these questions. Evasion suggests either non-compliance or a product not designed for regulated use.
  • No version change management policy. Models that change without notice can invalidate your own validation and compliance documentation.
  • No data processing agreement. If the vendor cannot provide a DPA, your legal team will not approve the deployment.
  • Incident response SLA measured in days or weeks. For AI systems affecting real-world decisions, serious incident notification should be measured in hours.
  • Open-source model with no active maintenance and known bias issues in the community tracker. Forking and self-hosting does not remove the bias; it removes the upstream fix path.

Contractual Protections

Due diligence is not a one-time event. Your contracts with AI vendors should include ongoing protections:

  • EU AI Act compliance warranty: vendor warrants their product complies with applicable AI Act obligations and will maintain compliance as the Act's requirements come into force.
  • Performance benchmark maintenance: if the product performance falls below the agreed metrics, vendor must remediate within a specified timeframe.
  • Change notification: vendor must notify you at least 30 days (or a commercially negotiated period) before material model changes that may affect performance or compliance.
  • Audit right: your organisation or its designee can audit the vendor's AI compliance documentation on reasonable notice.
  • Incident notification: vendor must notify you within 24 hours of any serious incident affecting your deployment or the platform generally.
  • Data deletion: vendor confirms deletion of inference data within a specified retention period.

Building an AI Vendor Register

Your AI supply chain risk management starts with visibility. Maintain an AI vendor register covering:

Field Description
Vendor name and product The specific AI product or model being used
Use case How you are using it — matched to your AI system inventory
EU AI Act risk tier Provider's stated classification + your own assessment
Last due diligence date When the assessment was last completed or reviewed
Key certifications held ISO 27001, SOC 2, EU AI Act conformity assessment, etc.
DPA in place? Yes/No. Date of last review.
Contract expiry / review date When the supply contract is next due for renewal/review
Known issues or open items Outstanding due diligence gaps, pending documentation requests 
Review your AI vendor register at every quarterly GOVERN review (if following NIST AI RMF) and at every ISO 42001 management review. Vendor compliance status changes — a certification lapse or a new public incident can materially change your risk exposure.