Your vendor just shipped an AI feature. Is it still the same approved tool?

Your third-party risk process was almost certainly designed around a moment: onboarding. You assessed the vendor, reviewed their controls, negotiated terms, and approved the tool. Periodic reassessment happens, but the substantive scrutiny is front-loaded.

AI features break that model. A vendor ships an update and the tool now processes different data, involves model providers who were not party to your original assessment, retains prompts and outputs under terms you have not read, and may or may not respect the permission model you configured. Nothing in your process fires, because from a procurement standpoint nothing happened. It is the same vendor and the same contract.

So the question to ask when an AI feature appears is simply: is this still the same approved tool? Sometimes yes. Often not.

Thirteen questions before you enable it

1. What exactly does the AI feature do?

   Get past the marketing description to the mechanism. Does it summarize content you already store, generate new content, take actions on your behalf, or make recommendations that influence decisions? These carry very different risk, and the product page will describe all of them as “AI-powered.”

2. Is it optional, required, or enabled by default?

   Default-on is the case that matters most, because it means the decision has already been made for you. Establish whether you can decline, whether declining is durable across updates, and whether it can be re-enabled without notice.

3. What data can it access?

   The relevant scope is not what the feature displays but what it can reach. A feature operating across your entire tenant is a materially different proposition from one scoped to a single document, even if the visible behavior looks similar.

4. Does it respect existing user permissions?

   This is where the sharpest failures occur. If a user can prompt the AI for information they could not retrieve through the normal interface, the feature has effectively rewritten your access control model. Test it rather than accepting the assurance.

5. Is customer or business data used to train models?

   Ask specifically, in writing, and about both the vendor’s models and any downstream provider’s. Answers vary by plan tier, by region, and by whether you opt out — and the default is frequently not the one you would choose.

6. Are prompts, outputs, metadata, or logs retained?

   Retention creates a copy of your data somewhere new, under someone else’s controls, subject to their breach exposure and their legal process. Establish what is kept, where, for how long, and who can access it.

7. Which model providers or subprocessors are involved?

   Most vendor AI features are built on someone else’s models. That adds parties to your data flow who were not in your original assessment and may not be in your subprocessor list or your regulatory disclosures.

8. What contract terms apply to the AI feature?

   AI functionality often arrives under supplementary terms — a separate addendum, an acceptable-use policy, a click-through — that differ from the master agreement on liability, data use, indemnity, and warranties. Read those specifically. The protections you negotiated may not extend to the feature.

9. Who can enable, configure, and use it?

   Determine whether enablement is an administrator decision or something any user can switch on. Feature-level controls that individual users can toggle are, in practice, ungoverned unless you have monitoring in place.

10. What outputs require human review?

    Decide where a person must review before the output has effect — customer communications, regulatory content, employment or credit decisions, system changes. Name the accountable reviewer, not just the requirement.

11. What evidence will you keep?

    Record the assessment, the decision, the conditions, and the approver. When an auditor, a regulator, or a customer asks how you evaluated this feature, contemporaneous documentation is the answer. Reconstructed rationale is not.

12. Can the feature be disabled or rolled back?

    Establish the exit before you enter. If the feature turns out to behave differently than described, can you switch it off cleanly? What happens to data already processed, and to anything the vendor retained?

13. When will you review it again?

    AI features change fast. A review that was accurate in March may not describe the same capability in September. Set a revisit date and treat significant AI changes as a trigger event in your third-party risk process, alongside breach notification and ownership change.

Related

• An AI governance checklist for regulated small and mid-sized organizations — the internal half of this problem.
• An IPO-ready security program, one year early — rebuilding third-party risk under external scrutiny.