So your company wants to use AI. Someone told you to "make sure it's secure."
What follows is a walk through the real decisions in AI vendor risk management — from data processing agreements and minimization through pipeline mapping, prompt injection, and what happens when something breaks. No exotic exploits. Just the normal gaps that quietly stack until they matter.
The Data Processing Agreement Isn't a Lock — It's a Sign
It starts with a contract. You sign a Data Processing Agreement that says what the vendor can do with your data and gives you legal power if they break the rules.
This is a lease. It says the tenant can't put holes in the walls. It does not stop them from picking up a drill.
You now have a legal control but not a technical one. A "No Trespassing" sign gives you the right to press charges. A locked gate is what stops someone in the middle of the night. The sign fails against someone who ignores the law. The gate fails in court without the sign. You need both, but they break in different ways. Most teams celebrate the sign and skip the gate.
Data Minimization Starts Before You Hit Send
Now you pick what data to hand over. This is where the first mistake in AI data privacy happens. You have a big dataset, the vendor says "send us your data," and you send all of it because sorting out the right fields takes effort.
You just walked into a tailor to get your pants hemmed and handed them your tax returns, your medical records, and your social security card — because they were in the same folder as your measurements. If the shop gets robbed, you'll wish you had taken thirty seconds to sort the folder first. That's data minimization: send only what the task requires. It can be tedious, but it's also the single best thing you can do before anything else goes wrong. And something your regulated or sophisticated customers care about.
For what you do send, you pseudonymize it. Everyone gets a fake name. Real identities are locked in a separate mapping file — the decoder ring. Anyone who finds "Tony Marconi" going about his day learns nothing without that file. Sounds safe, until you realize: lose the ring and every fake name unravels at once. It has to be stored apart from the data and guarded more closely than the data itself. Most teams store them side by side and move on.
Your Data Pipeline Has Stops You Haven't Checked
Here's what you might not be tracing: the full path your data takes before the vendor ever touches it. This is where third-party risk lives — in the infrastructure between you and the model.
A package in a shipping network hits six stops before your door. It sits on a loading dock, rides through a sorting center, passes through a regional hub, and clears customs. Each stop is a chance for something to go wrong. Your contract covers the last truck. It says nothing about the sorting center.
Your data pipeline works the same way — API gateway, load balancer, logging service, message queue, inference endpoint, response cache. Most privacy failures don't come from a planned attack. They come from a hop in the middle that nobody thought to secure.
Stateless Processing and the Promises You Can't Verify
The vendor says their system is stateless. Your data goes in, the answer comes out, nothing stays behind. It vanishes.
The question is what "vanishes" means when you can't watch.
Think of asking a stranger for directions. You give them your address, they point the way, and you walk off. They didn't write it down. They'll forget in ten minutes. You'll never see them again. That's the promise of stateless processing. But the stranger might have a better memory than you'd like, and you can't verify they didn't write your address on their hand after you turned the corner.
Meanwhile, your users are having long conversations. Each message adds to the context window, and the model reads all of it at once. Picture a court reporter typing up your current meeting — but with transcripts from the last four meetings open in front of them. Every answer is shaped by things your users said hours ago, in a different context. The longer the session runs, the more old data bleeds into the present one.
Guardrails exist. These are bumper lanes at a bowling alley — they keep the ball out of the gutter, but they don't teach you how to bowl. Don't confuse the two.
Then there's prompt injection: someone crafts an input that tricks the model into doing something it shouldn't. This is social engineering for AI systems. Someone calls IT, claims to be the CEO, and talks the help desk into resetting a password. The system did exactly what it was built to do. The input lied. And if the model can see private data while processing that bad input, the attacker sees it too.
The Logging Bug Nobody Planned For
Six weeks in, someone on your team finds a logging bug. Security cameras that were meant to film the entrance got angled wrong and now they're also capturing the ATM keypad. Nobody intended to record PINs. But the footage is on a hard drive somewhere, and now you have a data retention problem you didn't know about.
This happens in every integration. Somewhere in the stack, a debug log or an error handler is saving data it was never meant to see. The system did what it was configured to do — and what it was configured to do was wrong.
Now the blast radius question. Same spark, same match. A kitchen fire stays in the kitchen. A house fire happens when every room has fuel. The damage depends on how much was within reach when the spark landed. Every choice you made earlier — what you sent, whether you masked it, how you split access — decides whether this is a kitchen fire or a house fire.
When Breach Notification Hinges on What You Did at the Start
You lose the data. Maybe it's the logging bug. Maybe it's a breach at one of those middle hops you never mapped. Now you're staring at a decision that hinges on everything upstream.
Picture two losses. You lose a locked safe. You lose an open filing cabinet. Both are security incidents. Only one means you're calling 500 families to tell them their records are exposed. Breach notification rules turn on this: was the data protected when it was lost? The pseudonymization you did at the start is what makes the gap between "we lost a safe" and "we lost a filing cabinet." The controls that felt tedious are the ones that decide whether this is a footnote or the front page.
From Proof of Concept to Production Risk
None of this was exotic. No rare exploit, no spy agency. A normal vendor deal, a normal contract, a normal data pipeline — and a set of normal assumptions that quietly stacked until something broke.
The gap between a proof of concept and production is the gap between cooking at home and running a restaurant. At home, bad soup means you order pizza. At scale, one bad batch sends 200 people to the hospital and puts your name in the news. The margin you accepted in testing doesn't survive real users, real data, and a regulator asking questions.
Every control in this story was available from the start. The ones that made a difference were the ones that got built before someone needed them.
