Josh said something on our podcast that I have quoted in client conversations ever since. We were talking about what would make him comfortable putting client data near an AI tool, and he said this:
He can redact things all day, but there's going to be one or two times that he uploads a file and forgets to redact everything.
Sit with that for a second. This is a careful accountant, thinking hard about client confidentiality, who has already decided he is going to mess it up eventually.
He is right. And most data policies I see are written as though he is wrong.
The shape of a bad control
Here's the pattern. A company decides AI is allowed if employees remove sensitive information first. It goes in the handbook. Everybody nods.
That policy has a single point of failure, and it's a human being doing a repetitive task under time pressure, correctly, forever.
Nobody does anything correctly forever. Not because people are bad at their jobs. Because attention is finite and Thursdays are long.
Any control that requires perfect human vigilance isn't a control. It is a way of assigning blame after the fact. Which is worse than having no policy, because it lets everyone feel protected while nothing is actually protecting them.
What a real control looks like
The test is simple: does this still hold when somebody is tired?
Choose tools that can't retain the data. Josh's first requirement was that whatever he used couldn't be greedy with client information. If the tool never keeps it, forgetting to redact stops being a catastrophe and becomes a mistake with a ceiling. Move the guarantee from the person to the system.
Limit what the tool can reach. If AI can only see what it needs for the task in front of it, a slip exposes a slice instead of everything. This isn't an AI idea, it's just least privilege, and every other part of your business already works this way.
Make exposure visible. The thing that turns a small incident into a large one is time. If nobody can see what was sent where, the discovery happens months later, usually from outside.
Assume the slip and design the recovery. What happens after somebody uploads the wrong file? If your answer is "that shouldn't happen," you don't have an answer. You have an assumption.
The part people skip
There is a second group Josh and I both noticed, and they need something different from a policy.
Some employees pasting data into AI tools haven't thought about the data question at all. Not because they are cavalier. Because it genuinely never occurred to them, the way it would not occur to most people to think about where an email attachment physically lives.
You don't fix that with a rule. You fix it by explaining, once, in plain language, what happens to text after you paste it into a box on the internet. Most people adjust immediately when they understand. Almost nobody adjusts because a handbook told them to.
Blame is cheap and it doesn't scale. Explaining is more work and it actually holds.
Why this is worth caring about
There is an ethical layer here that gets lost in the compliance framing.
That data is somebody's. A client's finances. A patient's history. An employee's address. They handed it over because they had to in order to do business with you, and they had no practical way to check what you would do with it.
That is a trust relationship, and it doesn't stop mattering because the exposure was accidental. "We didn't mean to" is true and irrelevant to the person whose information is now somewhere they never agreed to.
Designing systems that survive a tired human on a Thursday isn't paranoia. It is the ordinary work of looking after something that was entrusted to you by someone with no ability to verify you were doing it.
The question
Not "do our people know the policy." They probably do.
The better question is: if the most conscientious person on our team makes the exact mistake they already expect to make, what does the damage look like, and who finds out?
If you can't answer that, the policy isn't protecting anyone. It is just deciding in advance whose fault it will be.
From a conversation with Josh on J&C Unscripted.