When ChatGPT Becomes Evidence: Why Your AI Stack's Data Layer Just Got a Lot More Important
Last week, CNN ran a story that should make every founder building with AI sit up. Prosecutors are now treating ChatGPT conversations as a "treasure trove" for criminal investigations. A Florida murder suspect's chat logs. The LA wildfires arson case. A Snapchat AI conversation used as key evidence in a Virginia murder trial. Florida's attorney general just opened a criminal investigation into OpenAI itself for the advice ChatGPT allegedly gave a mass shooting suspect.
The legal experts CNN talked to were blunt. "Anything that somebody's typing into ChatGPT is something that could be discoverable." There is no doctor-patient privilege with a chatbot. No attorney-client. No therapist confidentiality. Sam Altman himself said he's "very afraid" of how this plays out.
This is a courtroom story, but the operational lesson is for anyone shipping AI products: the data layer of your stack just got upgraded from boring infra to a strategic decision.
What actually changed
For the last two years, most teams treated their LLM API choice the way they treated their hosting provider in 2015. Pick the one with the best price-to-performance ratio, plug it in, ship the feature. The model was the product. The pipe was a commodity.
That framing is dead.
Every API call you make routes user data through someone else's infrastructure under someone else's retention policy under someone else's jurisdiction. When a subpoena lands, your provider's defaults become your defaults. When a regulator comes knocking, "we just used the OpenAI SDK" is not a compliance story. It's a confession.
The questions that matter now
If you're building an AI product, your data architecture diagram should answer all of these without you having to call your lawyer:
Where does the prompt go? Direct to OpenAI? Through Azure? Through Bedrock? Each one has a different retention default, a different DPA, and a different posture on government data requests.
How long is it retained? Zero data retention is not a default. You usually have to ask for it, sign for it, and verify it.
Who has subpoena access? A consumer ChatGPT account, a Tier 1 API account, and an enterprise contract are three completely different legal surface areas, even though they hit the same model.
What jurisdiction does the data sit in? This matters double for regulated verticals. Healthcare providers, defense contractors, law firms, and financial services all have non-overlapping rules about where data can live and who can see it.
What does your terms of service actually say? Because the moment you process user data through a third-party API, you've inherited a chain of custody problem you did not write.
Why this is a vertical AI story
The horizontal AI products that most founders are obsessed with are the ones that are most exposed here. Generic chat interfaces sitting on top of OpenAI are pure pass-through. Whatever the user types, OpenAI sees. Whatever OpenAI keeps, you keep.
Vertical AI is structurally better positioned. When you build for one workflow, in one industry, for one operator, you control the schema, the retention, and the routing. You can run the inference on a model hosted in a jurisdiction you chose. You can strip PII before it leaves your infrastructure. You can give the operator a real answer when they ask "what happens if we get subpoenaed."
This is a moat, not a constraint. A hospital is not going to plug patient data into a ChatGPT wrapper that could end up in evidence. A law firm is not going to draft motions on infrastructure that retains the prompt for thirty days. A defense contractor cannot use a model that processes data outside the United States.
The actionable takeaway
Three things to do this week if you're shipping anything with an LLM in the loop:
- 1.Map every external API call your product makes and write the retention policy for each one in plain English. If you cannot, you do not actually know what your product does.
2. Decide whether you need a Bedrock-class deployment, a zero-retention enterprise tier, or self-hosted inference. The right answer depends on your customer, but "OpenAI default" is rarely it.
3. Tell your customers what you do with their data. Specifically. With dates and jurisdictions. Vague privacy policies were tolerated when AI was a novelty. They are not going to be tolerated now that AI conversations are being introduced as exhibits.
The model layer is going to keep getting cheaper and more interchangeable. The data layer is where the real product decisions live. The companies that figure this out before their first subpoena will own the regulated verticals. The ones who do not will spend their Series A on legal fees.
