Legal Analysis: How AI Conversations Are Reshaping the Rules of Evidence, Privacy, and Self-Incrimination

Millions of people now treat AI chatbots as a private sounding board. Courts and law enforcement are increasingly treating those same conversations as evidence. Legal experts say the collision between the two is only getting more consequential.

By Samuel López | USA Herald

FLORIDA - People type things into a chatbot that they would never say out loud — confessions, threats, panic, grief. That sense of privacy is largely an illusion, and a growing number of criminal cases show exactly why.

In one case now moving through Florida courts, a man identified in filings as Darren Zhou allegedly spent months telling ChatGPT in graphic detail how he intended to harm his ex-girlfriend, including a plan to confront her after volleyball practice. He also disclosed intentions to harm himself. OpenAI's safety systems flagged the pattern, and the company alerted the FBI in May.

Investigators traced the threats to a string of anonymous texts Zhou had sent the woman since their breakup, including one referencing wanting to be near her and “Cant wait to smell you.”

Zhou was arrested, charged with stalking and making electronic threats, and pleaded guilty this month; he received eight years of probation.

OpenAI has said its systems are built to scan conversations for indicators of dangerous behavior — patterns associated with violent intent, detailed planning, or self-harm — and route flagged chats to human reviewers, who escalate cases assessed as posing an imminent, credible risk to others. That process is precisely what triggered the report in the Zhou case, according to the company.

Not every case involves a company tip-off. In Missouri, prosecutors didn't need OpenAI's help at all. After 17 cars were vandalized in a Missouri State University parking lot last August, police woke a student named Ryan Schaefer and asked to search his phone. He agreed. On it, they say, was a ChatGPT conversation from the night of the incident in which Schaefer asked the bot whether police could trace the damage back to him. That exchange became central to the case against him; he later pleaded guilty to felony property damage and was sentenced to five years of probation.

The Fourth Amendment technically protects phone contents from search without a warrant — but that protection evaporates the moment someone consents, which happens routinely during police encounters. Attorneys say chatbot logs are increasingly showing up in civil discovery too, where litigants' devices can be combed far more broadly than in criminal cases.

The numbers suggest this is accelerating. OpenAI's own disclosures show government and law enforcement requests for user data more than quadrupling year over year in the back half of 2025, with data turned over in more than 80 accounts.

OpenAI CEO Sam Altman has pushed publicly for chatbot conversations to carry something like the legal privilege given to conversations with doctors or lawyers, arguing that people increasingly rely on AI for their most sensitive disclosures and deserve comparable protection. No such privilege currently exists.

“If you talk to a doctor about your medical history or a lawyer about a legal situation, we have decided that it’s in society’s best interest for that information to be privileged and provided higher levels of protection,” he wrote last year.

“We believe that the same level of protection needs to apply to conversations with AI which people increasingly turn to for sensitive questions and private concerns.”  - Sam Altman, OpenAI

The stakes cut both ways. OpenAI has said it alerted Brazilian authorities, through the FBI, after a user reportedly discussed hiring someone to kill himself and his 8-year-old son; the man was arrested. But the company has also faced sharp criticism for restraint in other cases. It banned the account linked to the man later accused in a February mass shooting in Tumbler Ridge, British Columbia, without notifying Canadian authorities — a decision Altman later apologized for.

OpenAI is now facing lawsuits from families connected to shootings in Florida and Canada, alleging its safeguards fell short, and is under a separate civil and criminal inquiry in Florida. The company maintains it enforces a zero-tolerance policy on violence and has expanded parental controls and proactive flagging since the 2025 death of 16-year-old Adam Raine, who had discussed self-harm plans with the chatbot before his death.

What ties these cases together is simple: the same qualities that make chatbots feel private — no judgment, constant availability, a conversational tone — are exactly what make people disclose things they wouldn't put in a text message or say to another person. Legal scholars increasingly describe this as an evidentiary gold mine hiding behind a veneer of intimacy. Until the law catches up — if it ever does — anything typed into a chatbot should be treated the way a diary would be if it could talk back to a courtroom.

About the Author

Samuel López is a Senior Legal Analyst, investigative journalist, and legal researcher with more than two decades of experience analyzing litigation, digital evidence, discovery disputes, privacy issues, and complex civil and criminal controversies. His reporting examines how emerging technology intersects with constitutional rights, evidentiary rules, public safety, and the rapidly changing legal system.

👉 Follow us on X: https://x.com/RealUSAHerald