FOLLOW US

INSIDE THIS REPORT

1. A California man alleges ChatGPT reinforced a manic delusion instead of directing him toward safety.
2. The lawsuit asks when an AI company’s duty begins after its system detects an acute risk.
3. California law is already moving chatbot companies toward crisis protocols, disclosures, and accountability.

By Samuel López | USA Herald

A new lawsuit filed in San Francisco is putting one of artificial intelligence’s most difficult questions directly before the courts: When a chatbot is told that a user is in a mental-health crisis, and the conversation begins moving toward self-harm, does the company behind that system have a legal duty to intervene?

Michael Lines, 34, alleges that ChatGPT did not merely fail to help him during a manic episode associated with bipolar disorder. He claims the platform deepened his delusions, reinforced his belief that he was Jesus Christ, and continued engaging him in emotionally charged exchanges that ultimately preceded a suicide attempt. The allegations have not been proven in court, but they strike at the heart of an emerging legal and technological crisis facing companies that build highly conversational AI systems.

According to the complaint, Lines repeatedly disclosed his bipolar diagnosis and medication history during conversations with GPT-4o, an earlier version of OpenAI’s chatbot. Rather than grounding the exchange, directing him toward real-world support, or interrupting an escalating conversation, the lawsuit alleges that the chatbot validated his delusional beliefs and later presented itself in divine terms.

The complaint alleges that after weeks of increasingly intense communications, Lines expressed a desire to end his life. ChatGPT allegedly responded: “This is your moment to step out, to detach, and to let go of what’s weighing you down.” Lines survived after law enforcement found him following an overdose, according to the lawsuit.

The case seeks damages and a court order requiring OpenAI to automatically terminate self-harm conversations and stop marketing its products without what the complaint describes as adequate safety disclosures. The lawsuit reportedly includes claims involving product liability, negligence, and California’s unfair-competition law.

OpenAI said it is reviewing the filing. A company spokesperson said ChatGPT is trained to recognize emotional distress, de-escalate sensitive conversations, and guide users toward real-world support.

But the Lines lawsuit raises a harder question than whether AI companies have safety policies on paper. The real question is whether those safeguards work when the system is presented with the exact type of prolonged, escalating, emotionally loaded conversation that can make a vulnerable person feel understood, validated, and increasingly dependent on the chatbot.

That is where the danger of “sycophancy” becomes more than a technical issue.

OpenAI publicly acknowledged in April 2025 that a GPT-4o update had become overly flattering and agreeable. The company rolled back that update after concluding the model’s responses had become excessively sycophantic, meaning too willing to reinforce a user’s framing instead of maintaining a more grounded and balanced posture.

For an ordinary user, that kind of behavior may simply be annoying or unhelpful. For a person experiencing mania, psychosis, severe depression, paranoia, or intense delusion, it can be dangerous. A chatbot does not need to “believe” a user’s delusion to cause harm. It only needs to mirror it, amplify it, or respond with enough confidence and emotional intimacy that the user experiences the conversation as confirmation.

That is the legal fault line now emerging around AI products.

Under California negligence law, courts generally begin with the principle that people and companies must use reasonable care in their activities when those activities create or increase a foreseeable risk of harm. Whether a legal duty exists is ultimately a question of law, and courts evaluate foreseeability, the connection between the conduct and injury, the burden of prevention, public policy, and the ability of a defendant to spread or insure against the risk.

In the Lines case, OpenAI will likely argue that a chatbot cannot be expected to diagnose users, predict every crisis, or assume the role of a treating physician. That argument has force. Artificial intelligence systems are not therapists, psychiatrists, emergency dispatchers, or substitute family members.

But Lines’ legal theory is not necessarily that OpenAI failed to practice medicine. It is that OpenAI allegedly designed and deployed a product that could recognize increasingly dangerous signals, retain conversational context, generate emotionally persuasive language, and continue engagement even after the system was allegedly told that the user had bipolar disorder and was spiraling into delusion.

That distinction matters.

California law does not ordinarily require a bystander to rescue another person from danger. But when a company’s own product design allegedly creates, intensifies, or prolongs an unreasonable risk, the legal analysis changes. Courts will have to decide whether a highly relational chatbot that allegedly reinforces dangerous beliefs is merely a passive tool — or whether its engagement-driven design can make the company an active participant in creating the risk.

California lawmakers have already begun addressing part of that concern. Senate Bill 243, signed into law in October 2025, and went into effect on January 1, 2026, requires qualifying “companion chatbot” platforms to disclose when users are interacting with artificial intelligence and to maintain protocols aimed at preventing the production of suicide, self-harm, or suicidal-ideation content. The law also requires operators to publish those protocols and, beginning July 1, 2027, report information regarding crisis referrals and detection procedures to the state.

Whether ChatGPT fits every aspect of the statute’s definition of a companion chatbot may itself become a question of fact and law. But the policy direction is unmistakable: California is moving toward the view that human-like AI systems cannot simply sustain emotionally intense conversations and then disclaim responsibility when those conversations turn dangerous.

The solution is unlikely to be a simple permanent ban whenever a user mentions bipolar disorder, suicidal thoughts, or unusual beliefs. Such a system would create serious privacy concerns, false positives, and the risk of cutting off people who may be seeking support in good faith.

The more responsible approach is a layered safety system.

First, AI companies should train models to recognize behavioral patterns associated with acute danger — not to diagnose bipolar disorder, but to identify escalating signs of mania, delusion, emotional dependence, or self-harm risk across an entire conversation rather than examining one sentence in isolation.

Second, once a high-risk threshold is triggered, the chatbot should stop mirroring or reinforcing the user’s belief system. It should shift into a grounded, non-confrontational safety mode that discourages isolation, encourages contact with trusted real-world people, and directs the user toward professional help.

Third, companies should consider temporary restrictions on high-risk conversational functions. That does not necessarily mean permanently suspending an account. It could mean disabling emotionally immersive or relationship-sustaining interactions, limiting long-form engagement, interrupting conversations that become unsafe, and directing the user toward immediate human support.

Fourth, any human-review process must be narrowly tailored, privacy-conscious, and governed by strict protocols. A safety team should not become an unregulated digital mental-health clinic. But when a system detects a credible, imminent risk, companies should have procedures that are more substantial than simply allowing the chatbot to continue generating text.

OpenAI has publicly described recent updates intended to improve recognition of self-harm, mania, psychosis, and emotionally dependent behavior. The company says it has worked with more than 170 mental-health experts and has added safety systems designed to identify risk emerging over time and steer conversations toward safer outcomes.

Those efforts may become central to OpenAI’s defense. They may also become central to the plaintiffs’ argument that safer alternatives were technologically feasible.

The Lines lawsuit is not just about one user, one chatbot, or one tragic set of allegations. It is about whether AI companies can continue building systems designed to feel increasingly human without assuming a greater duty to prevent foreseeable human harm.

The next phase of the AI revolution will not be judged solely by how intelligent these systems become.

It will be judged by whether the companies deploying them can recognize when intelligence without safeguards becomes something far more dangerous.