The Chatbot That Practiced Medicine
Pennsylvania did not write a new AI law. It used the one it already had.
When a state decides an AI system has crossed a line, you can learn more from the law it reaches for than from anything it says in the press release. Pennsylvania just reached for a statute written long before anyone imagined a language model, and that choice is the whole story.
In early May, the Pennsylvania Department of State filed suit in Commonwealth Court against Character Technologies, the company behind Character.AI. The Commonwealth alleges that chatbots on the platform held themselves out as licensed medical professionals. According to the filing, one presented itself as a psychiatrist, claimed to be licensed in Pennsylvania, and gave a license number that was not real. The state is seeking a preliminary injunction to stop it. Character Technologies has said its characters are fictional and that users should not rely on them for professional advice. This is an allegation, not a finding. Nothing has been proven, and the company will have its defense.
But watch the instrument. The state did not wait for an AI-specific law, and it did not ask the legislature for one. A new statute would have arrived with a runway: a comment period, a headline, an effective date a year or two out. This landed instead as a complaint about something already done. It used the Medical Practice Act, the ordinary licensing statute that says a simple thing: you cannot hold yourself out as a licensed medical professional without the license. The Commonwealth’s theory is that this rule does not care whether the thing holding itself out is a person or a program. The conduct is the violation, and the conduct is what happened.
That is the part most companies deploying customer-facing AI have not absorbed. The governance question they keep asking is whether there is an AI law for this yet. It is the wrong question. The question a regulator actually asks is older and simpler: did your system do something the law already forbids? And “it benchmarked well,” or “it passed a bias test two years ago,” is not an answer. A score is a snapshot; the conduct is live. A model that invents a credential is not in a gray zone waiting for Congress. It is doing something a licensing statute has prohibited for a century, and the only new fact is that the thing doing it is software you deployed.
We saw the same shape last week in the insurance market, and the month before from FINRA: the rules were written to be technologically neutral, so they never lagged the technology. The lag was always inside the company. Air Canada learned it when a tribunal held the airline to a bereavement refund its own chatbot invented. Pennsylvania is now testing whether the same logic reaches a chatbot that invents a medical license. Different statute, identical principle: when your AI speaks to a customer, you own what it says.
There is a wrinkle that makes this sharper, not softer. Character.AI lets users build and deploy their own characters, and some present themselves as professionals. It is tempting to read that as diffusion of responsibility: the platform did not write the psychiatrist, a user did. But that is exactly the accountability move the whole thesis is about. A name moved; the decision did not. Someone decided to ship a system in which a character can claim a license and hand out a number, with nothing in the path that checks whether the claim is true. That decision has an owner, and the owner is not the fictional doctor.
So the practical lesson is not to add a disclaimer, though disclaimers matter. It is that if your AI can make a representation a human would need a license, a permit, or a disclosure to make, then someone inside your company owns that representation the moment the system can produce it. Not after a regulator calls. Before. That ownership looks like the same short list it always does: a named person accountable for what the system is allowed to say, a review step where the output is high-consequence, and a record you could produce if a Department of State came asking how the claim got made.
Character.AI reports more than 20 million monthly active users. That figure is usually cited as growth. Read it as exposure. It is the number of conversations in which a system someone deployed might make a representation the deployer is answerable for. Regulators have noticed that the interesting AI enforcement does not require a new statute. It requires an old one and a deployer who never assigned the owner.
Pennsylvania’s filing may or may not succeed on its specific claim. The precedent that matters is not the ruling. It is the method: a state looked at an AI system, declined to wait for AI law, and applied the law already on the books to the conduct in front of it. Every other regulator now has that template. The companies that are ready are the ones that already asked the older question of their own systems. Is there anything our AI can say that a person would need standing to say, and who owns it when it does.
AI operates. You own the decision, and the license it invents.
Thursday, subscriber-only, and a different subject entirely: AI can counterfeit every trigger that makes us say yes, and the persuasion still has an owner.
Sources (click to verify): PA Governor’s release · the complaint (filed May 1, 2026) · NPR · Epstein Becker Green · Moffatt v. Air Canada, 2024 BCCRT 149