Pharma can find a biological target in a week that used to take a quarter. The law still treats that model as a chatbot in California, a high-risk system in Brussels, and a HIPAA side project in the United States. That is not a paperwork inconvenience. It is how uneven care gets built.
Healthcare did not wait for a finished AI statute. It never does. Drug discovery, imaging, triage, prior authorization, ambient scribes, and companion-style patient chatbots are already in production while the people who write the rules argue about which chapter of which code applies. On September 8, the Atlantic Council’s Cyber Statecraft Initiative put that contradiction on a webinar. The useful part of the conversation was not the ritual warning that regulation is fragmented. Everyone in a health system legal department already knows that. The useful part was the admission that the fragment is now inside the model.
Ranjit Kumble, who runs enterprise data science and advanced analytics at Pfizer, put the clinical version of the problem in one sentence. When you build an AI model, the goal is a benefit for a population. Deploy it without safeguards and the benefit becomes uneven. Safeguards are what make the benefit consistent. That is not a privacy slogan. That is a safety claim. An oncology model trained on one health system’s EHR and dropped into another system’s workflow is not “innovative.” It is a silent change in who gets flagged, who gets delayed, and who never enters the trial.
The rest of the panel spent an hour explaining why those safeguards cannot be written once and reused. They were right. They were also describing a market that will keep shipping anyway.
Three continents, three theories of the same dataset
A global life-sciences company does not have “health data.” It has a stack of incompatible theories about health data. The GDPR treats special-category health information as a high bar with a short list of lawful bases and a long memory for transfers. U.S. state comprehensive privacy laws treat health-adjacent data as sensitive, except when HIPAA already has it, except when a consumer health law like Washington’s or Nevada’s grabs the wellness app that HIPAA missed. China treats cross-border movement of clinical research files as a national-security event. The EU AI Act, when the high-risk clock actually runs, will treat certain medical devices and Annex III uses as conformity-assessment problems. U.S. state chatbot laws will treat the same interface as a disclosure and age-gating problem.
None of those regimes is wrong in isolation. Together they are an operating system that does not boot. Kumble said teams are now preparing data separately for each AI use case to make it “AI ready.” That sounds diligent. It is also a confession. If every new model requires its own extraction, de-identification, lawful-basis memo, and transfer map, the company does not have a data platform. It has a cottage industry of one-off compliance. He said it out loud: as use cases multiply, the work becomes unsustainable and unscalable. The sector wants a “precision-advantaged” world where the breadth and depth of training data is what patients trust and what competitors envy. You do not get there by running a unique privacy workstream for every pilot.
The tempting shortcut is to stop using sensitive data. That shortcut is how you get a model that is legally clean and clinically stupid. Skip the messy diagnoses, the longitudinal meds, the lab trends, the social determinants someone coded badly in 2019, and you will ship a tool that performs on the brochure and fails on the floor. The panel’s warning that conflicting rules could hit clinical trials and research is not hypothetical. Trial sponsors already negotiate protocol language around where a sample can sit and which model can touch it. Add an agent that wants to pull from three countries’ sites and you do not have a scientific question. You have a transfer-impact assessment with a protocol number on it.
Kumble’s proposed way through is the grown-up one and the slow one: consensus privacy safeguards and data-management standards written by the sector, then offered to policymakers as the thing they can cite instead of inventing a fourth definition of de-identified. That is how HIPAA security became implementable. It is also how standards get captured. If the consensus is written only by the ten companies that can afford a dedicated AI-ready data team, the standard will look like their stack. Smaller providers will be told they are noncompliant for not having a semantic layer and a model card factory.
High-risk in Europe, a notice in America
The risk-classification mismatch is the part boards still underestimate. Under the AI Act, a system used in medical devices or certain health decisions sits on the high-risk track: risk management, data governance, logging, human oversight, conformity assessment. The high-risk calendar has slipped, which is its own scandal, but the architecture is product-safety architecture. In U.S. states, the political energy this year went to companion chatbots, addictive feeds, and disclosure. A patient-facing symptom bot in California may owe an independent child-safety audit if a minor can reach it, and a transparency blurb if an adult can. The same bot, pointed at a care pathway in the Union, is a different legal object.
Daria Bahrami, head of policy at Dreadnode, described the operational consequence. AI is accelerating work the sector was already doing. Results arrive faster. The risk conversation that used to live on a quarterly slide now has a timestamp. Companies have to decide, in real time, what they will tolerate. That is the sentence that should be taped to the wall of every health-system AI review board. The model will not wait for the committee to finish mapping Annex III against the state chatbot statute against the OCR bulletin against the IRB amendment.
Flexible governance is the phrase everyone uses to sound modern. Bahrami was careful: flexible cannot mean skip the pre-deployment assessment of what the tool collects, how long it keeps it, and who can query it. In healthcare that assessment is not optional decoration. It is how you find out the ambient scribe is retaining raw audio, the prior-auth agent can see a neighbor’s claim file, or the discovery model was fine-tuned on a trial cohort that never consented to secondary use.
An agent that leaves the lab does not get a HIPAA mulligan
The panel’s scare story was the right scare story. An OpenAI-powered agent left a test environment and hit Hugging Face. That incident was an industry embarrassment. The same pattern in a hospital is a patient event. Stephen Moon, Snowflake’s global public-sector CTO, said the control has to sit in two places at once: agent governance and data security. The agent can only do what it can reach. Build a semantic layer so the system knows what it is looking at, then enforce the permission at the data layer, not in a prompt that says “please do not open the wrong chart.”
That is the only architecture that survives contact with a curious agent. Prompt-level manners are not access control. If the agent can retrieve diagnoses, prescriptions, and notes, you have given it a badge. Treat it like a workforce member: least privilege, logging, break-glass, kill switch, and a human who is accountable when it writes a refill it should not have written. Healthcare already knows how to do this for people. It keeps pretending software that acts is still “just a model.”
A rogue agent in this sector does not leak a repo. It can touch the record, the pharmacy queue, the scheduling engine, and the message that goes home to a family. The blast radius is clinical. That is why “we will monitor outputs” is not a control. Monitoring is how you write the incident report. Containment is how you do not write it.
The other failure mode is the press conference
Kumble’s second warning does not get enough airtime. Incidents produce coverage. Coverage produces panic. Panic produces rules written for the headline rather than the failure. He wants fact-based communication next to the technical guardrails, because once the conversation leaves the facts it is almost impossible to pull back. He is not wrong. He is also describing a sector that has earned some of the suspicion. If your first public sentence after an agent touches the wrong chart is “no evidence of patient harm” before you have finished the audit, you are participating in the panic cycle you claim to fear.
Proportionate communication is part of governance. So is refusing to launder a clinical incident into a brand exercise. The public can hold two ideas: the Hugging Face breach was not a hospital breach, and the hospital version is foreseeable. Treating every outage as extinction, or every near-miss as a nothing, are both ways to lose the room.
What to build before the next statute lands
Do not wait for Washington, Brussels, and Sacramento to pick a single theory of health AI. They will not. Build the layer that survives all three.
Inventory models by use case, data classes, and decision type — discovery, diagnostic support, administrative, patient-facing, agentic. Map each one against HIPAA, applicable state privacy and chatbot rules, GDPR/transfer tools, and the AI Act role you actually occupy (provider, deployer, or both). Stop pretending a vendor BA agreement is an AI governance program.
Stop preparing a unique data extract for every pilot. That process does not scale, and Kumble already told you it will not. Stand up one AI-ready health-data environment with purpose limitation, lineage, role-based and attribute-based access, and a documented de-identification standard the counsel team will defend in two jurisdictions.
Require a pre-deployment review that names retention, training-use, human fallback, and what happens when the agent tries to leave its box. Put the kill switch in the infrastructure, not the policy binder.
Write the incident script now. Who calls OCR, the notified body, the state AG, the IRB, the sponsor. What you will say in the first hour that is true. What you will not say.
And if you sit on the industry side of Kumble’s consensus-standards idea, write safeguards that a regional hospital can implement, not just a company that has a vice president of enterprise data science. Otherwise the patchwork does not get replaced. It gets privatized.
Healthcare AI will keep finding targets, drafting notes, and routing patients. The law will keep arriving in pieces. The organizations that treat that mismatch as a reason to ship without a consistent benefit are the ones that will teach regulators, the hard way, why the next statute is even less coherent than this one.