Artificial intelligence is moving quickly into one of the most sensitive places it can operate: the conversation between a person and a healthcare service.
AI medical chatbots can collect symptoms, medication histories, diagnoses, age, lifestyle information and other highly personal details before producing health-related recommendations in seconds. Some tools simply provide general information. Others help triage patients, support clinicians or operate as part of regulated medical products.
For the companies building and deploying these systems in Europe, that distinction matters.
An AI medical chatbot may find itself governed simultaneously by the EU General Data Protection Regulation and the EU AI Act. The two laws overlap, but they regulate different things.
The GDPR asks questions about the personal data: Why was it collected? What is the lawful basis? Is it accurate? How long is it retained? Who received it? Can the patient access it, correct it or have it deleted?
The AI Act focuses more directly on the system: What risks does it create? How was it developed and tested? Can humans oversee it? Are its outputs sufficiently transparent? Is there adequate logging, documentation and risk management?
That sounds manageable on paper.
It becomes much harder when a patient asks a deceptively simple question:
“Why did the AI tell me this?”
Health Data Starts With the GDPR
The GDPR does not treat a conversation about a medical condition like an ordinary customer-service chat.
Health information is a special category of personal data under Article 9 of the GDPR and receives heightened protection.
That means an organization generally needs both a lawful basis under Article 6 and an applicable condition permitting the processing of special-category data under Article 9.
Explicit consent can be one of those routes, but it is not the only possible one.
The GDPR also contains provisions allowing certain processing for medical diagnosis, healthcare and treatment where the statutory requirements and safeguards are satisfied. The answer can therefore depend heavily on who operates the chatbot and what the service actually does.
A hospital deploying an AI assistant inside an established healthcare relationship may be in a different position from a technology startup offering a direct-to-consumer symptom checker.
That distinction is important because discussions about medical AI sometimes reduce the analysis to: “It is health information, so the company needs consent.”
GDPR compliance is rarely that simple.
The AI Act Adds a Second Regulatory Layer
The AI Act does not replace the GDPR.
It adds another set of obligations.
And not every medical chatbot is automatically classified as high-risk AI.
Under Article 6 of the AI Act, certain AI systems associated with regulated products can qualify as high-risk where the statutory classification requirements are met. This is particularly relevant to AI used in or as medical devices, but classification depends on the system’s intended purpose and regulatory status rather than the fact that a chatbot happens to discuss health.
For systems that do qualify as high-risk, the obligations are considerably more demanding.
The AI Act addresses areas including risk management, technical documentation, recordkeeping, transparency, human oversight, accuracy, robustness and cybersecurity.
High-risk systems must also support automatic logging of relevant events. Providers and deployers have specific record-retention obligations for those logs, subject to applicable privacy law.
This is where the two regulatory systems begin to interact.
The GDPR may require an organization to minimize personal data and avoid retaining it longer than necessary.
The AI Act may require extensive records and logs so the operation of a high-risk AI system can later be reconstructed and examined.
Those requirements are not necessarily contradictory. But they do mean companies cannot build their GDPR program and AI governance program independently and assume the two will fit together afterward.
“Why Did the AI Reach That Conclusion?”
One of the hardest issues involves explainability.
Suppose a patient tells a medical chatbot that she is 48 years old, has persistent headaches, takes a particular medication and has a family history of cardiovascular disease.
The system recommends that she seek urgent medical evaluation.
She then asks why.
A conventional computer system might have a relatively simple decision tree:
If condition A and condition B are present, recommend action C.
Modern machine-learning systems can be much harder to explain at that level.
The system may be able to identify the information it considered. It may also provide an understandable explanation of factors relevant to its output.
That does not mean the provider can necessarily reconstruct the precise mathematical contribution of every internal model parameter that resulted in a particular response.
This creates a real GDPR question.
Article 15 gives individuals a right of access to personal data and, in situations involving the automated decision-making covered by Article 22, includes access to meaningful information about the logic involved and the significance and envisaged consequences of the processing.
But there is an important legal qualification.
Not every recommendation produced by an AI system automatically becomes an Article 22 decision.
Article 22 specifically addresses decisions based solely on automated processing that produce legal effects or similarly significantly affect the individual.
Whether a particular medical chatbot crosses that line depends on what it does.
A chatbot offering general wellness information is different from an automated system deciding whether someone receives medical treatment, determining eligibility for a healthcare benefit or making a decision that a healthcare provider relies upon without meaningful human involvement.
Companies therefore need to understand not merely whether their system “uses AI,” but what role its output plays in the actual decision-making process.
Explainability Cannot Be Solved With a Privacy Policy
There is also an important distinction between transparency and technical explainability.
A company can write an excellent privacy notice explaining that it uses an AI system, identifies the categories of information processed and describes the purposes of the processing.
That does not automatically make the model itself understandable.
The AI Act recognizes this problem.
High-risk AI systems must be sufficiently transparent for deployers to interpret their outputs and use them appropriately. Providers must supply information about matters such as the system’s intended purpose, capabilities, limitations, expected accuracy and circumstances that may affect performance.
Human oversight is another part of that structure. The AI Act requires high-risk systems to be designed so that appropriately qualified people can oversee their operation, recognize anomalies and, where appropriate, disregard, override or reverse an output.
For healthcare AI, this becomes especially important.
Displaying a disclaimer saying “this is not a doctor” does not solve the underlying governance problem if the organization knows clinicians or patients are effectively treating the system’s recommendations as medical decisions.
What Happens When the Patient’s Information Is Wrong?
The GDPR also gives individuals the right to correct inaccurate personal data.
In a conventional patient record, the concept is straightforward.
A medical record says a patient is allergic to a particular drug. The information is wrong. The record is corrected.
AI complicates the picture because the same information can exist in several places.
It might appear in the patient’s profile.
It might exist in a conversation history.
It might have been copied into another clinical system.
It could form part of a profile or vector database used for retrieval.
And, depending on how the system was designed, data may have been used for testing, fine-tuning or other model-development purposes.
The original IAPP analysis identifies this as a major problem: correcting information displayed to the user does little good if an AI system continues making recommendations based on stale or inaccurate information somewhere downstream.
That is a valid operational concern.
But it does not mean Article 16 automatically requires a company to retrain an entire foundation model every time an individual corrects a medical record.
The better compliance question is more concrete:
Where does the inaccurate personal data still exist, and where can it continue affecting processing concerning that individual?
If the answer includes downstream databases, profiles, retrieval systems or third parties, correcting the front-end record may not be enough.
GDPR Article 19 also requires controllers, subject to its limitations, to communicate rectification, erasure or restriction to recipients to whom the personal data were disclosed.
For an AI healthcare product, the data lineage behind a correction therefore matters almost as much as the correction itself.
Consent Creates Its Own Problem
Now consider another situation.
A patient voluntarily provides detailed health information to an AI consultation service and explicitly consents to its use.
Later, the patient withdraws that consent.
The GDPR makes clear that consent can be withdrawn, and withdrawal does not retroactively make processing that occurred while valid consent existed unlawful.
But what happens next depends on why the company was processing the information.
If consent was the only applicable legal basis for processing health data needed to provide the AI service, withdrawing it may mean the company can no longer perform that personalized service.
That is not really a loophole or contradiction in the GDPR.
It is the consequence of relying on consent.
The more difficult issue arises when companies use the same patient information for several different purposes.
A person may provide health information because she wants an answer to a medical question.
That does not necessarily mean she also agreed to have the conversation used indefinitely for:
model training;
algorithm improvement;
product research;
behavioral analytics;
marketing;
or development of future products.
The original IAPP analysis recommends separating consent for providing the actual consultation from consent for secondary activities such as model improvement and research.
That separation makes practical sense, but companies should go further than simply adding more checkboxes.
They need to identify a legal basis for each processing purpose, determine whether consent is actually appropriate, apply data minimization and decide what happens to the information when permission for a particular use ends.
Medical AI Makes Purpose Limitation Much More Important
This is one of the areas where AI products can get into trouble quickly.
A company begins collecting conversations to provide medical answers.
Someone later realizes that those conversations would be extremely valuable for improving the model.
A third team wants them for product analytics.
Another wants to study them to identify new healthcare trends.
Technically, those uses may all involve the same dataset.
Legally, they are not necessarily the same purpose.
The GDPR’s purpose-limitation principle exists precisely because organizations should not treat the collection of personal information as an unlimited license to find additional uses for it later.
With health information, the stakes are considerably higher.
AI developers therefore need to distinguish between data required for inference, data retained as part of the patient’s account or medical record, data used to monitor safety, and data used to train or improve future models.
Those categories should not be blurred together simply because they pass through the same AI system.
Bias Becomes Both a Privacy and AI Governance Problem
Medical AI also creates a second risk that is less visible to patients: bias.
If training, validation or testing data inadequately represent particular populations, the system may perform differently across age groups, sexes, ethnic populations or patients with particular medical histories.
Healthcare makes those differences consequential.
An error in a movie recommendation is inconvenient.
An error in symptom triage can affect someone’s decision about whether to seek care.
For high-risk AI systems, the AI Act places specific requirements around data governance, risk management, performance and human oversight. Providers also have to give deployers information that may include performance characteristics for particular persons or groups where appropriate.
The GDPR reaches the problem from another direction.
Accuracy, fairness and transparency are data-protection principles, and automated decision-making involving special-category data faces additional restrictions.
A healthcare organization therefore cannot treat algorithmic bias as exclusively a data-science issue.
It is part of the legal governance of the system.
Medical Chatbots Must Now Tell Users They Are Dealing With AI
There is also a more immediate AI Act obligation.
As of August 2, 2026, Article 50 transparency requirements are applicable. Providers of AI systems intended to interact directly with people generally must ensure users are informed that they are interacting with an AI system unless that fact is obvious in the circumstances.
For a healthcare chatbot, this should be one of the easier requirements to satisfy.
The harder question is what information comes after the disclosure.
Simply telling someone “you are chatting with AI” says very little about whether the conversation is stored, which company receives it, whether another AI provider processes it, whether it is used for model improvement, how long it is retained or whether a human clinician reviews the result.
Those remain privacy-governance questions.
The High-Risk AI Timeline Has Changed
Companies should also be careful with older AI Act compliance timelines.
Following the EU’s 2026 AI Act amendments, the high-risk requirements no longer all take effect on the schedule originally contemplated when the regulation was adopted.
The European Commission now says the rules for Annex III high-risk systems apply beginning December 2, 2027. Rules covering high-risk AI embedded in regulated products, including relevant medical devices, apply from August 2, 2028.
That gives medical AI companies additional implementation time.
It should not be mistaken for permission to postpone the work.
The GDPR already applies today.
And some AI Act provisions, including chatbot transparency requirements, are already applicable.
Organizations building products that are likely to fall into the high-risk category will also need data governance, technical documentation, logging and human-oversight systems that cannot realistically be assembled a few weeks before a compliance deadline.
One Patient, Two Regulatory Questions
The easiest way to understand the overlap is to look at the same event through both laws.
A patient tells an AI system about chest pain.
The GDPR asks:
What personal and health data did the service collect? Why was it lawful to collect it? Where was it sent? How long will it remain there? Can the patient access or correct it?
The AI Act asks a different set of questions:
What kind of AI system produced the recommendation? Is it subject to high-risk requirements? Was it tested appropriately? Can the output be interpreted? Are there logs? Can a qualified human intervene? What happens when the system fails?
A mature healthcare AI program has to answer both.
That means the privacy team cannot stop at the privacy notice, and the AI governance team cannot stop at the model documentation.
The organization needs to understand the entire path from patient input to model output and everything that happens to the data in between.
For healthcare AI, that is where compliance actually lives.