Canada’s Privacy Regulators Wrote the Health AI Audit Checklist Before the Law Exists

Canada’s privacy regulators have put the health sector on notice that artificial intelligence will be judged by the same privacy rules as every other system that touches patient information, and that those rules need to get stronger. On October 29, 2026, the federal, provincial and territorial privacy commissioners and ombuds published a joint resolution on the responsible use of AI in health care and health research. They adopted it in Ottawa on September 15, 2026, at their annual meeting, hosted by the Privacy Commissioner of Canada and the Information Commissioner of Canada.

The resolution is not a law and creates no new obligations by itself. What makes it worth reading closely is how specific it is. It does not stop at asking governments to legislate. It splits the work among four groups (governments, AI developers and service providers, health care providers, and the research community) and tells each one what the regulators expect. For a vendor selling an AI scribe to a clinic in Ontario, or a hospital in Alberta weighing a diagnostic tool, it reads like an audit checklist the regulators have published in advance.

This piece walks through what the resolution asks of each group, where it lines up with laws already in force, and how to turn it into evidence an organization can show a regulator.

Why the regulators acted now

The resolution opens with the pressures every Canadian health system knows: rising costs, an aging population, shortages of health care workers and delays in access to care. Governments and providers are turning to AI to stretch limited resources, and the regulators accept that. Their concern is pace. In the words of the resolution, AI systems “are being widely developed and implemented at an unprecedented rate across Canada,” and the health sector “must act now” to protect personal information and keep patient trust.

The document names two families of harm. The first is unauthorized collection, use and disclosure: AI systems that combine de-identified datasets with other information in a way that re-identifies people, models trained on illegally sourced data, outputs that are commercialized or disclosed without authority, and large volumes of information processed without adequate security. The second is algorithmic bias and patient harm: noisy or unrepresentative training data that produces wrong predictions, false diagnoses and poor treatment plans, with the greatest risk falling on groups that were under-represented in the data.

The resolution also makes a point that matters for contracting. Health data in Canada moves across federal, provincial and territorial lines and through public and private hands. A hospital that builds its own model is both a provider and a developer. An AI company that runs its own studies is both a developer and a researcher. The regulators say each party must meet every legal duty that attaches to each role it plays. That cuts off a familiar argument, the one where a vendor says privacy is the hospital’s problem and the hospital says the model is the vendor’s.

The six principles

The resolution asks everyone involved to make sure AI systems in health are, across their whole life cycle:

• Privacy protective, collecting, using, disclosing and retaining only what the law permits.
• Valid and reliable, producing outputs that are accurate for the people they are used on.
• Safe, so that errors do not translate into harm to patients.
• Human rights affirming, guarding against discrimination and systemic bias.
• Transparent, so that patients and clients can understand what the system does with their information.
• Accountable, with someone answerable for each decision and a way to challenge it.

The regulators draw these from the principles for responsible AI published by Ontario’s Information and Privacy Commissioner and the Ontario Human Rights Commission in January 2026. The life cycle language is deliberate. It runs from design and development through procurement, deployment, use and monitoring, to decommissioning. A tool that passed a privacy review at purchase and was never looked at again does not meet that standard.

What governments are asked to do

The calls to governments are the part most news coverage has focused on. Federal, provincial and territorial governments should review, amend or adopt laws that balance support for innovation against the six principles. They should consult the privacy regulator in their jurisdiction when they draft those frameworks. They should engage the public often and openly, including patient and family committees and people from marginalized communities. And they should legally require everyone who develops, procures or uses AI in health to have an AI governance and accountability framework built on real risk assessment.

That last call is the one with teeth if governments take it up. Today, most Canadian health privacy statutes regulate personal health information without saying anything specific about AI. A legal duty to maintain an AI governance framework would turn what is now good practice into something an auditor can check. It also connects with federal reform already in motion. Bill C-36, the proposed Protecting Privacy and Consumer Data Act, which was introduced and given first reading in the House of Commons on June 15, 2026, and as of late October 2026 is listed by Parliament as at second reading, would add privacy impact assessments and explanations of automated decision-making to federal private-sector law if Parliament passes it. Our guide to the 10 principles of PIPEDA covers the federal baseline that applies today.

A numbered checklist for AI developers and service providers

The developer section is the most detailed in the resolution, and it is where companies selling into Canadian health care should look first. Here is what the regulators expect, translated into tasks a product and privacy team can own.

1. Minimize and substitute data. Follow data minimization, and use de-identified or synthetic data instead of identifiable personal information wherever possible. Our explainer on what counts as de-identified data is a useful starting point, since the resolution itself warns that de-identified data can be re-identified when combined with other sources.
2. Know where training data came from. Be able to show the provenance of the information used to train each system and that it was lawfully sourced.
3. Build legal limits into the system. Put controls in place so each system only collects, uses, discloses and keeps personal information as privacy law allows, with safeguards proportionate to the amount and sensitivity of the data.
4. No manipulative design. Avoid interface patterns that push users into decisions against their privacy interests, and ship the most protective privacy settings as the default.
5. Make the system testable. Enable auditing and routine real-world testing to confirm the system performs as intended.
6. Fight bias with data quality. Keep training data accurate, representative and complete, and validate continuously to catch biased models and mistakes before they reach patients.
7. Keep humans in the loop. Adopt a governance framework covering the full life cycle, with human oversight and a way for clients and end users to challenge outputs.
8. Assess continuously and share the results. Run privacy, security and AI-specific assessments throughout the life cycle, give clients evidence of how the system was trained and a general explanation of how it works, demonstrate that its use complies with privacy law, and tell clients about ongoing findings and the mitigations they need to apply downstream.

Items 7 and 8 change the sales conversation. The regulators are telling hospitals to ask for this material, and telling vendors to hand it over. A vendor that cannot explain the factors and sources its model relied on, or that treats its risk assessments as confidential, will find procurement harder.

What health care providers are asked to do

The provider section has ten calls. Several restate existing law, such as complying with the privacy statute in the provider’s jurisdiction and limiting collection to what is necessary. Others go further and are worth singling out:

• Consent and opt-out. Where privacy law requires consent, patients should get clear communications about how their information will be used with the AI system, and, where practicable, a chance to opt out before the system is deployed.
• A named contact and a challenge route. Designate a person patients can raise concerns or objections with, and give them a practical way to challenge consequential decisions or inferences.
• Vendor assessments that can stop a purchase. Consider the results of the developer’s privacy, security and AI assessments, and let the risk of breaches, inaccuracy, bias and discrimination decide whether the system is adopted at all.
• Contracts with limits. Exercise “utmost diligence” in contracting and subcontracting, with terms on training, record keeping, auditing and limits on use and disclosure, including a ban on unauthorized commercialization of the data.
• Shadow AI controls. Train staff and stop the use of AI tools without approval, such as a hospital employee using a personally procured tool that does not meet hospital policy.
• Human verification of outputs. Keep people reviewing outputs to catch results that are inaccurate, discriminatory or harmful.

The shadow AI example comes from guidance on AI scribes that Ontario’s commissioner published in January 2026. Scribes are the clearest case of a tool that can spread through a clinic one physician at a time. We have covered Canada’s uneven approach to AI scribes in the exam room and an Ontario scribe incident that exposed patients, and both show why the regulators want approval and oversight in place before the tool is on every laptop.

The research community

The resolution asks the Panel on Research Ethics and Canada’s three federal research funding agencies to update the Tri-Council Policy Statement (TCPS 2) for AI and AI-assisted research, including more granular guidance on re-identification risk when de-identified, anonymized or synthetic data is used, and on bias caused by missing populations. It asks institutions to give research ethics boards the resources and training to review AI projects, and it asks those boards to require a “persuasive demonstration” that benefits outweigh risks. Researchers are asked to refine their risk-benefit assessments and to publish outcomes.

Where this sits in Canada’s patchwork

Health privacy in Canada is mostly provincial. Public hospitals and clinicians fall under provincial health information statutes, private clinics and vendors may fall under PIPEDA or a substantially similar provincial private-sector law, and federal institutions fall under the Privacy Act. That is why this resolution comes from every regulator together, and why the regulators commit to coordinating enforcement “to the extent possible.” We have argued before that health care AI has a jurisdiction problem more than a governance gap, and this resolution is the regulators’ answer to that problem: one set of expectations, applied through many statutes.

In the announcement, Privacy Commissioner Philippe Dufresne said governments, regulators and service providers “must work together” so that these technologies are developed and used “with strong privacy and data protection frameworks.” The regulators also say what they will do: keep calling for legislation, take steps to ensure compliance within their mandates, coordinate action across jurisdictions, issue guidance and educate the public about their rights. Companies should read “take steps to ensure compliance” as a signal that investigations of AI in health will refer back to these expectations. A finding that a hospital deployed a tool without assessing it, or that a vendor could not account for its training data, would now be measured against a published standard.

Turning the resolution into evidence

A policy that says “we follow the six principles” proves very little. The useful question is what a regulator would ask to see. For most organizations, the answer is a short list of records:

• An inventory of AI systems in use, including tools staff adopted on their own, with an owner for each.
• A data map for each system showing what personal health information goes in, which vendors and subprocessors receive it, where it is stored, how long it is kept and whether it is used to train or improve the model.
• A privacy impact assessment and an AI risk assessment for each system, dated, with the mitigations that came out of them and evidence they were implemented.
• Vendor files holding the developer’s training-data provenance statement, explanation of how the system works, assessment results and contract clauses on use limits and commercialization.
• Patient-facing notices and, where needed, consent and opt-out records.
• Logs of human review and of patient challenges, with outcomes.

The weak points are predictable. An assessment is not proof that the controls it recommended work. A contract clause banning secondary use is not proof the vendor’s pipeline honours it. A notice on the patient portal is not proof that the scribe on a physician’s phone was approved. Each of those gaps needs its own evidence.

Where Captain Compliance fits

Captain Compliance’s healthcare privacy software covers part of this work: identifying the trackers, chat tools, SDKs and third-party services running across health websites, portals and apps, connecting them to data maps, vendor records and PIAs, and keeping timestamped consent and notice records. It does not validate clinical models, test them for bias or decide whether information is personal health information, and those calls still require clinical, technical and legal review. What it can do is make sure the data flows around an AI deployment are documented and that the assessments are tied to what is actually running.

Frequently asked questions

Is the resolution legally binding?

No. It is a statement of expectations and a call to action from Canada’s privacy regulators. The binding rules are the existing federal, provincial and territorial privacy and health information laws, which the regulators enforce within their own mandates.

Who adopted it, and when?

Canada’s federal, provincial and territorial information and privacy commissioners and ombuds with responsibility for privacy oversight adopted it in Ottawa on September 15, 2026. The Office of the Privacy Commissioner of Canada announced it on October 29, 2026.

Does it apply to vendors based outside Canada?

The resolution addresses anyone developing, procuring or using AI systems in Canadian health care. Whether a given foreign vendor is directly subject to a Canadian statute depends on the facts, but Canadian providers are being told to demand the evidence anyway, so in practice the expectations reach vendors through procurement and contracts.

Does it ban the use of identifiable data to train health AI?

No. It asks developers to use de-identified or synthetic data wherever possible and to be able to show that training data was lawfully sourced. Identifiable data may still be used where the law allows it.

What is “shadow AI” in a hospital context?

The resolution’s example is a hospital employee using an AI tool they procured personally that does not comply with hospital policy. Providers are asked to train staff and take measures to prevent unapproved use.

What should an AI vendor prepare first?

A training-data provenance statement, a plain explanation of how the system works and uses information, current privacy and security assessment results, and a description of the human oversight and challenge mechanisms built into the product. Those are the items the resolution tells providers to ask for.

The regulators were careful to frame all of this as support for innovation, not a brake on it. But they have now written down, in detail, what a responsible deployment looks like, and they have committed to coordinating how they act on it. The first investigation into a health AI deployment after this resolution will show whether it is guidance or the opening outline of a case.