That sequence is how organizations buy software. It is a poor way to buy a system that will ingest patient conversations, chart data, or billing text and send any of it outside the four walls of the health system.
The algorithm is rarely the hardest part. The hard part is the data path: what leaves the encounter, who receives it, what the vendor keeps, what the tool invents, and what happens when the product grows six months later. Treat the project as an AI mystery and you will argue about models. Treat it as a data-sharing project and you can still change the deal.
The Tool Looks Narrow. The Data Path Is Not.
Take an ambient scribe that listens to a visit and drafts a note. The stated purpose is documentation burden. The legal reflex is to get a business associate agreement and move on. That is necessary. It is not a map of the flow.
Does the vendor receive raw audio, a transcript, or both? Is audio stored or processed and discarded? Can vendor staff listen for quality review? Is encounter text used to improve the product or train other models? Which subprocessors sit in the path? What happens when the patient talks about substance use, reproductive care, behavioral health, or another category that carries extra legal and reputational weight?
The same questions apply when the AI is less visible. A patient-portal assistant that drafts replies from the chart. A revenue-cycle engine that reads notes and suggests codes. An outreach model that blends clinical, demographic, and utilization data. Each one opens a new route for access, transformation, or disclosure. The interface is new. The privacy problem is old: know where information travels and what is done with it.
A BAA Is a Floor, Not a Picture of the Product
Healthcare teams are trained to ask whether the vendor is a business associate. That question still matters. It does not tell you how the product actually behaves.
Contracts often allow use of information for management, administration, or aggregation. Product terms may separately say customer data can be used to maintain or improve the service. Technical docs may show logs, cached prompts, or human review during troubleshooting. None of that is automatically unlawful. All of it is material. If sales, the MSA, and the architecture tell three different stories, that is not a tidy-up item for legal redlines. It is a sign the organization does not yet know what it is authorizing.
Privacy review should therefore cover proposed data use, configuration options, and the real workflow — not only the PDF attached to the procurement ticket. Before signature, someone should be able to answer, in one coherent account:
- What data does this use case actually require?
- Who receives it, including subprocessors?
- What new records or scores will the tool create?
- How long will inputs and outputs remain available?
- Can any of it be used beyond delivering the contracted service?
If those answers only appear after go-live, the organization has already given up its cheapest leverage.
Minimization Has to Happen While the Scope Is Still Soft
AI products reward volume. Privacy programs ask whether the volume is needed. Vendors will often request the whole record because the model can take it. Operations will ask for years of history to “improve performance.” A pilot that was supposed to use a limited extract becomes production access because nobody revisited the original scope.
The useful intervention is early and blunt: what is the least information this specific job requires? That conversation can cut fields, exclude sensitive encounter types, shorten retention, or keep a pilot off production systems. It can also expose a worse problem — that no one has defined what success looks like, so the default is to share everything and hope the model figures it out.
Minimization is not paperwork. It ties sharing to a purpose the organization can defend to patients, regulators, and its own board.
Outputs Are Records Too
AI does not only consume PHI. It produces summaries, risk scores, draft messages, and suggested codes. Those outputs enter clinical and administrative work whether or not anyone files them in the official chart. They can influence a decision while looking more finished than they are.
Governance has to cover the output layer. Who may see it? How is it checked? Can a patient obtain it? Is it stored in a system that can be audited? What is the process when the output is wrong, incomplete, or built on information that should never have been in the prompt?
Those are organizational decisions, not vendor features. The covered entity still owns how the result is used and where human review sits. A mature process makes the “recommendation only” limit visible in the workflow. Relying on a busy clinician to remember a footnote from last quarter’s training is not a control.
The Product You Approved Will Not Stay That Product
The version that clears contracting is not the version the workforce will be using after the next few releases. Vendors add features. Models change. Someone connects the tool to a new system because it is convenient. A product bought to summarize internal documents ends up sitting on patient messages.
Annual vendor recertification is too slow for that pace. Someone has to own material changes: new data uses, new integrations, new model behavior that affects patients. Workforce members need a simple way to flag a tool that is doing something it was not approved to do. That does not mean a committee for every patch. It means a named owner, a short escalation path, and enough monitoring to notice when risk has moved.
Bring Privacy In Before the Calendar Is Frozen
Privacy teams do not need to score models. They need to understand movement of information, the gap between legal permission and patient expectation, and how a small operational yes becomes a standing disclosure pattern.
That work is cheapest before the vendor is selected and the implementation date is announced in a leadership meeting. After the contract is circulating for “final review,” you can still cut risk. You will do it with fewer options and more political cost.
Healthcare will keep buying AI. The mistake is to treat each purchase as an unfamiliar technical puzzle that privacy can bless at the end. Start with the data. Decide what is shared, what is created, who uses it, and how those facts will be watched as the product changes. That is not a sidecar to AI governance. It is the work.