The AI procurement question hiding in your doctor’s office

Ross Mitchell has two questions he wants every Canadian to put to their family doctor at their next appointment.

Where is your data being stored, and is it being used to train the vendor’s model?

If the doctor can’t answer both, Mitchell says, the clinic may have a legal problem it doesn’t know about yet.

Most family doctors won’t know. The vendor contract, the architecture, and whoever decided to let an AI tool near patient data are where the answers are.

These questions apply to any Canadian organization buying AI. Where does the data go, who can use it, and who has to explain it later if the answer turns out to be ugly?

On June 4, Prime Minister Mark Carney launched AI for All, Canada’s new national AI strategy. Its sovereign AI pillar focuses on compute infrastructure under Canadian governance, along with Canadian AI researchers and talent. 

Mitchell’s team in Edmonton has already been testing what that looks like inside a health system.

Mitchell is the Alberta Health Services (AHS) chair in AI in health and a professor at the University of Alberta, where he leads AI adoption work inside AHS. He came back to Canada in late 2022 after a decade working on AI in healthcare in the U.S.

At Upper Bound in Edmonton last month, he walked through the AI scribe his team built inside the AHS firewall, why they built it instead of buying one, and why a clinical scribe is also a procurement story.

The tool is called Jenkins. You may have seen a sign about AI scribes at your own family doctor’s office, or noticed a small device on the counter. Jenkins is one of those. It runs on Snowflake inside a virtual private cloud, within AHS’s protected environment, and doctors launch it from inside Connect Care, Alberta’s provincial electronic medical record.

Almost 1,000 users across 12 specialties have used it, logging more than 80,000 patient sessions and 12,000 hours of recordings. The system has been hitting about 600 sessions a day. 

During the emergency department pilot, about 200 physicians used Jenkins across the province for 22,000 sessions, with a roughly 20% productivity gain per shift.

The patient audio stays inside AHS’s secure environment. A lot of commercial AI scribes aren’t built that way.

The same two questions apply whether the regulator on the other end is a privacy commissioner, a financial services watchdog, or a provincial law society.

Why build something you could buy

Most AI scribes are commercial products. A clinic signs up, a doctor records the visit, and their notes come back with a myriad of baffling hallucinations.

Where the data lives after that depends on the contract, the vendor, and the jurisdiction. Too many clinics don’t know the answer, but it’s the same blind spot that shows up in every industry buying AI tools right now.

“You want to know if the scribe vendor that sold the scribe to the clinic is using the data on the back end to train their system,” said Mitchell.

The right answer, according to Mitchell, is that the vendor can say exactly where the data is stored, who has accessed it, whether it can be deleted, and that the answer to the training question is no.

“I have not consented to you, as a patient, to use your data to improve that foreign company’s model,” Mitchell said.

Swap “patient” for “customer” or “client” and the sentence works for almost any technology leader in the country.

In May, Ontario’s auditor general found that every one of the 20 AI scribes approved for use in the province had accuracy issues in testing. Nine fabricated information, 12 captured a different drug than prescribed, and 17 missed important mental-health details.

The auditor’s office and the province disagreed on how serious the operational implications were, but the report made the vendor risk harder to dodge. Commercial AI tools can fail in ways that matter, and the buyer is the one left holding the question.

Jenkins was built to avoid that whole conversation. Its open-source version, Berta, gives other organizations the architecture to run a similar tool on their own infrastructure, released under an Apache 2.0 licence. Any company can take the source code, commercialize it, and owe the team nothing.

“If you’re a company, you can download our source code and turn around and sell it to people, and you don’t owe us anything,” said Mitchell. “This is by design.”

Mitchell said the system currently uses GPT-5, with other models available on the back end, and the team estimates it runs at well under $30 per seat per month on Snowflake.

Mitchell’s team built data control into the system instead of fighting for it in the paperwork.

Who’s accountable when the model gets it wrong

No AI tool gets it right every time.

Every Jenkins note is reviewed, edited if needed, and signed off by the physician before it goes into the patient record, he said.

Mitchell’s team treats the failure modes as three separate problems. 

Hallucinations are when the model adds something that wasn’t said, omissions are when it leaves out something that should be in the note, and factual errors are when it gets a dose, measurement, or other detail wrong. Each has a different cause and a different fix.

The team has a grant from the Canadian AI Safety Institute to build agentic systems that review the transcript and final note, identify errors, and help create an open benchmark for scribes. AI for All committed a further $50 million to expand the Institute as part of the national strategy.

Unless someone can say what ‘better’ means, “my scribe is better than your scribe” is a fairly useless metric. 

Any regulated organization using AI has to answer the same boring, expensive, absolutely necessary questions about who reviews the output, who signs off, and who’s accountable when the model sounds confident and gets it wrong. 

Why give it away

The open-source release is where this gets interesting.

Berta is a working reference architecture that other organizations can run on their own infrastructure, with an AWS account and a handful of configuration questions. The practical case is the cost, but the sovereignty case is bigger.

An organization that runs the code itself knows where the data is stored, who accessed it, what it was used for, and how long it lives. It doesn’t have to hope the vendor’s answer is good enough after the contract is already signed. For a CIO defending an AI deployment to a board, that’s the difference between showing the work and hoping for the best.

AI for All’s “build-partner-buy” approach tells organizations to build sovereign capability first, partner with allies if they can’t build, and buy only when neither works. Amii has pointed to Jenkins as the kind of deployment the strategy is built to scale.

Ottawa is now writing the broader version of those two questions into national strategy, years after Mitchell started asking them in a clinic.

More AI contracts in the country are going to face the same questions, some sooner than others.

Final shots

• The two vendor questions (where does the data live, and is it training the vendor’s model) are simple enough to ask in a doctor’s office and sharp enough to use in any AI procurement meeting.
• Sovereignty is determined by how a system is built. The cost of getting it wrong shows up after everyone has already moved on to the next pilot.
• The architecture is moving faster than the rules around it, which means organizations deploying now need to act as if future scrutiny is already coming.