AI companies need to show their work

Does hearing that “AI could wipe out all of humanity” make you get a little sweaty with panic?

Cohere CEO Aidan Gomez says it’s a “doomsday narrative.”

One of the first panels at ALL IN 2026 in Montreal was about AI sovereignty, and how middle powers can align their AI interests, beyond dependence on the world’s most dominant tech powers (*cough* the U.S. *cough*). 

The event’s Country of Honour this year is Germany, following the Canada-Germany Digital Alliance announced in December 2025.

But right out of the gate, the issue of public trust and safety was punted to the beginning of the panel. 

And rightfully so. 

What happened

Discussion moderator, CBC’s Catherine Cullen, opened by citing former AI researcher Jacob Coxen. He recently went viral on social media, writing on X that “the people building AI earnestly believe that it could kill us all by the end of the decade.” 

Gomez takes issue with turning that fear into a number.

“What concerns me about these doomsday narratives is that people like to assign specific probabilities,” he said. “That’s not a mathematical result, and it’s extremely misleading to the public to throw out these numbers that are just like, your gut vibe check on what might be coming, as though it’s a concrete quantitative measurement.” 

He said he’s more focused on risks that companies and governments can already observe and measure, including cyberattacks and the potential use of AI systems to help develop biological weapons.

Gomez also criticized part of a proposal from Anthropic CEO Dario Amodei, published last week. 

In calling for AI labs to work together on slowing “the pace at which we improve the capabilities of AI models,” 

Amodei said that U.S. antitrust laws could prohibit such collaboration. He said the government would need to issue a “narrow waiver” or exemption for these companies to work together on safety conversations.

“I think that is a huge red flag,” Gomez said, arguing for a government-led process that brings stakeholders together, keeping the “reality” of the tech in front of them.

“It shouldn’t be a few companies in Silicon Valley setting the bar and trying to pull up the ladder behind them and implement regulatory capture,” he added.

Who was there

Joining Cullen and Gomez, Canada’s Minister of Artificial Intelligence and Digital Innovation Evan Solomon said they’re taking these risks seriously, and that we need to be “pragmatic about building reliable AI,” capturing benefits while maintaining safety. 

Also on stage was German Federal Minister for Digital Transformation and Government Modernisation Karsten Wildberger, who said that we need to use the moment to “work on safety by design.” 

Aleph Alpha co-founder and co-chief research officer Samuel Weinbach rounded out the panel. 

Cohere and Aleph Alpha just signed a definitive agreement to merge the two companies, five months after the two companies first announced the deal in April.

Weinback came out against letting fear set the terms of the conversation.

“What we need is transparency,” he said, calling for researchers and policymakers to define risks, measure them, and decide what action should follow.

The takeaways

When Cullen asked Gomez where the focus of AI safety should be, he had a list of three tasks.

First, define the risks. Governments and companies need to decide which threats they are trying to mitigate and establish ways to measure them. 

Second, test the models. Every organization developing advanced AI should carry “a burden of proof” that its models are sufficiently safe across the risks governments decide to track.

Third, require transparency when something goes wrong.

“Make it public,” Gomez said. “Show how it failed.”

He pointed to OpenAI’s disclosure of an incident involving Hugging Face as an example of why failures should be visible. Gomez said the incident exposed weaknesses in the sandbox where the model had been placed.

“If you do those three things, we’re in an infinitely better position than we are today,” he said.

What to think about if you’re a tech leader

Forget the straight-outta-sci-fi doomsday risk-of-human-extinction point.

(It’s hard, I know, but stick with me.)

Gomez shared the stage with two government ministers, but his three points can be turned into a set of questions for vendors.

Starting with the risks, ask what kinds of failures the vendor tests for, how those tests work, and what evidence they can provide so the model meets its own safety thresholds.

Then, ask what happens when something goes wrong. 

Does the provider disclose incidents? What information will customers receive? How quickly? What will it tell you about the conditions that produced the failure?

When a model has access to sensitive data, internal systems, or tools that can take actions on an organization’s behalf, these questions can’t really be swept under the rug.

What to share with your C-suite

Higher-ups are going to hear plenty about whether AI could pose an existential threat. It’s hard to ignore the screaming headlines dominating the news right now.

Stepping down a little closer to the day-to-day, the fight is over who gets to write the rules. As Gomez said, a handful of major U.S. labs shouldn’t get to decide that on their own.

Technology carries risk. That part is never really up for debate. 

But what’s on the table is which risks deserve the most attention, how to prove they’re being managed, and who gets to write the rules.

Digital Journal is at ALL IN in Montreal this week. Follow our coverage here.

Final shots

• Ask vendors which AI risks they measure and how they test for them.
• Put incident disclosure into procurement and governance conversations before deployment.
• Safety claims are more useful when an organization can show the evidence behind them.