Scientists want tighter controls after AI containment failures

After a series of high-profile AI containment failures in recent months, more than 400 scientists, including Canadian AI pioneers Yoshua Bengio and Geoffrey Hinton, are calling for stronger safeguards.

In the open letter, published Oct. 13, the 489 (as of Oct. 16) signatories warn that “future incidents will be far more damaging to our exposed critical infrastructure” unless developers change how they build and deploy AI.

Other Canadians who signed the letter include Nicolas Papernot and Catherine Régis, co-leads of the Canadian AI Safety Institute Research Program at CIFAR, and Gillian Hadfield, professor at Johns Hopkins University, faculty member at the Vector Institute, and a member of Prime Minister Mark Carney’s new National Council on Artificial Intelligence.

While the researchers don’t name specific incidents, they say ‘multiple organizations’ have reported AI models breaking out of test environments and compromising other organizations’ systems.

In July, OpenAI disclosed that two of its models hacked Hugging Face, an incident that raised questions about how developers can keep AI agents contained when they find unexpected ways to do what they’re told.

Anthropic and Meta have also disclosed recent hacks of external systems.

The open letter calls for “improved AI alignment, leading to more predictable model behavior.” 

The group’s immediate, short-term recommendation is that “stronger” sandbox technology, including airgapping (i.e. disconnecting from the internet) should be the default requirement to ensure better containment.

When these models need internet access, there needs to be stronger restrictions on what they can do, taking into account how reliably developers can predict their actions. As the letter says:

“For all internet-connected AI models, the range of model actions permitted by a containment system needs to be balanced with our currently limited ability to predict that the model’s actions will not result in a security failure (e.g., due to misaligned goals).”

Additionally, the letter calls for an international body, with government and AI and cybersecurity experts, to establish standards for containment and to collect reports of security failures. 

Those reports would eventually become public, with learning and accountability as the goal. 

Canada has already raised the possibility of international AI oversight. 

In September, Minister of Artificial Intelligence and Digital Innovation Evan Solomon said he had discussed a proposed “technology stability board” with G7 counterparts, to evaluate models for testing.

No such body has been established, but Carney’s AI council is mandated to help improve AI safety. 

Final shots

• The researchers warn that without stronger safeguards, future containment failures could cause serious damage to critical infrastructure.
• The letter calls for immediate action on containment while research continues into making AI behaviour more predictable.
• Scientists want international standards and mandatory reporting of containment failures, with those reports eventually made public.