FDA seeks public input on regulating generative AI medical devices as healthcare enters a new era

The U.S. Food and Drug Administration (FDA) has opened a public consultation on one of the most challenging questions facing modern healthcare technology: how should regulators oversee medical devices powered by generative artificial intelligence (GenAI)? The agency is accepting comments until October 19, 2026, as it considers new approaches to risk assessment, premarket review, and post-market monitoring of AI-enabled products.

The consultation represents a significant milestone in the evolution of digital health regulation. While the FDA has already authorized more than 1,000 AI-enabled medical devices, most use conventional artificial intelligence models rather than generative systems capable of producing novel text, images, recommendations, or other outputs. According to the FDA, generative AI-enabled devices offer transformative potential for patient care but also introduce unique risks that existing regulatory frameworks may not fully address.

Why generative AI is different

Traditional medical software generally operates within predictable boundaries. A diagnostic imaging algorithm, for example, is typically trained to identify specific patterns and generate consistent outputs from similar inputs.

Generative AI operates differently. These systems can accept open-ended prompts, perform multiple subtasks, and produce variable responses even when presented with similar information. Many are built upon large foundation models developed by third parties, adding another layer of complexity regarding transparency, validation, and accountability. This variability creates a fundamental regulatory challenge. A device that generates clinical recommendations, summarizes patient records, drafts treatment plans, or assists healthcare professionals in decision-making may not always respond in exactly the same way. Regulators therefore face questions that extend beyond traditional device evaluation.

The risk is not simply technical failure. Generative AI systems can produce convincing but incorrect outputs, sometimes referred to as “hallucinations.” In healthcare settings, such errors could potentially influence clinical decisions and affect patient safety. At the centre of the FDA’s discussion paper is a proposed two-axis risk framework designed specifically for generative AI-enabled medical devices. The concept evaluates both the nature of the software’s function and the potential consequences of reliance on an incorrect output. Under this approach, a system providing informational support to clinicians might face different regulatory expectations than a device capable of initiating autonomous actions. Likewise, software associated with low-consequence administrative functions would be viewed differently from systems influencing critical medical decisions.

The FDA believes such an approach could better align regulatory oversight with actual patient risk rather than treating all AI tools in the same manner. For developers, this could ultimately create a clearer pathway to market while maintaining safeguards for higher-risk applications.

One of the most interesting aspects of the FDA’s proposal is its exploration of a competency-based evaluation model for generative AI devices. Rather than relying solely on conventional validation approaches, the FDA suggests a framework inspired by how healthcare professionals themselves are trained and assessed. Devices could undergo non-clinical benchmarking to evaluate knowledge, safety behaviour, communication quality, robustness, and generalizability before moving to clinical confirmation studies. The goal would be to demonstrate not only that a system performs effectively under controlled conditions but also that it behaves appropriately in realistic clinical scenarios. This reflects a broader recognition that AI systems cannot always be assessed using the same methodologies developed for conventional software because their outputs may vary based on context and user interaction.

Perhaps the greatest challenge lies beyond initial approval. Historically, medical device reviews have focused heavily on premarket evidence. However, generative AI systems may evolve through updates, changes to underlying foundation models, shifts in real-world data, or altered patterns of use. As a result, the FDA is actively seeking input on postmarket monitoring strategies. Potential approaches include periodic re-benchmarking, ongoing review of real-world interactions, surveillance for performance degradation, and monitoring for changes in model behaviour over time.

This reflects a growing acceptance that AI oversight may need to become a continuous lifecycle activity rather than a one-time regulatory event. For healthcare providers, this could mean greater confidence that AI-enabled systems remain safe and effective after deployment. For manufacturers, however, it may introduce new obligations relating to data collection, monitoring, quality management, and transparency.

Implications for the healthcare industry – including Canada

The FDA’s consultation arrives against a backdrop of rapid AI adoption across healthcare. Generative AI is already being explored for clinical documentation, patient communication, medical imaging support, workflow automation, drug development, and decision-support systems. Major technology companies and healthcare organizations are investing heavily in the field, while regulators globally are struggling to define appropriate oversight models.

The FDA’s initiative may therefore have influence beyond the United States. The agency’s approach frequently serves as a reference point for regulators worldwide, including Health Canada, the European Medicines Agency, and the UK’s Medicines and Healthcare products Regulatory Agency (MHRA).

The consultation could help establish regulatory principles that shape international discussions on AI governance for years to come.

Canadian healthcare systems face increasing pressure from workforce shortages, growing patient demand, and the need to improve efficiency. Generative AI offers potential solutions through automation, clinical decision support, and enhanced patient engagement. However, the same concerns identified by the FDA apply equally to Canadian healthcare providers. If AI-generated recommendations are inaccurate, biased, poorly validated, or insufficiently monitored, patient safety could be compromised.

Canadian regulators are likely to watch the FDA process closely as they develop their own approaches to AI-enabled medical technologies. Harmonization with FDA expectations could ultimately simplify market access for device manufacturers operating across North America while supporting a consistent standard of patient protection.

Importantly, the FDA’s discussion paper is not a new regulation, nor does it establish binding policy. Instead, it represents the beginning of a broader conversation between regulators, clinicians, researchers, technology companies, and patients. Yet the consultation signals that regulators recognize generative AI as fundamentally different from earlier generations of software. The challenge is to create a framework that encourages innovation while ensuring safety, effectiveness, and public trust.