AI Clinical Summarization Tools Sit In 'Regulatory Grey Area', Experts Warn

The EU lacks concrete guidance on whether clinical summarization large language models are considered medical devices. Experts push for EU adoption of the FDA “non-device” category.

Clinical Summarization
• Source: Shutterstock

Clinical summarization large language models (LLMs) are AI tools that help clinicians generate and summarize medical records.

These tools sit in a “regulatory grey area,” Hugh Harvey, managing director at Hardian Health, said in a recent LinkedIn post.

Different jurisdictions have different rules governing the products, or sometimes none at all, said Harvey.

For example, while the US Food and Drug Administration (FDA) has established criteria to regulate clinical summarization tools under a non-device category, the EU has not issued any regulations defining the products.

However, Harvey noted in the LinkedIn post, the EU is notoriously cautious. “I would expect further clarity and guidance from the Medical Device Coordination Group,” he wrote.

Clinicians use clinical summarization tools as a time-saving alternative to notetaking. The tools record physician-patient interactions and collect patient data, then summarize the data to assist the clinician with diagnosis and treatment. The clinician can then review the notes to check for accuracy and add other details.

Last year, Google Research released MedLM, a group of generative AI models for “medical Q&A and summarizations.” The product can be used in the creation of after-visit summaries or history and physical examinations. (Also see "News We’re Watching: GE And Google Announce AI Initiatives, Medtronic Announces Approvals, And More" - Medtech Insight, 9 January, 2024.)

Tech startup, TORTUS, has just secured $4.2m from Khosla Venture for generative AI model OSLER (Operating System Leverage in Electronic Records). OSLER processes physician’s notes in real time, providing a summary and the option for a patient letter.

EU MDR Regulation Of Clinical Summarization Tools

"It could be argued either way whether clinical summarization tools are medical devices or not when referring to the MDR medical device definition, Stephen Gilbert medical device regulatory professor, told Medtech Insight.

If a system is delivering diagnostic and/or therapeutic advice (a medical purpose) developers must clearly delineate the medical intended purpose and follow regulatory approval pathways for the system, said Gilbert. These systems are only medical devices if they go beyond simple database functions, said Gilbert. Systems which process input data through an algorithm to produce different output data, would be considered to be “beyond simple database functions”. (Also see "Chatbots Are Not Ready To Be Medical Devices, Experts Argue" - Medtech Insight, 7 July, 2023.)

Because clinical summarization tools are not actually doing the diagnosis, or deciding treatment, they are unlikely to be considered medical devices, however they are clearly part of the medical process, said Gilbert.

“In my view it is unlikely that an EU regulator would decide clinical summarization tools are part of the diagnosis or therapeutic process, and that they must be classed as a medical device,” Gilbert told Medtech Insight.

Lessons From The FDA

The FDA has adapted its clinical decision support software guidance to include a new “non-device” category.

To be considered a non-device, the software must meet four criteria:

  • The software function does NOT acquire, process, or analyze medical images, signals, or patterns.
  • The software function displays, analyzes, or prints medical information normally communicated between health care professionals (HCPs).
  • The software function provides recommendations (information/options) to an HCP rather than providing a specific output or directive.
  • The software function provides the basis of the recommendations so that the HCP does not rely primarily on any recommendations to make a decision.

Under this guidance, clinical summarization tools would fit the “non-device” category.

A recent Nature Portfolio Journal opinion piece from regulatory experts in Germany and France argued that the FDA’s non-device approach should be adopted by the EU, with careful post-market surveillance.

“If the FDA carries out careful market surveillance of the effects of this policy, then it may introduce enormous flexibility and innovation potential with minimal risk,” the paper said.

However, researchers from the University of Maryland School of Medicine are concerned that FDA guidance provides an “unintentional roadmap for how LLMs could avoid FDA regulation”.

More from Europe

Faster Device Certification Among Keys To Unlocking Innovation In EU MedTech Sector

 

While MedTech Europe sharpens its focus on what the EU most urgently needs to advance the medtech market in four targeted measures, the European Commission has also unveiled four priorities for 2025.

Consumer Healthtech Investment Totaled $4.5BN In 2024, But Bar Is High Amid Economic Uncertainty

 
• By 

Global investment in consumer healthtech increased by 9% year-over-year in 2024, totaling $4.5bn, with significant interest in mental health solutions, according to Galen Growth. While the first quarter of 2025 saw raised confidence and investments, the Trump administration’s new tariffs and sweeping changes to healthcare have introduced new uncertainties.

MedTech Europe’s Bisazza Urges Action As US Tariffs Jeopardize Critical Global Supply Chains

 

The intricate assembly of medical devices, often involving over 1,000 globally sourced components, faces severe disruption due to new US tariffs. These barriers could halt production and devastate small and medium-sized enterprises (SMEs), MedTech Europe’s CEO, Oliver Bisazza, warns in an interview with Medtech Insight.

Don’t Sleep On Inspections: Experts Warn FDA Maintains Enforcement Resources

 

Manufacturers of medical products would be foolish to think that recent upheavals at the US FDA will result in a lack of inspections in the coming years. A panel of experts discussed the current state of inspections during a webinar hosted by the Food and Drug Law Institute.

More from Geography

Consumer Healthtech Investment Totaled $4.5BN In 2024, But Bar Is High Amid Economic Uncertainty

 
• By 

Global investment in consumer healthtech increased by 9% year-over-year in 2024, totaling $4.5bn, with significant interest in mental health solutions, according to Galen Growth. While the first quarter of 2025 saw raised confidence and investments, the Trump administration’s new tariffs and sweeping changes to healthcare have introduced new uncertainties.

FDA And Boston Scientific Face Legal Action Over Spinal Device Safety

 
• By 

A patient, Dena Lawler, is suing Boston Scientific and the US FDA over alleged harm from a spinal stimulator device. Lawler claims inadequate regulatory review allowed dangerous modifications to the device, which then caused her pain and other health issues. She is calling for reconsideration of the device approvals.

Tariff Pressures Creep Into Dexcom’s Margin Outlook Despite Domestic Manufacturing Shield

 
• By 

Dexcom’s relatively strong position amid tariff-related uncertainty is rooted in its long-standing investments in domestic manufacturing, but raw material inflation is driving cost concerns even in the face of operational efficiencies.