Can Jack-Of-All-Trades AI Reshape Medicine?

Newswise — The vast majority of AI models used in medicine today are “narrow specialists,” trained to perform one or two tasks, such as scanning mammograms for signs of breast cancer or detecting lung disease on chest X-rays.

But the everyday practice of medicine involves an endless array of clinical scenarios, symptom presentations, possible diagnoses, and treatment conundrums. So, if AI is to deliver on its promise to reshape clinical care, it must reflect that complexity of medicine and do so with high fidelity, says Pranav Rajpurkar, assistant professor of biomedical informatics in the Blavatnik Institute at HMS.

Enter generalist medical AI, a more evolved form of machine learning capable of performing complex tasks in a wide range of scenarios.

Akin to general medicine physicians, Rajpurkar explained, generalist medical AI models can integrate multiple data types — such as MRI scans, X-rays, blood test results, medical texts, and genomic testing — to perform a range of tasks, from making complex diagnostic calls to supporting clinical decisions to choosing optimal treatment. And they can be deployed in a variety of settings, from the exam room to the hospital ward to the outpatient GI procedure suite to the cardiac operating room.

While the earliest versions of generalist medical AI have started to emerge, its true potential and depth of capabilities have yet to materialize.

“The rapidly evolving capabilities in the field of AI have completely redefined what we can do in the field of medical AI,” writes Rajpurkar in a newly published perspective in Nature, on which he is co-senior author with Eric Topol of the Scripps Research Institute and colleagues from Stanford University, Yale University, and the University of Toronto.

Generalist medical AI is on the cusp of transforming clinical medicine as we know it, but with this opportunity come serious challenges, the authors say.

In the article, the authors discuss the defining features of generalist medical AI, identify various clinical scenarios where these models can be used, and chart the road forward for their design, development, and deployment.

Features of generalist medical AI

Key characteristics that render generalist medical AI models superior to conventional models are their adaptability, their versatility, and their ability to apply existing knowledge to new contexts.

For example, a traditional AI model trained to spot brain tumors on a brain MRI will look at a lesion on an image to determine whether it’s a tumor. It can provide no information beyond that. By contrast, a generalist model would look at a lesion and determine what type of lesion it is — a tumor, a cyst, an infection, or something else. It may recommend further testing and, depending on the diagnosis, suggest treatment options.

“Compared with current models, generalist medical AI will be able to perform more sophisticated reasoning and integrate multiple data types, which lets it build a more detailed picture of a patient’s case,” said study co-first author Oishi Banerjee, a research associate in the Rajpurkar lab, which is already working on designing such models.

According to the authors, generalist models will be able to:

Clinical scenarios for use of generalist medical AI

The researchers outline many areas in which generalist medical AI models would offer comprehensive solutions.

Some of them are:

Ahead, promise and peril

Generalist medical AI models have the potential to transform health care, the authors say. They can alleviate clinician burnout, reduce clinical errors, and expedite and improve clinical decision-making.

Yet, these models come with unique challenges. Their strongest features — extreme versatility and adaptability — also pose the greatest risks, the researchers caution, because they will require the collection of vast and diverse data.

Some critical pitfalls include:

“These are serious but not insurmountable hurdles,” Rajpurkar said. “Having a clear-eyed understanding of all the challenges early on will help ensure that generalist medical AI delivers on its tremendous promise to change the practice of medicine for the better.”

Authorship, funding, disclosures

Co-authors included Michael Moor and Jure Leskovec of Stanford; Zahra Shakeri Hossein Abad of the University of Toronto; and Harlan Krumholz of Yale.

Researchers on this perspective receive funding from the National Institutes of Health (grants UL1TR001114, R61 NS11865, 3U54HG010426-04S1), the Defense Advanced Research Projects Agency (DARPA) (grants N660011924033, HR00112190039, and N660011924033), the Army Research Office (W911NF-16-1-0342 and W911NF-16-1-0171), the National Science Foundation (OAC-1835598, OAC-1934578, and CCF-1918940), Stanford Data Science Initiative, Amazon, Docomo, GSK, Hitachi, Intel, JPMorgan Chase, Juniper Networks, KDDI, NEC, Toshiba, and Wu Tsai Neurosciences Institute.

Krumholz has received expenses and/or personal fees from UnitedHealth, Element Science, Eyedentifeye, and F-Prime; is a co-founder of Refactor Health and HugoHealth; and is associated with contracts, through Yale New Haven Hospital, from the Centers for Medicare & Medicaid Services and through Yale University from the U.S. Food and Drug Administration, Johnson & Johnson, Google, and Pfizer.