Foundation models for generalist medical artificial intelligence
Michael Moor,O. Banerjee,Zahra F.H. Abad,Harlan M. Krumholz,Jure Leskovec,Eric J. Topol,Pranav Rajpurkar +6 more
TL;DR: Generalist medical AI (GMAI) as mentioned in this paper is a new paradigm for medical AI, which is capable of carrying out a diverse set of tasks using very little or no task-specific labelled data.
read more
Abstract: The exceptionally rapid development of highly flexible, reusable artificial intelligence (AI) models is likely to usher in newfound capabilities in medicine. We propose a new paradigm for medical AI, which we refer to as generalist medical AI (GMAI). GMAI models will be capable of carrying out a diverse set of tasks using very little or no task-specific labelled data. Built through self-supervision on large, diverse datasets, GMAI will flexibly interpret different combinations of medical modalities, including data from imaging, electronic health records, laboratory results, genomics, graphs or medical text. Models will in turn produce expressive outputs such as free-text explanations, spoken recommendations or image annotations that demonstrate advanced medical reasoning abilities. Here we identify a set of high-impact potential applications for GMAI and lay out specific technical capabilities and training datasets necessary to enable them. We expect that GMAI-enabled applications will challenge current strategies for regulating and validating AI devices for medicine and will shift practices associated with the collection of large medical datasets. This review discusses generalist medical artificial intelligence, identifying potential applications and setting out specific technical capabilities and training datasets necessary to enable them, as well as highlighting challenges to its implementation.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
LoRKD: Low-Rank Knowledge Decomposition for Medical Foundation Models
Haolin Li,Yuhang Zhou,Ziheng Zhao,Siyuan Du,Jiangchao Yao,Weidi Xie,Ya Zhang,Yanfeng Wang +7 more
- 28 Sep 2024
TL;DR: This paper proposes LoRKD, a novel framework for decomposing large-scale medical foundation models into lightweight expert models, enhancing specialization and reducing resource consumption, achieving state-of-the-art performance and superior transferability on downstream tasks.
A review of handcrafted and deep radiomics in neurological diseases: transitioning from oncology to clinical neuroimaging
Elizaveta Lavrova,Henry C. Woodruff,Hamza Khan,Eric Salmon,Philippe Lambin,Christophe Phillips +5 more
TL;DR: This review explores the application of handcrafted and deep radiomics in neurological diseases, highlighting its potential for improved diagnostic precision and treatment quality, but also emphasizing the need for collaborative efforts to overcome implementation challenges.
Diagnostics
Ronald M. Razmi
- 05 Jan 2024
TL;DR: Diagnostics involve radiology, pathology, dermatology, ophthalmology, and cardiology. Images and visual inspections are key to diagnosis in these specialties.
Surprisingly Fragile: Assessing and Addressing Prompt Instability in Multimodal Foundation Models
I. M. Stewart,Sameera Horawalavithana,Brendan Kennedy,Sai Munikoti,Karl Pazdernik +4 more
- 26 Aug 2024
TL;DR: Multimodal foundation models exhibit prompt instability, leading to performance drops, but can be mitigated with additional training on augmented data, improving accuracy and stability across modalities and domains.
References
•Proceedings Article
Attention is All you Need
Ashish Vaswani,Noam Shazeer,Niki Parmar,Jakob Uszkoreit,Llion Jones,Aidan N. Gomez,Lukasz Kaiser,Illia Polosukhin +7 more
- 12 Jun 2017
TL;DR: This paper proposed a simple network architecture based solely on an attention mechanism, dispensing with recurrence and convolutions entirely and achieved state-of-the-art performance on English-to-French translation.
•Posted Content
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TL;DR: A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
81.7K
Highly accurate protein structure prediction with AlphaFold
John M. Jumper,Richard O. Evans,Alexander Pritzel,Tim Green,Michael Figurnov,Olaf Ronneberger,Kathryn Tunyasuvunakool,Russell Bates,Augustin Žídek,Anna Potapenko,Alex Bridgland,Clemens Meyer,Simon A. A. Kohl,Andrew J. Ballard,Andrew Cowie,Bernardino Romera-Paredes,Stanislav Nikolov,R. D. Jain,Jonas Adler,Trevor Back,Stig Petersen,David Reiman,Ellen Clancy,Michal Zielinski,Martin Steinegger,Michalina Pacholska,Tamas Berghammer,Sebastian Bodenstein,David L. Silver,Oriol Vinyals,Andrew W. Senior,Koray Kavukcuoglu,Pushmeet Kohli,Demis Hassabis +33 more
TL;DR: For example, AlphaFold as mentioned in this paper predicts protein structures with an accuracy competitive with experimental structures in the majority of cases using a novel deep learning architecture. But the accuracy is limited by the fact that no homologous structure is available.
•Proceedings Article
Language Models are Few-Shot Learners
Tom B. Brown,Benjamin Mann,Nick Ryder,Melanie Subbiah,Jared Kaplan,Prafulla Dhariwal,Arvind Neelakantan,Pranav Shyam,Girish Sastry,Amanda Askell,Sandhini Agarwal,Ariel Herbert-Voss,Gretchen Krueger,Thomas Henighan,Rewon Child,Aditya Ramesh,Daniel M. Ziegler,Jeffrey Wu,Clemens Winter,Christopher Hesse,Mark Chen,Eric Sigler,Mateusz Litwin,Scott Gray,Benjamin Chess,Jack Clark,Christopher Berner,Samuel McCandlish,Alec Radford,Ilya Sutskever,Dario Amodei +30 more
- 28 May 2020
TL;DR: GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic.
UK biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age
Cathie Sudlow,John Gallacher,Naomi E. Allen,Valerie Beral,Paul Burton,John Danesh,Paul Downey,Paul Elliott,Jane Green,Martin J Landray,Bette Liu,Paul M. Matthews,Giok Ong,Jill P. Pell,Alan J. Silman,Alan Young,Tim Sprosen,Tim Peakman,Rory Collins +18 more
TL;DR: The UK Biobank is described, a large population-based prospective study, established to allow investigation of the genetic and non-genetic determinants of the diseases of middle and old age.