Journal Article10.1016/j.neucom.2023.126720
Artificial intelligence accelerates multi-modal biomedical process: A Survey
Jiajia Li,Xue Han,Yiming Qin,Feng Tan,Yulong Chen,Zikai Wang,Hai-Tao Song,Xi Zhou,Yuan Zhang,Lun Hu,Pengwei Hu +10 more
13
TL;DR: This survey provides an overview of multi-modal biomedical AI, covering applications, data, methods, and analytics, and identifies potential research directions for future healthcare advancements leveraging AI's capabilities in handling complex medical scenarios and data.
read more
Abstract: The abundance of artificial intelligence AI algorithms and growing computing power has brought a disruptive revolution to the smart medical industry. Its powerful data abstraction and representation capabilities enable the modeling of hundreds of millions of medical data, such as sub-Computed Tomography tumor identification, retinal lesion screening, and survival curve analysis. However, all of these applications demonstrate AI’s use of unimodal data for specific tasks. In contrast, clinicians deal with multi-modal data from multiple sources when diagnosing, performing prognostic assessments, and deciding on treatment plans. These requirements have facilitated the development of multi-modal AI solutions and improved the performance of AI models in handling complex medical scenarios and data. In this paper, we provide an overview of the current state of the art and research in multi-modal biomedical AI, including applications, data, methods, and analytics. Additionally, we summarize potential research directions for multi-modal AI technologies in the future of healthcare.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Fetal cardiac ultrasound standard section detection model based on multitask learning and mixed attention mechanism
Jie He,Yang Lei,Bo Liang,Shengli Li,Caixu Xu +4 more
TL;DR: A single-stage fetal cardiac ultrasound standard plane detection model based on multitask learning and mixed attention mechanism improves detection efficiency and reduces misdiagnosis and omission.
6
A survey on advancements in image-text multimodal models: From general techniques to biomedical implementations
Ruifeng Guo,Jingxuan Wei,Linzhao Sun,Bihui Yu,Gang Chang,Dawei Liu,Sibo Zhang,Zhibin Yao,Min Xu,Liping Bu +9 more
TL;DR: A survey on advancements in image-text multimodal models focusing on general techniques and biomedical implementations. Covers technological evolution, common components, challenges, architecture, and applications. Provides guidance for future research directions.
6
Making the Implicit Explicit: Depression Detection in Web across Posted Texts and Images
Pengwei Hu,Jiajia Li,Feng Tan,Xue Han,Xi Zhou,Lun Hu +5 more
- 05 Dec 2023
TL;DR: An implicit and explicit multi-modal feature fusion (IEMFF) model is proposed for depression detection and successfully makes the implicit information inherent in users’ posted images explicit and further incorporate such explicit features with the textual features directly extracted from user-posted texts.
2
Optimizing Image Enhancement: Feature Engineering for Improved Classification in AI-Assisted Artificial Retinas
Asif Mehmood,Jungbeom Ko,Hyunchul Kim,Jungsuk Kim +3 more
- 23 Apr 2024
TL;DR: Using the suggested set of selective features not only improves the classification model’s performance, but aligns with the specific problem to address the challenge of correctly identifying objects in multi-object scenarios.
1
Special Issue: Artificial Intelligence Technology in Medical Image Analysis
László Szilágyi,Levente Kovács +1 more
TL;DR: Special Issue: AI in Medical Image Analysis explores the latest advancements in AI for medical image analysis, focusing on its impact on diagnosis and treatment.
1
References
•Posted Content
Deep Residual Learning for Image Recognition
TL;DR: This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth.
117.9K
Densely Connected Convolutional Networks
Gao Huang,Zhuang Liu,Laurens van der Maaten,Kilian Q. Weinberger +3 more
- 21 Jul 2017
TL;DR: DenseNet as mentioned in this paper proposes to connect each layer to every other layer in a feed-forward fashion, which can alleviate the vanishing gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters.
•Proceedings Article
Auto-Encoding Variational Bayes
Diederik P. Kingma,Max Welling +1 more
- 01 Jan 2014
TL;DR: A stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case is introduced.
Multimodal Machine Learning: A Survey and Taxonomy
TL;DR: This paper surveys the recent advances in multimodal machine learning itself and presents them in a common taxonomy to enable researchers to better understand the state of the field and identify directions for future research.
3.4K