The state-of-the-art models for medical image segmentation are variants of U-Net and fully convolutional networks (FCN). Despite their success, these models have two limitations: (1) their optimal depth is apriori unknown, requiring extensive architecture search or inefficient ensemble of models of varying depths; and (2) their skip connections impose an unnecessarily restrictive fusion scheme, forcing aggregation only at the same-scale feature maps of the encoder and decoder sub-networks. To overcome these two limitations, we propose UNet++, a new neural architecture for semantic and instance segmentation, by (1) alleviating the unknown network depth with an efficient ensemble of U-Nets of varying depths, which partially share an encoder and co-learn simultaneously using deep supervision; (2) redesigning skip connections to aggregate features of varying semantic scales at the decoder sub-networks, leading to a highly flexible feature fusion scheme; and (3) devising a pruning scheme to accelerate the inference speed of UNet++. We have evaluated UNet++ using six different medical image segmentation datasets, covering multiple imaging modalities such as computed tomography (CT), magnetic resonance imaging (MRI), and electron microscopy (EM), and demonstrating that (1) UNet++ consistently outperforms the baseline models for the task of semantic segmentation across different datasets and backbone architectures; (2) UNet++ enhances segmentation quality of varying-size objects—an improvement over the fixed-depth U-Net; (3) Mask RCNN++ (Mask R-CNN with UNet++ design) outperforms the original Mask R-CNN for the task of instance segmentation; and (4) pruned UNet++ models achieve significant speedup while showing only modest performance degradation. Our implementation and pre-trained models are available at https://github.com/MrGiovanni/UNetPlusPlus .

UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation

U-net is an image segmentation technique developed primarily for image segmentation tasks. These traits provide U-net with a high utility within the medical imaging community and have resulted in extensive adoption of U-net as the primary tool for segmentation tasks in medical imaging. The success of U-net is evident in its widespread use in nearly all major image modalities, from CT scans and MRI to X-rays and microscopy. Furthermore, while U-net is largely a segmentation tool, there have been instances of the use of U-net in other applications. Given that U-net’s potential is still increasing, this narrative literature review examines the numerous developments and breakthroughs in the U-net architecture and provides observations on recent trends. We also discuss the many innovations that have advanced in deep learning and discuss how these tools facilitate U-net. In addition, we review the different image modalities and application areas that have been enhanced by U-net.

U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications

The digital transformation in healthcare, propelled by the integration of deep learning models and the Internet of Things (IoT), is creating unprecedented opportunities for improving patient care. However, the utilization of low-resolution images, often generated by IoT devices, introduces biases in the deep learning models, thereby affecting the overall clinical decision-making process. While super-resolution techniques have been extensively employed to transform low-resolution images into high-resolution counterparts, the challenge of achieving highly accurate image restoration remains unresolved. This is especially critical in the medical imaging domain, where even minor inaccuracies can lead to significant biases in model training and, consequently, impact clinical outcomes. Although existing surveys have explored various super-resolution methods and their applications across different fields, a comprehensive review emphasizing the accuracy of image restoration in medical imaging and its subsequent influence on deep learning models is notably lacking. This survey seeks to bridge this gap by offering a systematic review of current state-of-the-art models, highlighting the limitations of existing surveys, and underscoring open questions that merit further research. Specifically, we delve into the intricacies of medical image restoration, identify research gaps and unmet challenges in achieving optimal restoration of medical images, and emphasize the crucial role of developing more precise and resilient super-resolution methods to enhance the quality of medical images and, consequently, the performance of deep learning models in healthcare applications. Ultimately, this survey fosters a deeper comprehension of the prevailing challenges and unresolved questions in the field, thus setting the stage for future research efforts focused on refining medical image restoration and, subsequently, boosting the efficacy of deep learning models in healthcare. 

Medical Image Super-Resolution for Smart Healthcare Applications: A Comprehensive Survey

This paper describes a new non-rigid approach to register images from same- and cross-imaging modalities such as magnetic resonance imaging, computed tomography, and 3D rotational angiography. The deformation is a key challenge in medical image registration. We have proposed a diffeomorphism-based method to tackle this problem using an optimized framework. A non stationary velocity field is used to minimize the effect of forces that are derived from the image gradients. Furthermore, we propose a similarity energy function, based on the gray scale distribution, to limit the fluctuations while approaching the local minima. The proposed method is evaluated on both private and public datasets; the results show that the values of mean square error (MSE), normalized cross-correlation (NCC), structural similarity (SS), mutual information (MI), feature similarity index (FSIM), and mean absolute error (MAE) are 1.3136, 0.9962, 0.9897, 0.883, 0.9922, and 1.52± 2.09, respectively. Both qualitative and quantitative evaluation show promising registration accuracy reflecting the potential of the proposed method. 

/pdf/toward-computing-cross-modality-symmetric-non-rigid-medical-2mszzrds.pdf

Toward Computing Cross-Modality Symmetric Non-Rigid Medical Image Registration

A high resolution representation network with multi-path scale for retinal vessel segmentation.

Multiscale dense convolutional neural network for DSA cerebrovascular segmentation

Vascular image registration is an essential approach to combine the advantages of preoperative 3D computed tomography angiography images and intraoperative 2D X-ray/digital subtraction angiography (DSA) images together. Cardiovascular deformation caused by heartbeat and respiration is one of the most vital factors that affect vascular image registration accuracy. Traditional optimized-based registration methods suffer severely from high computational complexity, which hinders the clinical applications of these methods seriously. To overcome these challenges, we developed a novel multi-channel convolutional neural network (MCNN) that combines a CNN with a periodic vascular diameter variation model. Our method is capable of registering simulated DSA images or real DSA images with their corresponding 3D models in a matter of milliseconds. Our presented MCNN model achieves excellent registration results of three different kinds of cardiovascular patients. The mean absolute error of all six transformation parameters of the MCNN model presented in this paper is less than 1 mm or 1°. The improvement to our MCNN model in registration accuracy is larger than 75% over a single-channel CNN model. Our MCNN method performs more effectively and stable than the state-of-the-art intensity-based methods, especially when vascular deformations occur.

/pdf/deformable-cardiovascular-image-registration-via-multi-52a5fwk49l.pdf

Deformable Cardiovascular Image Registration via Multi-Channel Convolutional Neural Network

Registration between the intraoperative 2D digital subtraction angiography (DSA) and the pre-operative 3D computed tomography angiography (CTA) is useful in the guidance of vascular interventional surgery (VIS). Because of the flow and diffusion of contrast in vessels, not all the vessels are visible in the DSA, which leads to difficulty in registration. Although the conventional mutual information (MI) and the normalized MI (NMI) perform well in orthopedic surgery, it cannot perform as well in VIS because of the missing vessel in DSA. In this paper, a novel similarity measure, WLMI (Weighted Local Mutual Information), is proposed to perform 2D-3D registration, which uses the patches selected in DSA to find the best match in the DRRs (digital radiography reconstruction) with weighted MI. At first, we choose several ways to measure the richness of block information to select small and scattered patches in the DSA image. Subsequently, different weights are assigned to these patches according to their information abundance by calculating the local MI. With the proposed WLMI, the 2D-3D registration experiments are conducted with the synthetic and real DSA images to CTA vessel model. In synthetic registration, the optimal way to measure information abundance and the optimal weight parameter value are selected. The registration with intact and partially absent vessels are compared with the conventional methods using MAE, mTREproj, and IoU metrics. In the registration of real DSA to CTA model, only IoU is used to compare the difference since the ground-truth cannot be derived. In synthetic registration with vessel excalation test, the mTREproj is 2.9 mm and the IoU is 96.7%. In a real registration test, the IoU reaches 81.3%. Both the synthetic and real DSA registration results show that the proposed WLMI can cope with the problem that partial vessel is invisible in the DSA, and outperformed the conventional methods such as MI and NMI. Therefore, the WLMI is more suitable to be used in the 2D/3D registration task in VIS.

/pdf/2d-3d-registration-with-weighted-local-mutual-information-in-26fcy40odg.pdf

2D-3D Registration With Weighted Local Mutual Information in Vascular Interventions

In order to detect X-corner (or X-point) features more accurately and apace, this article presents a novel and fast detection method based on block-by-block search strategy. Unlike general pixel-by...

https://journals.sagepub.com/doi/pdf/10.1177/1687814019834149

A fast X-corner detection method based on block-search strategy:

Segmenting ultra high-field MR images is an important first step in many applications. Segmentation methods based on machine learning have been shown to be valuable tools for this purpose. However, for ultra high-field MR images ( > 7 Tesla), the lack of training data is a problem. Therefore, in this work we propose to use super-resolution for augmenting the training set. Specifically, we describe an efficient super-resolution model based on Generative Adversarial Network(GAN) 1 . It produces synthetic images that simulate MR data at ultra high isotropic resolutions of 0 . 6 mm. We present the first results that show an improvement in segmentation accuracy of imaging data acquired at a 9.4 Tesla MRI scanner.

Super-Resolution for Ultra High-Field MR Images

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Qi Wang

Author Tools

Chat about Author