The Vision Transformer (ViT) architecture has been remarkably successful in image restoration. For a while, Convolutional Neural Networks (CNN) predominated in most computer vision tasks. Now, both CNN and ViT are efficient approaches that demonstrate powerful capabilities to restore a better version of an image given in a low-quality format. In this study, the efficiency of ViT in image restoration is studied extensively. The ViT architectures are classified for every task of image restoration. Seven image restoration tasks are considered: Image Super-Resolution, Image Denoising, General Image Enhancement, JPEG Compression Artifact Reduction, Image Deblurring, Removing Adverse Weather Conditions, and Image Dehazing. The outcomes, the advantages, the limitations, and the possible areas for future research are detailed. Overall, it is noted that incorporating ViT in the new architectures for image restoration is becoming a rule. This is due to some advantages compared to CNN, such as better efficiency, especially when more data are fed to the network, robustness in feature extraction, and a better feature learning approach that sees better the variances and characteristics of the input. Nevertheless, some drawbacks exist, such as the need for more data to show the benefits of ViT over CNN, the increased computational cost due to the complexity of the self-attention block, a more challenging training process, and the lack of interpretability. These drawbacks represent the future research direction that should be targeted to increase the efficiency of ViT in the image restoration domain.

/pdf/vision-transformers-in-image-restoration-a-survey-16ou29tw.pdf

Vision Transformers in Image Restoration: A Survey

To reduce the radiation dose, sparse-view computed tomography (CT) reconstruction has been proposed, aiming to recover high-quality CT images from sparsely sampled sinogram. To eliminate the artifacts present in sparse-view CT images, a new dual-domain diffusion model (DDDM) is proposed, which is composed of a sinogram upgrading module (SUM) and an image refining module (IRM) connected in series. In the sinogram domain, a novel degrading and upgrading framework is defined, in which SUM is trained to upgrade sparse-view sinograms step by step to reverse the degradation process of CT images caused by successive down-sampling of scanning views. In the image domain, IRM adopts an improved denoising diffusion framework to further reduce remaining artifacts and restore image details, where a skip connection from the original sparse-view sinogram is introduced to constrain the generation of details. Our DDDM shows significant improvement over deep-learning baseline models in both classical similarity metrics and perceptual loss, and has good generalization to untrained organs.

A Dual-domain Diffusion Model for Sparse-view CT Reconstruction

With Artificial Intelligence (AI) increasingly permeating various aspects of society, including healthcare, the adoption of the Transformers neural network architecture is rapidly changing many applications. Transformer is a type of deep learning architecture initially developed to solve general-purpose Natural Language Processing (NLP) tasks and has subsequently been adapted in many fields, including healthcare. In this survey paper, we provide an overview of how this architecture has been adopted to analyze various forms of healthcare data, including clinical NLP, medical imaging, structured Electronic Health Records (EHR), social media, bio-physiological signals, biomolecular sequences. Furthermore, which have also include the articles that used the transformer architecture for generating surgical instructions and predicting adverse outcomes after surgeries under the umbrella of critical care. Under diverse settings, these models have been used for clinical diagnosis, report generation, data reconstruction, and drug/protein synthesis. Finally, we also discuss the benefits and limitations of using transformers in healthcare and examine issues such as computational cost, model interpretability, fairness, alignment with human values, ethical implications, and environmental impact. 

Transformers and large language models in healthcare: A review

Deep learning based solutions are being succesfully implemented for a wide variety of applications. Most notably, clinical use-cases have gained an increased interest and have been the main driver behind some of the cutting-edge data-driven algorithms proposed in the last years. For applications like sparse-view tomographic reconstructions, where the amount of measurement data is small in order to keep acquisition time short and radiation dose low, reduction of the streaking artifacts has prompted the development of data-driven denoising algorithms with the main goal of obtaining diagnostically viable images with only a subset of a full-scan data. We propose WNet, a data-driven dual-domain denoising model which contains a trainable reconstruction layer for sparse-view artifact denoising. Two encoder-decoder networks perform denoising in both sinogram- and reconstruction-domain simultaneously, while a third layer implementing the Filtered Backprojection algorithm is sandwiched between the first two and takes care of the reconstruction operation. We investigate the performance of the network on sparse-view chest CT scans, and we highlight the added benefit of having a trainable reconstruction layer over the more conventional fixed ones. We train and test our network on two clinically relevant datasets and we compare the obtained results with three different types of sparse-view CT denoising and reconstruction algorithms.

WNet: A Data-Driven Dual-Domain Denoising Model for Sparse-View Computed Tomography With a Trainable Reconstruction Layer

To prevent patients from being exposed to excess of radiation in CT imaging, the most common solution is to decrease the radiation dose by reducing the X-ray, and thus the quality of the resulting low-dose CT images (LDCT) is degraded, as evidenced by more noise and streaking artifacts. Therefore, it is important to maintain high quality CT image while effectively reducing radiation dose. In recent years, with the rapid development of deep learning technology, deep learning-based LDCT denoising methods have become quite popular because of their data-driven and high-performance features to achieve excellent denoising results. However, to our knowledge, no relevant article has so far comprehensively introduced and reviewed advanced deep learning denoising methods such as Transformer structures in LDCT denoising tasks. Therefore, based on the literatures related to LDCT image denoising published from year 2016–2023, and in particular from 2020 to 2023, this study presents a systematic survey of current situation, and challenges and future research directions in LDCT image denoising field. Four types of denoising networks are classified according to the network structure: CNN-based, Encoder-Decoder-based, GAN-based, and Transformer-based denoising networks, and each type of denoising network is described and summarized from the perspectives of structural features and denoising performances. Representative deep-learning denoising methods for LDCT are experimentally compared and analyzed. The study results show that CNN-based denoising methods capture image details efficiently through multi-level convolution operation, demonstrating superior denoising effects and adaptivity. Encoder-decoder networks with MSE loss, achieve outstanding results in objective metrics. GANs based methods, employing innovative generators and discriminators, obtain denoised images that exhibit perceptually a closeness to NDCT. Transformer-based methods have potential for improving denoising performances due to their powerful capability in capturing global information. Challenges and opportunities for deep learning based LDCT denoising are analyzed, and future directions are also presented. 

A Review of deep learning methods for denoising of medical low-dose CT images

DuDoTrans: Dual-Domain Transformer for Sparse-View CT Reconstruction

Three-dimensional (3D) freehand ultrasound (US) reconstruction without a tracker can be advantageous over its two-dimensional or tracked counterparts in many clinical applications. In this paper, we propose to estimate 3D spatial transformation between US frames from both past and future 2D images, using feed-forward and recurrent neural networks (RNNs). With the temporally available frames, a further multi-task learning algorithm is proposed to utilise a large number of auxiliary transformation-predicting tasks between them. Using more than 40,000 US frames acquired from 228 scans on 38 forearms of 19 volunteers in a volunteer study, the hold-out test performance is quantified by frame prediction accuracy, volume reconstruction overlap, accumulated tracking error and final drift, based on ground-truth from an optical tracker. The results show the importance of modelling the temporal-spatially correlated input frames as well as output transformations, with further improvement owing to additional past and/or future frames. The best performing model was associated with predicting transformation between moderately-spaced frames, with an interval of less than ten frames at 20 frames per second (fps). Little benefit was observed by adding frames more than one second away from the predicted transformation, with or without LSTM-based RNNs. Interestingly, with the proposed approach, explicit within-sequence loss that encourages consistency in composing transformations or minimises accumulated error may no longer be required. The implementation code and volunteer data will be made publicly available ensuring reproducibility and further research.

pdf/trackerless-freehand-ultrasound-with-sequence-modelling-and-17svpums.pdf

Trackerless freehand ultrasound with sequence modelling and auxiliary transformation over past and future frames

Three-dimensional (3D) freehand ultrasound (US) reconstruction without using any additional external tracking device has seen recent advances with deep neural networks (DNNs). In this paper, we first investigated two identified contributing factors of the learned inter-frame correlation that enable the DNN-based reconstruction: anatomy and protocol. We propose to incorporate the ability to represent these two factors - readily available during training - as the privileged information to improve existing DNN-based methods. This is implemented in a new multi-task method, where the anatomical and protocol discrimination are used as auxiliary tasks. We further develop a differentiable network architecture to optimise the branching location of these auxiliary tasks, which controls the ratio between shared and task-specific network parameters, for maximising the benefits from the two auxiliary tasks. Experimental results, on a dataset with 38 forearms of 19 volunteers acquired with 6 different scanning protocols, show that 1) both anatomical and protocol variances are enabling factors for DNN-based US reconstruction; 2) learning how to discriminate different subjects (anatomical variance) and predefined types of scanning paths (protocol variance) both significantly improve frame prediction accuracy, volume reconstruction overlap, accumulated tracking error and final drift, using the proposed algorithm. 

pdf/privileged-anatomical-and-protocol-discrimination-in-30wpo3weql.pdf

Privileged Anatomical and Protocol Discrimination in Trackerless 3D Ultrasound Reconstruction

PURPOSE
To develop a deep learning network that treats the three-dimensional respiratory motion signals as a whole and considers the inter-dimensional correlation between signals of different directions for accurate respiratory tumor motion prediction.


METHODS
We propose a deep learning framework, named as LSTM-Global Temporal Convolution-External Attention Network (LGEANet). In LGEANet, we first feed each of the univariate time series into the Long Short-Term Memory (LSTM) module respectively and utilize the strength of the global temporal convolutional layer to discover the temporal pattern of the univariate signals from hidden states of the LSTM. Then, External attention is adopted to capture the dynamic dependence of the multiple time series. Also, a traditional autoregressive linear model in parallel to the non-linear neural network part was integrated to mitigate the scale insensitivity of the networks. A total of 304 motion traces for 31 patients are acquired from a public dataset in the experiments and four representative cases were selected for model evaluation. The respiratory signals were sampled at intervals of about 37.5 ms (26 frames per second) for an average duration of 71 min.


RESULTS
The proposed LGEANet achieved better performance with higher empirical correlation coefficient value (CORRs) and lower mean absolute error value (MAEs) and relative squared error value (RSEs) than other investigated models. For the four representative datasets, when the response time is less than 231 ms, the model can achieve CORRs more than 0.96. And the averaged position error reduction by using the proposed model was about 67% in the superior-inferior (SI) direction, 41% in the anterior-posterior (AP) direction and 38% in the right-left (RL) direction compared to that without prediction. The proposed network achieved the greatest error reduction in the SI direction, which is the main direction of tumor motion.


CONCLUSIONS
The LGEANet achieves promising performance in minimizing the prediction error due to system latencies during real-time tumor motion tracking. This article is protected by copyright. All rights reserved.

LGEANet: LSTM-Global temporal convolution-external attention network for respiratory motion prediction.

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Qian Li

Author Tools

Chat about Author

Papers

DuDoTrans: Dual-Domain Transformer for Sparse-View CT Reconstruction

Trackerless freehand ultrasound with sequence modelling and auxiliary transformation over past and future frames

Privileged Anatomical and Protocol Discrimination in Trackerless 3D Ultrasound Reconstruction

LGEANet: LSTM-Global temporal convolution-external attention network for respiratory motion prediction.