This paper focuses on the crack detection and digital restoration of ancient mural cultural heritage, proposing a comprehensive method that combines the Unet network structure and diffusion model. Firstly, the Unet network structure is used for efficient crack detection in murals by constructing an ancient mural image dataset for training and validation, achieving accurate identification of mural cracks. Next, an edge-guided optimized masking strategy is adopted for mural restoration, effectively preserving the information of the murals and reducing the damage to the original murals during the restoration process. Lastly, a diffusion model is employed for digital restoration of murals, improving the restoration performance by adjusting parameters to achieve natural repair of mural cracks. Experimental results show that comprehensive method based on the Unet network and diffusion model has significant advantages in the tasks of crack detection and digital restoration of murals, providing a novel and effective approach for the protection and restoration of ancient murals. In addition, this research has significant implications for the technological development in the field of mural restoration and cultural heritage preservation, contributing to the advancement and technological innovation in related fields.

MuralDiff: Diffusion for Ancient Murals Restoration on Large-Scale Pre-Training

Image and video inpainting is a classic problem in computer vision and computer graphics, aiming to fill in the plausible and realistic content in the missing areas of images and videos. With the advance of deep learning, this problem has achieved significant progress recently. The goal of this paper is to comprehensively review the deep learning-based methods for image and video inpainting. Specifically, we sort existing methods into different categories from the perspective of their high-level inpainting pipeline, present different deep learning architectures, including CNN, VAE, GAN, diffusion models, etc., and summarize techniques for module design. We review the training objectives and the common benchmark datasets. We present evaluation metrics for low-level pixel and high-level perceptional similarity, conduct a performance evaluation, and discuss the strengths and weaknesses of representative inpainting methods. We also discuss related real-world applications. Finally, we discuss open challenges and suggest potential future research directions.

Deep Learning-based Image and Video Inpainting: A Survey

Structure Matters: Tackling the Semantic Discrepancy in Diffusion Models for Image Inpainting

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI 

pdf/transref-multi-scale-reference-embedding-transformer-for-u3b7tzjwlr.pdf

Transref: Multi-Scale Reference Embedding Transformer for Reference-Guided Image Inpainting

Image inpainting aims to complete missing regions by extracting the features of the image through the information of the known region. Traditional image inpainting approaches like patch-based and diffusion-based methods are robust for simple images, but fail to reconstruct photo-realistic results when the image contains a lot of texture and objects or the object covers a large region in the images. In recent years, transformer-based models are proven to be capable of achieving promising results in numerous image processing tasks. Transformer-based models can better cope with image long-range dependencies which are important for image inpainting. In this paper, we propose a transformer-based generative adversarial network for edge-guided image inpainting. We present the two-step image inpainting approach that first generates edge information and then completes the image. Our experiments demonstrate that the proposed network is able to regenerate the missing regions in images smoothly and realistically and produce visually appealing results against the state-of-the-art image inpainting approaches. 

Edge-Guided Image Inpainting with Transformer

Image inpainting is an ill-posed problem to recover missing or damaged image content based on incomplete images with masks. Previous works usually predict the auxiliary structures (e.g., edges, segmentation and contours) to help fill visually realistic patches in a multi-stage fashion. However, imprecise auxiliary priors may yield biased inpainted results. Besides, it is time-consuming for some methods to be implemented by multiple stages of complex neural networks. To solve this issue, we develop an end-to-end multi-modality guided transformer network, including one inpainting branch and two auxiliary branches for semantic segmentation and edge textures. Within each transformer block, the proposed multi-scale spatial-aware attention module can learn the multi-modal structural features efficiently via auxiliary denormalization. Different from previous methods relying on direct guidance from biased priors, our method enriches semantically consistent context in an image based on discriminative interplay information from multiple modalities. Comprehensive experiments on several challenging image inpainting datasets show that our method achieves state-of-the-art performance to deal with various regular/irregular masks efficiently. The code is available at https://github.com/yeates/MMT . 

Unbiased Multi-modality Guidance for Image Inpainting

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Lawrence A. Hoffman

Author Tools

Chat about Author

Papers

Unbiased Multi-modality Guidance for Image Inpainting