Free-Form Image Inpainting With Gated Convolution
Jiahui Yu,Zhe Lin,Jimei Yang,Xiaohui Shen,Xin Lu,Thomas S. Huang +5 more
- 22 Oct 2019
- pp 4470-4479
TL;DR: Yu et al. as mentioned in this paper proposed a generative image inpainting system to complete images with free-form mask and guidance, which is based on gated convolutions learned from millions of images without additional labeling efforts.
read more
Abstract: We present a generative image inpainting system to complete images with free-form mask and guidance. The system is based on gated convolutions learned from millions of images without additional labelling efforts. The proposed gated convolution solves the issue of vanilla convolution that treats all input pixels as valid ones, generalizes partial convolution by providing a learnable dynamic feature selection mechanism for each channel at each spatial location across all layers. Moreover, as free-form masks may appear anywhere in images with any shape, global and local GANs designed for a single rectangular mask are not applicable. Thus, we also present a patch-based GAN loss, named SN-PatchGAN, by applying spectral-normalized discriminator on dense image patches. SN-PatchGAN is simple in formulation, fast and stable in training. Results on automatic image inpainting and user-guided extension demonstrate that our system generates higher-quality and more flexible results than previous methods. Our system helps user quickly remove distracting objects, modify image layouts, clear watermarks and edit faces. Code, demo and models are available at: \url{https://github.com/JiahuiYu/generative_inpainting}.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
•Posted Content
ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models
TL;DR: In this article, an iterative Latent Variable Refinement (ILVR) method is proposed to guide the generative process in DDPM to generate high-quality images based on a given reference image.
310
Artificial intelligence in the creative industries: a review
TL;DR: In this article, a review of the current state of the art in artificial intelligence (AI) technologies and applications in the context of the creative industries is provided, including convolutional neural networks, generative adversarial networks (GANs), recurrent neural networks (RNNs), and deep reinforcement learning (DRL).
A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications
Laith Alzubaidi,Jinshuai Bai,Aiman Abdul Razzak Al-Sabaawi,José Santamaría,Ahmed Shihab Albahri,Bashar S. Al-Dabbagh,Mohammed A. Fadhel,Mohamed Manoufali,Ali H. Al-Timemy,Ye Duan,Laith Farhan,Yi Lu,Ashish Gupta,Yuantong Gu +13 more
TL;DR: In this article , the authors present a survey on state-of-the-art techniques to deal with training DL models to overcome three challenges including small, imbalanced datasets, and lack of generalization.
Neural Point-Based Graphics
Kara-Ali Aliev,Artem Sevastopolsky,Maria Kolos,Dmitry Ulyanov,Victor Lempitsky +4 more
- 23 Aug 2020
TL;DR: This work presents a new point-based approach for modeling the appearance of real scenes that uses a raw point cloud as the geometric representation of a scene, and augments each point with a learnable neural descriptor that encodes local geometry and appearance.
298
Reaction or Speculation: Building Computational Support for Users in Catching-Up Series Based on an Emerging Media Consumption Phenomenon
Riku Arakawa,Hiromu Yakura +1 more
TL;DR: In this paper, a series of studies were conducted to understand how people engage with speculation during media consumption and designed two prototypes for supporting catching-up users based on their quantitative analysis of Twitter data in regard to reaction-and speculation-based media consumption.
268
References
U-Net: Convolutional Networks for Biomedical Image Segmentation
Olaf Ronneberger,Philipp Fischer,Thomas Brox +2 more
- 05 Oct 2015
TL;DR: Neber et al. as discussed by the authors proposed a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently, which can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks.
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky,Jia Deng,Hao Su,Jonathan Krause,Sanjeev Satheesh,Sean Ma,Zhiheng Huang,Andrej Karpathy,Aditya Khosla,Michael S. Bernstein,Alexander C. Berg,Li Fei-Fei +11 more
TL;DR: The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) as mentioned in this paper is a benchmark in object category classification and detection on hundreds of object categories and millions of images, which has been run annually from 2010 to present, attracting participation from more than fifty institutions.
Squeeze-and-Excitation Networks
Jie Hu,Li Shen,Samuel Albanie,Gang Sun,Enhua Wu +4 more
- 18 Jun 2018
TL;DR: This work proposes a novel architectural unit, which is term the "Squeeze-and-Excitation" (SE) block, that adaptively recalibrates channel-wise feature responses by explicitly modelling interdependencies between channels and finds that SE blocks produce significant performance improvements for existing state-of-the-art deep architectures at minimal additional computational cost.
Image-to-Image Translation with Conditional Adversarial Networks
Phillip Isola,Jun-Yan Zhu,Tinghui Zhou,Alexei A. Efros +3 more
- 21 Jul 2017
TL;DR: Conditional adversarial networks are investigated as a general-purpose solution to image-to-image translation problems and it is demonstrated that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks.
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Justin Johnson,Alexandre Alahi,Li Fei-Fei +2 more
- 08 Oct 2016
TL;DR: In this paper, the authors combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for image style transfer, where a feedforward network is trained to solve the optimization problem proposed by Gatys et al. in real-time.