Journal Article10.48550/arXiv.2212.11966
Removing Objects From Neural Radiance Fields
Silvan Weder,Guillermo Garcia-Hernando,Aron Monszpart,Marc Pollefeys,Gabriel J. Brostow,Michael Firman,Sara Vicente +6 more
33
TL;DR: In this paper , a confidence-based view selection procedure is used to select the individual 2D inpainted images to use in the creation of the NeRF, so that the resulting inpainted NeRF is 3D consistent.
read more
Abstract: Neural Radiance Fields (NeRFs) are emerging as a ubiquitous scene representation that allows for novel view synthesis. Increasingly, NeRFs will be shareable with other people. Before sharing a NeRF, though, it might be desirable to remove personal information or unsightly objects. Such removal is not easily achieved with the current NeRF editing frameworks. We propose a framework to remove objects from a NeRF representation created from an RGB-D sequence. Our NeRF inpainting method leverages recent work in 2D image inpainting and is guided by a user-provided mask. Our algorithm is underpinned by a confidence based view selection procedure. It chooses which of the individual 2D inpainted images to use in the creation of the NeRF, so that the resulting inpainted NeRF is 3D consistent. We show that our method for NeRF editing is effective for synthesizing plausible inpaintings in a multi-view coherent manner. We validate our approach using a new and still-challenging dataset for the task of NeRF inpainting.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fields
Ashkan Mirzaei,Tristan Aumentado-Armstrong,Konstantinos G. Derpanis,Jonathan W. Kelly,Marcus A. Brubaker,Igor Gilitschenski,Alex Levinshtein +6 more
TL;DR: In this paper , a perceptual optimization-based 3D inpainting method is proposed to remove unwanted objects from a 3D scene, such that the replaced region is visually plausible and consistent with its context.
59
InpaintNeRF360: Text-Guided 3D Inpainting on Unbounded Neural Radiance Fields
TL;DR: InpaintNeRF360 as discussed by the authors employs a promptable segmentation model by generating multi-modal prompts from the encoded text for multiview segmentation, and further refine the inpainted NeRF model using perceptual priors to ensure visual plausibility.
10
RePaint-NeRF: NeRF Editting via Semantic Masks and Diffusion Models
TL;DR: In this article , a pre-trained diffusion model is used to guide the NeRF model to generate new 3D objects, which can improve the editability, diversity, and application range of NeRF.
10
Clutter Detection and Removal in 3D Scenes with View-Consistent Inpainting
TL;DR: In this article , the authors propose techniques for 3D segmentation from shared properties and 3D inpainting, both of which are important problems in 3D scene clutter removal.
DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editing
Minghao Chen,Iro Laina,Andrea Vedaldi +2 more
- 29 Apr 2024
TL;DR: DGE is a method for direct and efficient 3D editing based on consistent multi-view editing. It utilizes a training-free approach to make a high-quality image editor multi-view consistent and directly optimize the 3D object representation based on a multi-view consistent edited sequence of images.
References
Image quality assessment: from error visibility to structural similarity
TL;DR: In this article, a structural similarity index is proposed for image quality assessment based on the degradation of structural information, which can be applied to both subjective ratings and objective methods on a database of images compressed with JPEG and JPEG2000.
•Proceedings Article
Mask R-CNN
Kaiming He,Georgia Gkioxari,Piotr Dollár,Ross Girshick +3 more
- 20 Mar 2017
TL;DR: This work presents a conceptually simple, flexible, and general framework for object instance segmentation that outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners.
19.7K
•Posted Content
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
TL;DR: A new dataset of human perceptual similarity judgments is introduced and it is found that deep features outperform all previous metrics by large margins on this dataset, and suggests that perceptual similarity is an emergent property shared across deep visual representations.
7.5K
Context Encoders: Feature Learning by Inpainting
Deepak Pathak,Philipp Krähenbühl,Jeff Donahue,Trevor Darrell,Alexei A. Efros +4 more
- 27 Jun 2016
TL;DR: It is found that a context encoder learns a representation that captures not just appearance but also the semantics of visual structures, and can be used for semantic inpainting tasks, either stand-alone or as initialization for non-parametric methods.
Structure-from-Motion Revisited
Johannes L. Schonberger,Jan-Michael Frahm +1 more
- 27 Jun 2016
TL;DR: This work proposes a new SfM technique that improves upon the state of the art to make a further step towards building a truly general-purpose pipeline.