Open AccessPosted Content
Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth
TL;DR: This work proposes a new clustering loss function for proposal-free instance segmentation that pulls the spatial embeddings of pixels belonging to the same instance together and jointly learns an instance-specific clustering bandwidth, maximizing the intersection-over-union of the resulting instance mask.
read more
Abstract: Current state-of-the-art instance segmentation methods are not suited for real-time applications like autonomous driving, which require fast execution times at high accuracy. Although the currently dominant proposal-based methods have high accuracy, they are slow and generate masks at a fixed and low resolution. Proposal-free methods, by contrast, can generate masks at high resolution and are often faster, but fail to reach the same accuracy as the proposal-based methods. In this work we propose a new clustering loss function for proposal-free instance segmentation. The loss function pulls the spatial embeddings of pixels belonging to the same instance together and jointly learns an instance-specific clustering bandwidth, maximizing the intersection-over-union of the resulting instance mask. When combined with a fast architecture, the network can perform instance segmentation in real-time while maintaining a high accuracy. We evaluate our method on the challenging Cityscapes benchmark and achieve top results (5\% improvement over Mask R-CNN) at more than 10 fps on 2MP images. Code will be available at this https URL .
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Conditional Convolutions for Instance Segmentation
Zhi Tian,Chunhua Shen,Hao Chen +2 more
- 23 Aug 2020
TL;DR: A simpler instance segmentation method that can achieve improved performance in both accuracy and inference speed on the COCO dataset, and outperform a few recent methods including well-tuned Mask RCNN baselines, without longer training schedules needed.
850
BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation
Hao Chen,Kunyang Sun,Zhi Tian,Chunhua Shen,Yongming Huang,Youliang Yan +5 more
- 14 Jun 2020
TL;DR: The proposed BlendMask can effectively predict dense per-pixel position-sensitive instance features with very few channels, and learn attention maps for each instance with merely one convolution layer, thus being fast in inference.
YOLACT++: Better Real-time Instance Segmentation.
TL;DR: A simple, fully-convolutional model for real-time instance segmentation that achieves competitive results on MS COCO evaluated on a single Titan Xp, which is significantly faster than any previous state-of-the-art approach.
522
•Posted Content
Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation.
TL;DR: This paper factorizes 2D self-attention into two 1Dself-attentions, a novel building block that one could stack to form axial-att attention models for image classification and dense prediction, and achieves state-of-the-art results on Mapillary Vistas and Cityscapes.
•Posted Content
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers.
TL;DR: MaX-DeepLab, the first end-to-end model for panoptic segmentation, is presented, and shows a significant 7.1% PQ gain in the box-free regime on the challenging COCO dataset, closing the gap between box-based and box- free methods for the first time.
References
Mask R-CNN
Kaiming He,Georgia Gkioxari,Piotr Dollár,Ross Girshick +3 more
- 20 Mar 2017
TL;DR: This work presents a conceptually simple, flexible, and general framework for object instance segmentation, which extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition.
•Proceedings Article
Mask R-CNN
Kaiming He,Georgia Gkioxari,Piotr Dollár,Ross Girshick +3 more
- 20 Mar 2017
TL;DR: This work presents a conceptually simple, flexible, and general framework for object instance segmentation that outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners.
19.7K
•Proceedings Article
Faster R-CNN: towards real-time object detection with region proposal networks
Shaoqing Ren,Kaiming He,Ross Girshick,Jian Sun +3 more
- 07 Dec 2015
TL;DR: Ren et al. as discussed by the authors proposed a region proposal network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals.
Path Aggregation Network for Instance Segmentation
Shu Liu,Lu Qi,Haifang Qin,Jianping Shi,Jiaya Jia +4 more
- 18 Jun 2018
TL;DR: PANet as mentioned in this paper enhances the entire feature hierarchy with accurate localization signals in lower layers by bottom-up path augmentation, which shortens the information path between lower layers and topmost feature.
•Posted Content
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision
Alex Kendall,Yarin Gal +1 more
TL;DR: A Bayesian deep learning framework combining input-dependent aleatoric uncertainty together with epistemic uncertainty is presented, which makes the loss more robust to noisy data, also giving new state-of-the-art results on segmentation and depth regression benchmarks.
3.7K