Journal Article10.1007/S10489-021-02643-5
Incremental learning with open set based discrimination enhancement
7
TL;DR: In OSIL, an open-set incremental strategy is presented, which uses enhanced discrimination combined with open- set identification to overcome the imbalance between old classes and new classes and significantly outperforms state-of-the-art methods.
read more
Abstract: Multi-Class Incremental Learning (MCIL) is an important task in deep learning that aims to solve real world problems of allowing models to continuously learn new concepts without forgetting old ones. However, MCIL typically suffers from catastrophic forgetting. In addition, it is difficult to implement many MCIL algorithms, which require to store data associated with the number of existing classes, on edge devices with limited memory. We believe that the data imbalance between the old and new classes and the utilization of old classes in the incremental process, which increases the memory requirements, are the main reasons for the above problems. Hence, we propose a novel approach, called ‘Open Set Incremental Learning(OSIL)’, to preserve the information of old classes, without storing any of their data, while making the classifier progressively learn the new classes and forget less information about the old classes. In OSIL, we present an open-set incremental strategy, which uses enhanced discrimination combined with open-set identification to overcome the imbalance between old classes and new classes. Meanwhile we demonstrate that attention mechanisms help to address catastrophic forgetting. The proposed method is evaluated on CIFAR-100 and ImageNet-100 under various settings. Experimental results show that OSIL effectively alleviates catastrophic forgetting and significantly outperforms state-of-the-art methods.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
•Posted Content
Learning without Forgetting
Zhizhong Li,Derek Hoiem +1 more
TL;DR: This work proposes the Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities, and performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques.
2.6K
Class-incremental continual learning model for plunger pump faults based on weight space meta-representation
TL;DR: Wang et al. as discussed by the authors proposed a continual learning model based on weight space meta-representation (WSMR) applied to class-incremental fault diagnosis of switch machine plunger pumps.
16
Open set maize seed variety classification using hyperspectral imaging coupled with a dual deep SVDD-based incremental learning framework
TL;DR: This paper proposes a dual deep SVDD-based incremental learning framework for open set maize seed variety classification using hyperspectral imaging, achieving above 91% accuracy in receiving known and rejecting unknown varieties.
12
A robust and anti-forgettiable model for class-incremental learning
Jianting Chen,Yang Qi Xiang +1 more
TL;DR: The results show that the performance of CIL algorithms based on the approach is better than that of the original algorithms, indicating that the approach can enhance the model robustness and help the networks extract anti-forgettable feature representations.
Discrimination Correction and Balance for Class-Incremental Learning
Ruixiang Wang,Yong Luo,Yitao Ren,Keming Mao +3 more
TL;DR: A novel class incremental learning method DCBIL is proposed, which adopts discrimination correction and balance to avoid model bias and can be used as a plug-in to strengthen the performance of existing incremental learning model.
References
Deep Residual Learning for Image Recognition
Kaiming He,Xiangyu Zhang,Shaoqing Ren,Jian Sun +3 more
- 27 Jun 2016
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Squeeze-and-Excitation Networks
Jie Hu,Li Shen,Samuel Albanie,Gang Sun,Enhua Wu +4 more
- 18 Jun 2018
TL;DR: This work proposes a novel architectural unit, which is term the "Squeeze-and-Excitation" (SE) block, that adaptively recalibrates channel-wise feature responses by explicitly modelling interdependencies between channels and finds that SE blocks produce significant performance improvements for existing state-of-the-art deep architectures at minimal additional computational cost.
Fast R-CNN
Ross Girshick
- 07 Dec 2015
TL;DR: Fast R-CNN as discussed by the authors proposes a Fast Region-based Convolutional Network method for object detection, which employs several innovations to improve training and testing speed while also increasing detection accuracy and achieves a higher mAP on PASCAL VOC 2012.
•Dissertation
Learning Multiple Layers of Features from Tiny Images
Alex Krizhevsky
- 01 Jan 2009
TL;DR: In this paper, the authors describe how to train a multi-layer generative model of natural images, using a dataset of millions of tiny colour images, described in the next section.
•Posted Content
Distilling the Knowledge in a Neural Network
TL;DR: This work shows that it can significantly improve the acoustic model of a heavily used commercial system by distilling the knowledge in an ensemble of models into a single model and introduces a new type of ensemble composed of one or more full models and many specialist models which learn to distinguish fine-grained classes that the full models confuse.
21.2K
Related Papers (5)
Sudhanshu Mittal,Silvio Galesso,Thomas Brox +2 more
- 04 May 2021