Open AccessPosted Content
Learning without Forgetting
Zhizhong Li,Derek Hoiem +1 more
TL;DR: This work proposes the Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities, and performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques.
read more
Abstract: When building a unified vision system or gradually adding new capabilities to a system, the usual assumption is that training data for all tasks is always available. However, as the number of tasks grows, storing and retraining on such data becomes infeasible. A new problem arises where we add new capabilities to a Convolutional Neural Network (CNN), but the training data for its existing capabilities are unavailable. We propose our Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities. Our method performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques and performs similarly to multitask learning that uses original task data we assume unavailable. A more surprising observation is that Learning without Forgetting may be able to replace fine-tuning with similar old and new task datasets for improved new task performance.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Overcoming Catastrophic Forgetting for Multi-Label Class-Incremental Learning
Xiang Song,Kuang Shu,Songlin Dong,Jie Cheng,Xing Wei,Yihong Gong +5 more
- 03 Jan 2024
TL;DR: A novel AdaPtive Pseudo-Label-drivEn (APPLE) framework consisting of three components to solve the label absence problem, which leverages the old model to annotate old classes for new samples under the MLCIL setting better.
5
Privacy-preserving continual learning methods for medical image classification: a comparative analysis
Tanvi Verma,Liyuan Jin,Jun Zhou,Jia Huang,Mi-duo Tan,Benjamin Chen Ming Choong,Ting Fang Tan,Fei Gao,Xinxing Xu,D. Ting,Yong Liu +10 more
TL;DR: Continuous learning holds promise in mitigating catastrophic forgetting and facilitating continual model updates while preserving privacy in healthcare deep learning models, and presents a highly promising solution for the long-term clinical deployment of such models.
5
Distribution-Aware Knowledge Prototyping for Non-Exemplar Lifelong Person Re-Identification
Kunlun Xu,Xu Zou,Yuxin Peng,Jiahuan Zhou +3 more
- 16 Jun 2024
TL;DR: This paper proposes Distribution-aware Knowledge Prototyping (DKP) for lifelong person re-identification, addressing catastrophic forgetting and data privacy issues by modeling instance-level diversity and transforming it into identity-level distributions as prototypes for enhanced knowledge transfer and acquisition.
5
A Closer Look at Knowledge Distillation with Features, Logits, and Gradients
TL;DR: This work provides a new perspective to motivate a set of knowledge distillation strategies by approximating the classical KL-divergence criteria with different knowledge sources, making a systematic comparison possible in model compression and incremental learning.
5
Assessment of catastrophic forgetting in continual credit card fraud detection
B. Lebichot,W. Siblini,Gian Marco Paldino,Yann-Aël Le Borgne,F. Oblé,G. Bontempi +5 more
TL;DR: This study assesses catastrophic forgetting in continual credit card fraud detection, proposing an evaluation procedure and investigating six strategies and 13 methods on a real-size case study to maximize accuracy and minimize forgetting in e-commerce transactions.
5
References
ImageNet classification with deep convolutional neural networks
TL;DR: A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.
•Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
- 01 Jan 2015
TL;DR: In this paper, the authors investigated the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting and showed that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 layers.
51.9K
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky,Jia Deng,Hao Su,Jonathan Krause,Sanjeev Satheesh,Sean Ma,Zhiheng Huang,Andrej Karpathy,Aditya Khosla,Michael S. Bernstein,Alexander C. Berg,Li Fei-Fei +11 more
TL;DR: The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) as mentioned in this paper is a benchmark in object category classification and detection on hundreds of object categories and millions of images, which has been run annually from 2010 to present, attracting participation from more than fifty institutions.
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Ross Girshick,Jeff Donahue,Trevor Darrell,Jitendra Malik +3 more
- 23 Jun 2014
TL;DR: RCNN as discussed by the authors combines CNNs with bottom-up region proposals to localize and segment objects, and when labeled training data is scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, yields a significant performance boost.
Gradient-based learning applied to document recognition
Yann LeCun,Léon Bottou,Léon Bottou,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio,Patrick Haffner,Patrick Haffner +7 more
- 01 Jan 2001
TL;DR: This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task, and Convolutional neural networks are shown to outperform all other techniques.
32.7K