Deep Class-Incremental Learning From Decentralized Data
TL;DR: In this paper , a data-decentralized class-incremental learning (DCIL) framework is proposed to transfer knowledge from historical models and multiple local sites to the general model continually.
read more
Abstract: In this article, we focus on a new and challenging decentralized machine learning paradigm in which there are continuous inflows of data to be addressed and the data are stored in multiple repositories. We initiate the study of data-decentralized class-incremental learning (DCIL) by making the following contributions. First, we formulate the DCIL problem and develop the experimental protocol. Second, we introduce a paradigm to create a basic decentralized counterpart of typical (centralized) CIL approaches, and as a result, establish a benchmark for the DCIL study. Third, we further propose a decentralized composite knowledge incremental distillation (DCID) framework to transfer knowledge from historical models and multiple local sites to the general model continually. DCID consists of three main components, namely, local CIL, collaborated knowledge distillation (KD) among local models, and aggregated KD from local models to the general one. We comprehensively investigate our DCID framework by using a different implementation of the three components. Extensive experimental results demonstrate the effectiveness of our DCID framework. The source code of the baseline methods and the proposed DCIL is available at https://github.com/Vision-Intelligence-and-Robots-Group/DCIL.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Isolation and Impartial Aggregation: A Paradigm of Incremental Learning without Interference
TL;DR: Wang et al. as mentioned in this paper proposed an anchor-based energy self-normalization strategy to ensure the stage classifiers work at the same energy level, and designed a voting-based inference augmentation strategy for robust inference.
Learning from Crowds with Contrastive Representation
01 Jan 2023
TL;DR: In this paper , the authors investigate contrastive representation to improve learning from crowds, where they first sample confident instances and positive pairs using the pre-trained representation and human annotations, and then leverage the learned representations to train the classifier and annotator parameters.
2
Global prototype distillation for heterogeneous federated learning
Shu Wu,Jindou Chen,Xueli Nie,Yong Wang,Xiancun Zhou,Linlin Lu,Peng Wei,Yao Nie,Waseef Menhaj +8 more
TL;DR: Global prototype distillation (FedGPD) improves the performance of global model in heterogeneous federated learning by leveraging global class prototypes as knowledge to guide local training.
Flexibility and Privacy: A Multi-Head Federated Continual Learning Framework for Dynamic Edge Environments
Chunlu Chen,Kevin I-Kai Wang,Peng Li,Kouichi Sakurai +3 more
- 28 Nov 2023
TL;DR: A Federated Continual Learning (FCL) framework is presented that integrates the privacy-preserving benefits of Federated Learning into a continual learning system, ensuring both continual learning and privacy preservation in edge computing data processing and analysis.
1
Deep Learning: A Overview of Theory and Architectures
Hossein Shahinzadeh,Arezou Mahmoudi,Amirhosein Asilian,Hamidreza Sadrarhami,Mohammadreza Hemmati,Yaghoub Saberi +5 more
- 21 Feb 2024
References
Deep Residual Learning for Image Recognition
Kaiming He,Xiangyu Zhang,Shaoqing Ren,Jian Sun +3 more
- 27 Jun 2016
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Fully convolutional networks for semantic segmentation
Jonathan Long,Evan Shelhamer,Trevor Darrell +2 more
- 07 Jun 2015
TL;DR: The key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning.
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky,Jia Deng,Hao Su,Jonathan Krause,Sanjeev Satheesh,Sean Ma,Zhiheng Huang,Andrej Karpathy,Aditya Khosla,Michael S. Bernstein,Alexander C. Berg,Li Fei-Fei +11 more
TL;DR: The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) as mentioned in this paper is a benchmark in object category classification and detection on hundreds of object categories and millions of images, which has been run annually from 2010 to present, attracting participation from more than fifty institutions.
•Dissertation
Learning Multiple Layers of Features from Tiny Images
Alex Krizhevsky
- 01 Jan 2009
TL;DR: In this paper, the authors describe how to train a multi-layer generative model of natural images, using a dataset of millions of tiny colour images, described in the next section.
•Posted Content
Distilling the Knowledge in a Neural Network
TL;DR: This work shows that it can significantly improve the acoustic model of a heavily used commercial system by distilling the knowledge in an ensemble of models into a single model and introduces a new type of ensemble composed of one or more full models and many specialist models which learn to distinguish fine-grained classes that the full models confuse.
21.2K