Domain Generalization Using Ensemble Learning.
Yusuf Mesbah,Youssef Youssry Ibrahim,Adil Mehood Khan +2 more
- 02 Sep 2021
- pp 236-247
TL;DR: In this article, an ensemble model is built on top of base deep learning models trained on a single source to enhance the generalization of their collective prediction, and the results achieved thus far have demonstrated promising improvements of the ensemble over any of its base learners.
read more
Abstract: Domain generalization is a sub-field of transfer learning that aims at bridging the gap between two different domains in the absence of any knowledge about the target domain. Our approach tackles the problem of a model’s weak generalization when it is trained on a single source domain. From this perspective, we build an ensemble model on top of base deep learning models trained on a single source to enhance the generalization of their collective prediction. The results achieved thus far have demonstrated promising improvements of the ensemble over any of its base learners.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Figures
Fig. 1. Domain generalization is the problem of transferring the knowledge from a source domain (such as SVHN cropped on the left) to a different target domain (such as MNIST on the right) to solve the same task, with the absence of any knowledge regarding the distributional shift in the feature space of the inputs. 
Table 1. Results for object recognition experiments: (1) from CIFAR10 as the source domain to STL10 as the target domain, (2) from STL10 as the source domain to CIFAR10 as the target domain. 
Table 2. Results for digit recognition experiments: (1) from MNIST as the source domain to SVHN as the target domain, (2) from SVHN as the source domain to MNIST as the target domain. 
Table 3. Results for digit recognition experiments: (1) from USPS as the source domain to MNIST as the target domain, (2) from MNIST as the source domain to USPS as the target domain. 
Fig. 2. Base CNN used to learn CIFAR10 for the ensembles 
Table 4. Results for digit recognition experiments: (1) from USPS as the source domain to SVHN as the target domain, (2) from SVHN as the source domain to USPS as the target domain.
Citations
Diverse Weight Averaging for Out-of-Distribution Generalization
Alexandre Ramé,Matthieu Kirchmeyer,Thibaud Rahier,Alain Rakotomamonjy,Patrick Gallinari,Matthieu Cord +5 more
- 19 May 2022
TL;DR: Diverse Weight Averaging is proposed that makes a simple change to this strategy: DiWA averages the weights obtained from several independent training runs rather than from a single run, and highlights the need for diversity by a new bias-variance-covariance-locality decomposition of the expected error.
76
A Dual-Attention Deep Discriminative Domain Generalization Model for Hyperspectral Image Classification
Qingjie Zhao,Binglu Wang,Lei Wang,Wangwang Liu,Shanshan Li +4 more
TL;DR: A novel dual-attention deep discriminative domain generalization framework (DAD3GM) for cross-scene hyperspectral image classification without training the labeled target samples is proposed and achieves optimal results, which reveals its effectiveness and feasibility.
2
QT-DoG: Quantization-aware Training for Domain Generalization
Saqib Javed,Hieu Le,Mathieu Salzmann +2 more
- 08 Oct 2024
TL;DR: This paper proposes QT-DoG, a quantization-aware training method for domain generalization, which induces noise in model weights to find flatter minima, enhancing generalization across domains, and outperforms state-of-the-art DG approaches with reduced model size and computational overhead.
Test-Time Adaptation via Self-Training with Nearest Neighbor Information
Min-Uk Jang,Sae-Young Chung +1 more
- 08 Jul 2022
TL;DR: This work proposes a novel test-time adaptation method Test-time Adaptation via Self-Training with nearest neighbor information (TAST), based on the idea that a test data and its nearest neighbors in the embedding space of the trained classifier are more likely to have the same label.
References
ImageNet classification with deep convolutional neural networks
TL;DR: A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.
•Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky,Ilya Sutskever,Geoffrey E. Hinton +2 more
- 03 Dec 2012
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
A Survey on Transfer Learning
Sinno Jialin Pan,Qiang Yang +1 more
TL;DR: The relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift are discussed.
Reading Digits in Natural Images with Unsupervised Feature Learning
Yuval Netzer,Tao Wang,Adam Coates,Alessandro Bissacco,Bo Wu,Andrew Y. Ng +5 more
- 01 Jan 2011
TL;DR: A new benchmark dataset for research use is introduced containing over 600,000 labeled digits cropped from Street View images, and variants of two recently proposed unsupervised feature learning methods are employed, finding that they are convincingly superior on benchmarks.
Neural network ensembles
Lars Kai Hansen,Peter Salamon +1 more
TL;DR: It is shown that the remaining residual generalization error can be reduced by invoking ensembles of similar networks, which helps improve the performance and training of neural networks for classification.