Open AccessPosted Content
Multi-Objective Meta Learning
TL;DR: In this paper, a unified gradient-based multi-objective meta learning (MOML) framework was proposed to solve the MOBLP by alternatively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient based multiobjective optimization method, respectively.
read more
Abstract: Meta learning with multiple objectives can be formulated as a Multi-Objective Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to solve several possible conflicting targets for the meta learner. However, existing studies either apply an inefficient evolutionary algorithm or linearly combine multiple objectives as a single-objective problem with the need to tune combination weights. In this paper, we propose a unified gradient-based Multi-Objective Meta Learning (MOML) framework and devise the first gradient-based optimization algorithm to solve the MOBLP by alternatively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Theoretically, we prove the convergence properties of the proposed gradient-based optimization algorithm. Empirically, we show the effectiveness of the proposed MOML framework in several meta learning problems, including few-shot learning, neural architecture search, domain adaptation, and multi-task learning.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Ingredient-oriented Multi-Degradation Learning for Image Restoration
Jinghao Zhang,Jie Huang,Min Yao,Zizheng Yang,Huikang Yu,Man Zhou,Fengmei Zhao +6 more
- 01 Jun 2023
TL;DR: This paper proposes a novel perspective to delve into the degradation via aningredients-oriented rather than previous task-oriented manner for scalable learning, and exhibits the favorable generalization ability to unknown downstream tasks.
59
Learning Conflict-Noticed Architecture for Multi-Task Learning
Zhixiong Yue,Yu Zhang,Jie Liang +2 more
TL;DR: In this article , a Conflict-Noticed Architecture Learning (CoNAL) method is proposed to alleviate the gradient conflict by learning architectures by introducing purely-specific modules specific to each task in the search space.
Effective, Efficient and Robust Neural Architecture Search
18 Jul 2022
TL;DR: In this paper , an Effective, Efficient, and Robust Neural Architecture Search (E2RNAS) method is proposed to automatically search a neural network architecture that balances the performance, robustness, and resource consumption.
6
EMT-NAS: Transferring architectural knowledge between tasks from different datasets
Peng Liao,Yaochu Jin,Wen Du +2 more
- 01 Jun 2023
TL;DR: An evolutionary multitasking neural architecture search (EMT-NAS) algorithm to accelerate the search process by transferring architectural knowledge across multiple related tasks and a fitness re-evaluation method is suggested to alleviate fluctuations in performance evaluations resulting from parameter sharing and the mini-batch gradient descent training method.
6
Sensor drift compensation for gas mixture classification in batch experiments
Yong-Taek Oh,Sungil Kim +1 more
TL;DR: Wang et al. as discussed by the authors proposed an end-to-end deep learning model comprising a drift-compensation module and classification module, which corrects the drifted sensor readings in batch experiments by minimizing a scatteredness-based fitness function.
3
References
Deep Residual Learning for Image Recognition
Kaiming He,Xiangyu Zhang,Shaoqing Ren,Jian Sun +3 more
- 27 Jun 2016
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
•Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
- 01 Jan 2015
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
138.5K
ImageNet: A large-scale hierarchical image database
Jia Deng,Wei Dong,Richard Socher,Li-Jia Li,Kai Li,Li Fei-Fei +5 more
- 20 Jun 2009
TL;DR: A new database called “ImageNet” is introduced, a large-scale ontology of images built upon the backbone of the WordNet structure, much larger in scale and diversity and much more accurate than the current image datasets.
•Dissertation
Learning Multiple Layers of Features from Tiny Images
Alex Krizhevsky
- 01 Jan 2009
TL;DR: In this paper, the authors describe how to train a multi-layer generative model of natural images, using a dataset of millions of tiny colour images, described in the next section.
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Amir Beck,Marc Teboulle +1 more
TL;DR: A new fast iterative shrinkage-thresholding algorithm (FISTA) which preserves the computational simplicity of ISTA but with a global rate of convergence which is proven to be significantly better, both theoretically and practically.
14.3K