Journal Article10.1109/TCYB.2021.3067352
Accelerated Log-Regularized Convolutional Transform Learning and its Convergence Guarantee.
19
TL;DR: Wang et al. as discussed by the authors presented a new CTL framework with a log regularizer, which can not only obtain accurate representations but also yield strong sparsity, and provided a rigorous convergence analysis for the proposed algorithm under the accelerated PDCA.
read more
Abstract: Convolutional transform learning (CTL), learning filters by minimizing the data fidelity loss function in an unsupervised way, is becoming very pervasive, resulting from keeping the best of both worlds: the benefit of unsupervised learning and the success of the convolutional neural network. There have been growing interests in developing efficient CTL algorithms. However, developing a convergent and accelerated CTL algorithm with accurate representations simultaneously with proper sparsity is an open problem. This article presents a new CTL framework with a log regularizer that can not only obtain accurate representations but also yield strong sparsity. To efficiently address our nonconvex composite optimization, we propose to employ the proximal difference of the convex algorithm (PDCA) which relies on decomposing the nonconvex regularizer into the difference of two convex parts and then optimizes the convex subproblems. Furthermore, we introduce the extrapolation technology to accelerate the algorithm, leading to a fast and efficient CTL algorithm. In particular, we provide a rigorous convergence analysis for the proposed algorithm under the accelerated PDCA. The experimental results demonstrate that the proposed algorithm can converge more stably to desirable solutions with lower approximation error and simultaneously with stronger sparsity and, thus, learn filters efficiently. Meanwhile, the convergence speed is faster than the existing CTL algorithms.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Semi-supervised multi-view learning by using label propagation based non-negative matrix factorization
TL;DR: Zhang et al. as discussed by the authors proposed a label propagation based non-negative matrix factorization (LPNMF) method to address the problem of sparse labeled data, which can classify a large number of unlabeled data with few labeled data.
30
Compact Learning Model for Dynamic Off-Chain Routing in Blockchain-Based IoT
TL;DR: Wang et al. as discussed by the authors proposed a compact deep reinforcement learning (DRL) algorithm to learn the joint dynamic and lightweight routing policy for maximizing long-term transaction efficiency in payment channel network (PCN)-based IoT.
22
Structured Sparse Coding With the Group Log-regularizer for Key Frame Extraction
TL;DR: Wang et al. as mentioned in this paper proposed a nonconvex group log-regularizer with strong sparsity and a low reconstruction error for key frame extraction, which can reduce the redundancy of continuous frames and concisely express the entire video.
17
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks
Hao Sun,Li Shen,Qihuang Zhong,Liang Ding,Shi-Yong Chen,Jingwei Sun,Jing Li,Guangzhong Sun,Dacheng Tao +8 more
TL;DR: In this paper , the convergence rate of AdaSAM with adaptive learning rate and momentum acceleration is analyzed in the stochastic non-convex setting, and the authors theoretically show that AdaSAM admits a √ O(1/ √ bT ) convergence rate, which achieves linear speedup property with respect to mini-batch size.
AdaSAM: Boosting sharpness-aware minimization with adaptive learning rate and momentum for training deep neural networks
Hao Sun,Li Shen,Qihuang Zhong,Ding Liu,Shixiang Chen,Jing Sun,Jing Li,Guoqiang Sun,Dacheng Tao +8 more
TL;DR: AdaSAM optimizer achieves a O(1/bT) convergence rate in the stochastic non-convex setting, achieving linear speedup property with respect to mini-batch size.
14
References
•Dissertation
Learning Multiple Layers of Features from Tiny Images
Alex Krizhevsky
- 01 Jan 2009
TL;DR: In this paper, the authors describe how to train a multi-layer generative model of natural images, using a dataset of millions of tiny colour images, described in the next section.
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Amir Beck,Marc Teboulle +1 more
TL;DR: A new fast iterative shrinkage-thresholding algorithm (FISTA) which preserves the computational simplicity of ISTA but with a global rate of convergence which is proven to be significantly better, both theoretically and practically.
14.3K
An Iterative Thresholding Algorithm for Linear Inverse Problems with a Sparsity Constraint
TL;DR: It is proved that replacing the usual quadratic regularizing penalties by weighted 𝓁p‐penalized penalties on the coefficients of such expansions, with 1 ≤ p ≤ 2, still regularizes the problem.
Places: A 10 Million Image Database for Scene Recognition
TL;DR: The Places Database is described, a repository of 10 million scene photographs, labeled with scene semantic categories, comprising a large and diverse list of the types of environments encountered in the world, using the state-of-the-art Convolutional Neural Networks as baselines, that significantly outperform the previous approaches.
4.9K
Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods
TL;DR: This work proves an abstract convergence result for descent methods satisfying a sufficient-decrease assumption, and allowing a relative error tolerance, that guarantees the convergence of bounded sequences under the assumption that the function f satisfies the Kurdyka–Łojasiewicz inequality.