Learning sparse representations in reinforcement learning with sparse coding
Lei Le,Raksha Kumaraswamy,Martha White +2 more
- 19 Aug 2017
- pp 2067-2073
TL;DR: In this article, a supervised sparse coding objective for policy evaluation is developed, and the authors show that all local minima are global minima, making the approach amenable to simple optimization strategies.
read more
Abstract: A variety of representation learning approaches have been investigated for reinforcement learning; much less attention, however, has been given to investigating the utility of sparse coding. Outside of reinforcement learning, sparse coding representations have been widely used, with non-convex objectives that result in discriminative representations. In this work, we develop a supervised sparse coding objective for policy evaluation. Despite the non-convexity of this objective, we prove that all local minima are global minima, making the approach amenable to simple optimization strategies. We empirically show that it is key to use a supervised objective, rather than the more straightforward unsupervised sparse coding approach. We compare the learned representations to a canonical fixed sparse representation, called tile-coding, demonstrating that the sparse coding representation outperforms a wide variety of tile-coding representations.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Feature Selection by Singular Value Decomposition for Reinforcement Learning
Bahram Behzadian
- 01 Jan 2019
TL;DR: This work proposes a new method for feature selection, which is based on a low-rank factorization of the transition matrix, which derives features directly from high-dimensional raw inputs, such as image data.
Dictionary Learning-Based Reinforcement Learning with Non-convex Sparsity Regularizer
TL;DR: Wang et al. as discussed by the authors proposed a dictionary learning-based RL with the non-convex sparsity regularizer for RL control, which can capture more essential features from states and avoid unnecessary ones to avoid interference.
6
Image Classification based on Sparse Representation in the Quaternion Wavelet Domain
01 Jan 2022
TL;DR: Wang et al. as discussed by the authors proposed a novel sparse representation learning method in the Quaternion Wavelet (QW) domain for multi-class image classification, which takes advantage from: i) the QW decomposition, which promotes sparsity and provides structural information about the image data while allowing approximate shift-invariance, to extract meaningful features from low-frequency QW subbands, ii) the dimensionality reduction method using Principal Component Analysis (PCA) for reducing the complexity of the problem, and iii) the sparse representation of the generated QW features to efficiently learn and capture the meaningful and compact information of this data.
4
Deep sparse representation via deep dictionary learning for reinforcement learning
25 Jul 2022
TL;DR: Zhang et al. as discussed by the authors employed the deep dictionary learning (DDL) model for reinforcement learning as a joint optimization problem over the deep sparse representation and the function approximation, which considers a multi-layer dictionary for obtaining a deeper latent sparse representation with better data representation capability.
3
Dynamic sparse coding-based value estimation network for deep reinforcement learning
Haoli Zhao,Zhenni Li,Wensheng Su,Shengli Xie +3 more
TL;DR: Dynamic sparse coding-based value estimation network for deep reinforcement learning improves control performance and training efficiency by learning sparse representations and pruning weights dynamically.
3
References
Human-level control through deep reinforcement learning
Volodymyr Mnih,Koray Kavukcuoglu,David Silver,Andrei Rusu,Joel Veness,Marc G. Bellemare,Alex Graves,Martin Riedmiller,Andreas K. Fidjeland,Georg Ostrovski,Stig Petersen,Charles Beattie,Amir Sadik,Ioannis Antonoglou,Helen King,Dharshan Kumaran,Daan Wierstra,Shane Legg,Demis Hassabis +18 more
TL;DR: This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.
$rm K$ -SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
TL;DR: A novel algorithm for adapting dictionaries in order to achieve sparse signal representations, the K-SVD algorithm, an iterative method that alternates between sparse coding of the examples based on the current dictionary and a process of updating the dictionary atoms to better fit the data.
10K
Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1 ?
TL;DR: These deviations from linearity provide a potential explanation for the weak forms of non-linearity observed in the response properties of cortical simple cells, and they further make predictions about the expected interactions among units in response to naturalistic stimuli.
4.2K
Online Learning for Matrix Factorization and Sparse Coding
TL;DR: In this paper, a new online optimization algorithm based on stochastic approximations is proposed to solve the large-scale matrix factorization problem, which scales up gracefully to large data sets with millions of training samples.
A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion
Yangyang Xu,Wotao Yin +1 more
TL;DR: This paper considers regularized block multiconvex optimization, where the feasible set and objective function are generally nonconvex but convex in each block of variables and proposes a generalized block coordinate descent method.