Journal Article
Neural Network Optimization for Reinforcement Learning Tasks Using Sparse Computations
TL;DR: This method combines two ideas: neural network pruning and taking into account input data correlations; it makes it possible to update neuron states only when changes in them exceed a certain threshold, and reduces the number of multiplications when running neural networks.
read more
Abstract: This article proposes a sparse computation-based method for optimizing neural networks for reinforcement learning (RL) tasks. This method combines two ideas: neural network pruning and taking into account input data correlations; it makes it possible to update neuron states only when changes in them exceed a certain threshold. It significantly reduces the number of multiplications when running neural networks. We tested different RL tasks and achieved 20-150x reduction in the number of multiplications. There were no substantial performance losses; sometimes the performance even improved.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Figures
Table 3: Number of multiplications in an unoptimized neural network 
Figure 4: Results for Robotank, BattleZone, Enduro and Freeway. 
Figure 1: DQN architecture. DQN consists of three convolutional layers and two dense layers. This architecture is suitable for all the video games (the number of outputs at the last layer is the only value that changes) 
Table 1: Deep Q-Network structure 
Table 4: Number of multiplications in Breakout with 0.79 sparsity and 0.001 threshold 
Table 5: Number of multiplications in SpaceInvaders with 0.74 sparsity and 0.001 threshold
References
Human-level control through deep reinforcement learning
Volodymyr Mnih,Koray Kavukcuoglu,David Silver,Andrei Rusu,Joel Veness,Marc G. Bellemare,Alex Graves,Martin Riedmiller,Andreas K. Fidjeland,Georg Ostrovski,Stig Petersen,Charles Beattie,Amir Sadik,Ioannis Antonoglou,Helen King,Dharshan Kumaran,Daan Wierstra,Shane Legg,Demis Hassabis +18 more
TL;DR: This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.
•Proceedings Article
Optimal Brain Damage
Yann LeCun,John S. Denker,Sara A. Solla +2 more
- 01 Jan 1989
TL;DR: A class of practical and nearly optimal schemes for adapting the size of a neural network by using second-derivative information to make a tradeoff between network complexity and training set error is derived.
•Proceedings Article
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks.
Jonathan Frankle,Michael Carbin +1 more
- 04 Mar 2019
TL;DR: This work finds that dense, randomly-initialized, feed-forward networks contain subnetworks ("winning tickets") that - when trained in isolation - reach test accuracy comparable to the original network in a similar number of iterations, and articulate the "lottery ticket hypothesis".
•Proceedings Article
Second order derivatives for network pruning: Optimal Brain Surgeon
Babak Hassibi,David G. Stork +1 more
- 30 Nov 1992
TL;DR: Of OBS, Optimal Brain Damage, and magnitude-based methods, only OBS deletes the correct weights from a trained XOR network in every case, and thus yields better generalization on test data.
•Journal Article
Double Q-Learning
TL;DR: An alternative way to approximate the maximum expected value for any set of random variables is introduced and the obtained double estimator method is shown to sometimes underestimate rather than overestimate themaximum expected value.
1K