Proceedings Article10.1109/ICAIIC.2019.8668843
DecMac: A Deep Context Model for High Efficiency Arithmetic Coding
Qian Liu,Yiling Xu,Zhu Li +2 more
- 01 Feb 2019
- pp 438-443
16
TL;DR: A deep context model, named DecMac, is proposed, which combines a three-layer LSTM with adaptive arithmetic coding for lossless compression and introduces a cycle connection to preserve the end of hidden states and reuse it as the initial states for the next batch.
read more
Abstract: Conventional lossless compression techniques that use look up table method tend to be inefficient. We propose a deep context model, named DecMac, which combines a three-layer LSTM with adaptive arithmetic coding for lossless compression. In order to capture much more context information for better predicting, we introduce a cycle connection to preserve the end of hidden states and reuse it as the initial states for the next batch. We evaluate our method on the text compression task, resulting in averaged 25% compressed size reduction over the state of the art PAQ, and averaged 45% reduction over GZIP and ZIP.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Language Modeling Is Compression
Gr'egoire Del'etang,Anian Ruoss,Paul-Ambroise Duquenne,Elliot Catt,Tim Genewein,Christopher Mattern,Jordi Grau-Moya,Wenliang Kevin Li,Matthew Aitchison,Laurent Orseau,M. Hutter,Joel Veness +11 more
TL;DR: It is shown that large language models are powerful general-purpose predictors and that the compression viewpoint provides novel insights into scaling laws, tokenization, and in-context learning and the prediction-compression equivalence allows us to use any compressor to build a conditional generative model.
LFZip: Lossy Compression of Multivariate Floating-Point Time Series Data via Improved Prediction
Shubham Chandak,Kedar Tatwawadi,Wen Chengtao,Lingyun Wang,Juan Aparicio Ojea,Tsachy Weissman +5 more
- 24 Mar 2020
TL;DR: LFZip as discussed by the authors proposes an error-bounded lossy compressor for multivariate floating-point time series data that provides guaranteed reconstruction up to user-specified maximum absolute error.
•Posted Content
DZip: improved general-purpose lossless compression based on novel neural network modeling
TL;DR: DZip as discussed by the authors is a general-purpose compressor for sequential data that exploits the well-known modeling capabilities of neural networks (NNs) for prediction, followed by arithmetic coding.
25
•Posted Content
LFZip: Lossy compression of multivariate floating-point time series data via improved prediction.
TL;DR: This work proposes an error-bounded lossy compressor, LFZip, for multivariate floating-point time series data that provides guaranteed reconstruction up to user-specified maximum absolute error.
TRACE: A Fast Transformer-based General-Purpose Lossless Compressor
Yushun Mao,Yufei Cui,Tei-Wei Kuo,Chun Jason Xue +3 more
- 30 Mar 2022
TL;DR: A fast general-purpose lossless compressor is proposed, TRACE, by designing a compression-friendly structure based on a single-layer transformer, which achieves an overall ∼ 3x speedup while keeps a comparable compression ratio to the state-of-the-art compressors.
References
•Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
- 01 Jan 2015
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
138.5K
Long short-term memory
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
99K
A mathematical theory of communication
TL;DR: This final installment of the paper considers the case where the signals or the messages or both are continuously variable, in contrast with the discrete nature assumed until now.
74.4K
•Journal Article
Dropout: a simple way to prevent neural networks from overfitting
TL;DR: It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
Speech recognition with deep recurrent neural networks
Alex Graves,Abdelrahman Mohamed,Geoffrey E. Hinton +2 more
- 26 May 2013
TL;DR: This paper investigates deep recurrent neural networks, which combine the multiple levels of representation that have proved so effective in deep networks with the flexible use of long range context that empowers RNNs.
10.4K