A dual-stage attention-based recurrent neural network for time series prediction
Yao Qin,Dongjin Song,Haifeng Cheng,Wei Cheng,Guofei Jiang,Garrison W. Cottrell +5 more
- 19 Aug 2017
- pp 2627-2633
TL;DR: Zhang et al. as discussed by the authors proposed a dual-stage attention-based recurrent neural network (DA-RNN) to capture long-term temporal dependencies appropriately and select the relevant driving series to make predictions.
read more
Abstract: The Nonlinear autoregressive exogenous (NARX) model, which predicts the current value of a time series based upon its previous values as well as the current and past values of multiple driving (exogenous) series, has been studied for decades. Despite the fact that various NARX models have been developed, few of them can capture the long-term temporal dependencies appropriately and select the relevant driving series to make predictions. In this paper, we propose a dual-stage attention-based recurrent neural network (DA-RNN) to address these two issues. In the first stage, we introduce an input attention mechanism to adaptively extract relevant driving series (a.k.a., input features) at each time step by referring to the previous encoder hidden state. In the second stage, we use a temporal attention mechanism to select relevant encoder hidden states across all time steps. With this dual-stage attention scheme, our model can not only make predictions effectively, but can also be easily interpreted. Thorough empirical studies based upon the SML 2010 dataset and the NASDAQ 100 Stock dataset demonstrate that the DA-RNN can outperform state-of-the-art methods for time series prediction.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Improved predictive deep temporal neural networks with trend filtering
Youngjin Park,Deokjun Eom,Byoung Ki Seo,Jaesik Choi +3 more
- 15 Oct 2020
TL;DR: In this article, the authors proposed a new prediction framework based on deep neural networks and a trend filtering, which converts noisy time series data into a piecewise linear fashion, and showed that the predictive performance of deep temporal neural networks improves when the training data is temporally processed by a trend filter.
SASTA-Net: self-attention spatiotemporal adversarial network for typhoon prediction
TL;DR: Li et al. as discussed by the authors proposed a self-attentional spatiotemporal adversarial network (SASTA-Net) to solve the problems of poor authenticity and lack of clarity for short-time typhoon prediction.
•Posted Content
Graph Deep Factors for Forecasting.
TL;DR: This work proposes a deep hybrid probabilistic graph-based forecasting framework called Graph Deep Factors (GraphDF), a hybrid forecasting framework that consists of a relational global and relational local model that learns complex non-linear time-series patterns globally using the structure of the graph to improve both forecasting accuracy and computational efficiency.
Time-series to Image-transformed Adversarial Autoencoder for Anomaly Detection
Jiyoung Kang,Minseok Kim,Jinuk Park,Sanghyun Park +3 more
TL;DR: This study proposes T2IAE, a novel approach for anomaly detection in multivariate time-series data, leveraging image transformation and CNN-based adversarial learning to effectively capture local features and distinguish anomalies from normal data with superior performance.
Context-Aware Adaptive Anomaly Detection in IoT Systems
Rozhin Yasaei,Mohammad Abdullah Al Faruque +1 more
- 07 Oct 2023
TL;DR: Context-aware adaptive anomaly detection in IoT systems utilizes sensor association, LSTM, and Gaussian estimator to detect anomalies in real-time with high precision.
References
•Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
- 01 Jan 2015
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
138.5K
Long short-term memory
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
99K
•Posted Content
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: In this article, the adaptive estimates of lower-order moments are used for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimate of lowerorder moments.
82.5K
Learning representations by back-propagating errors
TL;DR: Back-propagation repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector, which helps to represent important features of the task domain.
30.1K
Learning Phrase Representations using RNN Encoder--Decoder for Statistical Machine Translation
Kyunghyun Cho,Bart van Merriënboer,Caglar Gulcehre,Dzmitry Bahdanau,Fethi Bougares,Holger Schwenk,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio +8 more
- 01 Jan 2014
TL;DR: In this paper, the encoder and decoder of the RNN Encoder-Decoder model are jointly trained to maximize the conditional probability of a target sequence given a source sequence.