Multiscale autoencoder-RNN architecture for mitigating error accumulation in long-term forecasting

Abstract

Recurrent Neural Networks (RNNs) have emerged as a vital tool for forecasting complex systems, ranging from natural language processing to high-dimensional dynamical systems. However, during deployment, accurate long-term forecasting requires iterative use of autoregressive scenarios, where predictions are fed back to the RNN as new inputs, leading to propagation and accumulation of error over time. This is further compounded when the state distributions vary, leading to inaccuracies in loss measurement. While previous studies have proposed solutions for language processing networks with probabilistic predictions, these approaches are inadequate for more complex systems. Therefore, in this study, we propose a novel approach to address this gap using a multiscale RNN architecture that leverages high-scale networks to reinitialize low-scale networks, thereby effectively mitigating iterative propagation of errors. To reduce the dimensionality, we integrate an auto encoder with the RNNs. Our experiments, using one of the most widely used RNNs, Long Short-Term Memory (LSTM) networks, demonstrate that our technique significantly reduces iterative errors and accurately forecasts long-term high-dimensional systems such as the FitzHugh-Nagumo model. This technique has the potential to enhance a wide range of applications, including robotics, finance, and climate modeling, by enabling precise long-term prediction of high-dimensional systems.

Cite this paper

@inproceedings{farooq2023multiscale,
  title   = {Multiscale autoencoder-RNN architecture for mitigating error accumulation in long-term forecasting},
  author  = {Farooq, Junaid and Bazaz, Mohammad Abid and Rafiq, Danish},
  booktitle = {ICETCI 2023, IEEE},
  year    = {2023},
  doi     = {10.1109/ICETCI58599.2023.10331086},
}

Topic: Scientific machine learning

All publications