From ARIMA models to Recurrent Neural Networks

Recently I have been applying a range of different time series models to rich transport industry data. From anticipating the development of faulty parts to understanding the flow of traffic and how to better strategize timetabling adjustments, time series modelling is extremely powerful at improving the efficiency of organisations and cutting costs. So how do we go from ARIMA models to recurrent neural networks? The answer lies in the complexity of the data and hypotheses formulated.

The typical workflow of time series modelling is highlighted below:

1). Start simple with basic ARIMA models. Once you understand your data, the question you want to ask, what you want to predict and have shaped your data and engineered features accordingly a time series model can be applied.

The figure below highlights the steps involved in modelling time series data. There are two main approaches for fitting simple time series models using R. Firstly, an auto.arima function can be applied which automates steps 3 – 5, to fit a model with appropriate parameters selected. Alternatively, each step can be carried out separately, and the output (AIC) of several models compared.

2) Dynamic regression. Whilst basic time series models may give adequate predictions for one parameter, they are likely to be improved with the consideration of additional information. For example, when predicting how late a bus will be at its next stop, a good prediction could be made by modelling the lateness throughout its journey so far, but a better prediction may be made by enriching the input data to also take into account the weather at different time points, the distance left to travel and so forth. Dynamic regressions, model time series data and make predictions based on this additional information.

3). Recurrent Neural Networks. The decision to use RNNs must be carefully evaluated given the drawbacks associated with explaining and visualising model outputs or predictions. Justifications for using RNNS include trying to obtain better predictions of time series data that shows subtle complex patterns or where you have a data set of multiple time series and multiple influential features which you would like to capture the interactions between in order to make predictions. For example, if you are trying to predict lateness of multiple buses on a network taking into consideration the lateness over journeys of multiple buses, journey paths of different buses on the network, distance of bus journeys, age of buses, passenger capacity, weather, etc. This type of RNN would be a many to many RNN but other architectures of RNN exist as shown below.