A Bayesian Approach to Time Series Forecasting

Today we are going to implement a Bayesian linear regression in R from scratch and use it to forecast US GDP growth. This post is based on a very informative manual from the Bank of England on Applied Bayesian Econometrics . I have translated the original Matlab code into R since its open source and widely used in data analysis/science. My main goal in this post is to try and give people a better understanding of Bayesian statistics, some of it’s advantages and also some scenarios where you might want to use it.

Let’s take a moment to think about why we would we even want to use Bayesian techniques in the first place. Well, there are a couple of advantages in doing so and these are particularly attractive for time series analysis. One issue when working with time series models is over-fitting particularly when estimating models with large numbers of parameters over relatively short time periods. This is not such a problem in this particular case but certainly can be when looking at multiple variables which is quite common in economic forecasting. One solution to the over-fitting problem, is to take a Bayesian approach which allows us to impose certain priors on our variables.

To understand why this works, consider the case of a Ridge Regression (L2 penalty). This is a regularisation technique helping us to reduce over-fitting (good explanation of ridge regression ) by penalising us when the parameter values get large. If we instead took a Bayesian approach to the regression problem and used a normal prior we would essentially be doing the exact same thing as a ridge regression. Here is a video going through the derivation to prove that they are the same (really good course BTW). Another big reason we often prefer to use Bayesian methods is that it allows us to incorporate uncertainty in our parameter estimates which is particularly useful when forecasting.