A Memo for Time Series Analysis

Time series analysis focus on the analysis of random variable sequences. In this article, I would briefly review the basic definitions and properties of time series, the typical models, and the prediction of time series. The concepts and formulas are from Applicational Time Series Analysis by Shuyuan He, published by Peking University Express. Note that this is only a memo containing only key knowledge. If you are a beginner, please refer to other tutorials.

I picked up most of my statistics knowledge in my university in Mandarin. Thus there might be slight mistakes or imperfections with the terms and the grammars in this article. If you have any suggestions, you are more than appreciated to email me and help me to correct them.

Basic Definitions

In this section, the basic definition of time series related knowledges, including stationary time series, correlation function, white noise, and spectrum function, will be reviewed. Some useful properties of these definitions will be reviewed along with.

Time Series

A sequence of random variables ordered by time index is called a time series, represented as For a time series, its observed samples are called an implementation of the time series, represented as Most of the time, a time series is considered as the sum of 3 components: the trend component , the seasonal component , and the random component . Usually, and are considered as non-random functions and can be estimated by methods such as regression. This article would mainly focus on the random component, .

Stationary Time Series

A time series is called a stationary time series if it satisfies the following.

• For any ,
• For any ,
• For any ,

That is, for a stationary time series, for every term, their second moment is finite, and the expectations are a fixed value. For every two terms, their correlation is decided by the difference of their time index instead of the time index itself.

Correlation Function

For a stationary time series with mean value and correlation defined by the following formula. The sequence is called the correlation function of the stationary time series. And the correlation matrix is defined as follows. The correlation function and the correlation matrix satisfies the following properties.

• is non-negative definite for any
• for any

Estimation of Correlation Function

Given samples from a stationary time series, the correlation function can be estimated as follows.

Partial Correlation Coefficient

For stationary time series with positive definite , for , the Levinson recursive formula holds. with

White Noise

Let be a stationary time series, if for any , Then is called white noise, usually represented as .

White Noise Test

Given the estimation of the correlation coefficient for samples from a time series The following random variable approximately obeys -dimensional normal distribution. Thus, the following random variable approximately obeys the chi-square distribution with the degree of freedom . By applying the chi-square test, it would be possible to decide whether the samples are from white noise.

Spectrum Function

For stationary time series with correlation function , if there exists a non-decreasing and right continuous function defined in such that: Then is called the spectrum distribution function for .

If there exists a non-negative function such that: Then is called the spectrum density function for .

For stationary time series, its spectrum function uniquely exists. The spectrum density function for real value stationary time series is an odd function.

Spectrum Density Function for Stationary Time Series

If the correlation function for stationary time series is absolute summable, then has its spectrum density function

Spectrum Density Function for Moving Average

Let be a , sequence are square summable, then the moving average over has spectrum density function

Hilbert Space for Stationary Time Series

Let be the set containing all limited linear combination of random variables from stationary time series : Then is a linear space. With the inner product for element and defined as , is a complete inner product space, also called a Hilbert space.

Time Series Models

In this section, 3 typical time series models: AR, MA, and ARMA, and their related properties will be reviewed.

AR(p) Model

Let be a , and be a polynomial that satisfies the differential equation is called an auto regression model, or model. If a stationary time series satisfies the model, it is called an series.

Stable Solution of AR(p) Model

For model, its only stable solution is defined as follows. Sequence is called the Wold coefficient of the time series. The Wold coefficient can be solved by the following recursive formula.

Spectrum Density Function of AR(p) Series

For series , its spectrum density function is

Yule-Walker Equation

For series, the following equation holds. with Yule-Walker equation can be used to calculate the correlation function given , or estimate given estimated correlation function.

MA(q) Model

Let be a , and be a polynomial that satisfies the equation is called a moving average model, or model. If a stationary time series satisfies the model, it is called a series.

Correlation Function for MA(q) Series

Given a series, its correlation function can be calculated as follows.

Spectrum Density Function for MA(q) Series

Given a series, its spectrum density function can be calculated as follows.

ARMA(p, q) Model

Let be a , and be polynomials having different roots, , , and the differential equation is called an auto regression moving average model, or an model. If a stationary time series satisfies the model, it is called an series.

Stable Solution of ARMA(p, q) Model

For model, its only stable solution is defined as follows. Sequence is called the Wold coefficient of the time series. The Wold coefficient can be solved by the following recursive formula.

Correlation Function for MA(q) Series

Given an series, its correlation function can be calculated as follows.

Spectrum Density Function for MA(q) Series

Given an series, its spectrum density function can be calculated as follows.

Time Series Prediction

One goal of analyzing time series is to make better a prediction for its futural terms. Since a time series can be considered as the sum of a trend component, a seasonal component, and a random component, and the random component is usually stationary, this section mainly focuses on the prediction for stationary time series.

Linear Prediction for Time Series

Let and be random variables with as their mean value and limited variance, if there exist , such that for any , then is called the best linear prediction for by , represented as .

Mean Square Error for Best Linear Prediction

If , such that then the following equations holds.

Stepwise Prediction for Time Series

Let be a stationary time series with mean value . If its correlation matrix is positive definite, then the best linear prediction can be derived from the prediction errors: The coefficients and the prediction square error satisfies the following recursive formula.