ARIMA - Two Common Processes

Autoregressive process
Most time series consist of elements that are serially dependent in the sense that one can estimate a coefficient or a set of coefficients that describe consecutive elements of the series from specific, time-lagged (previous) elements. This can be summarized in the equation:

where:

 ξ         is a constant (intercept)
 φ1, φ2, φ3 are the autoregressive model parameters
is the mean of x

Put in words, each observation is made up of a random error component (random shock, ε) and a linear combination of prior observations.

Stationarity requirement
Note that an autoregressive process will only be stable if the parameters are within a certain range; for example, if there is only one autoregressive parameter then it must fall within the interval of -1<φ1<+1. Otherwise, past effects would accumulate and the values of successive xt' s would move towards infinity, that is, the series would not be stationary. If there is more than one autoregressive parameter, similar (general) restrictions on the parameter values can be defined (e.g., see Box & Jenkins, 1976; Montgomery, 1990). The Time Series module automatically checks whether the stationarity requirement is met.
Moving average process
Independent from the autoregressive process, each element in the series can also be affected by the past error (or random shock) that cannot be accounted for by the autoregressive component, that is:

xt = μ + εt - θ1*ε(t-1) - θ2*ε(t-2) - θ3*ε(t-3) - ...

where

μ is a constant, and
θ1, θ2, θ3 are the moving average model parameters.

Put in words, each observation is made up of a random error component (random shock, ε) and a linear combination of prior random shocks.

Invertibility requirement
Without going into too much detail, there is a "duality" between the moving average process and the autoregressive process (e.g., see Box & Jenkins, 1976; Montgomery, Johnson, & Gardiner, 1990), that is, the moving average equation above can be rewritten (inverted) into an autoregressive form (of infinite order). However, analogous to the stationarity condition described above, this can only be done if the moving average parameters follow certain conditions, that is, if the model is invertible. Otherwise, the series will not be stationary. Again, the Time Series module automatically checks whether the invertibility requirement is met.