Stochastic processes

Introduction to processes

Stochastic processes

In a stochastic process we have a mapping from a variable (time) to a random variable.

Discrete and continuous time

Time could be discrete, or continuous.

Temperature over time is a stochastic process, as is the number of cars sold each day.

Discrete and continous state space

The state space for temperature is continous, the number of people on the moon is discrete.

Stochastic evolution

We can describe processes by their evolution.


Gaussian processes

Moments of stochastic processes

Autocovariance and autocorrelation


\(AC(a,b)=cov(X_a, X_b)\)


The autocorrelation between two time periods is their covariance, normlised by their variances

\(AC(a,b)=\dfrac{E[(X_a-\mu_a)(X_b-\mu_b)]}{\sigma_a \sigma_b}\)

This is also called serial correlation.

The Markov and Martingale properties

Markov property

For a process with the Markov property, only the current state matters for all probability distributions.

\(P(x_{t+n}|x_t)=P(x_{t+n}|x_t, x_{t-1}...)\)

Martingale property

For a process with the Martingale property, the expected value of all future variables is the current state.

This only restricts expectations.



A process can be Markov, Martingale, neither or both.


Weak- and wide-sense stationarity

Unconditional probabilities don’t change over time.

So GDP would not be stationary, but random noise would. A random walk is not stationary, because the variance increases over time.

Order of integration

How many differences to make it stationary?

Weak-sense stationary

Mean and autocovarinance don’t change over time.

Wide-sense stationary

All moments are the same.

Unit roots

White noise

Variables at each time are indepdendent.

Orders of integration

How many diffs do you need to do to get a stationary process?

If something is first order integrated it is \(I(1)\).

Trend stationary

If we can remove the trend as a function, eg linear or non-linear growth, and the rest is stationary, then the process is trend stationary

Ergodic processes

Ergodic processes

Sample moments must converge to generating momements. Not guaranteed.

Eg process with path dependence. 50 Generating average is £50, but sample will only convergen to £100 or £0

Processes decomposition (from uni forecasting?)

Wold’s theorem

Markov chains

Finite state Markov chains

Transition matrices

This shows the probability for moving between discrete states.

We can show the probability of being in a state by multiplying the vector state by the transition matrix.


Time-homogenous Markov chains

For time-homogenous Markov chains the transition matrix is independent of time.

For these we can calculate the probability of being in any given state in the future:


This becomes independent of v as we tend to infinity. The initial starting state does not matter for long term probabilities.

How to find steady state probability?


The eigenvectors! With associated eigenvector \(1\). There is only one eigenvector. We can find it by iteratively multiplying any vector by \(M\).

Random walks

Infinite state Markov chains

Multiple time series


If we have multiple variables, we can explore the order of integration of linear combinations.

If two series have time trends, a linear combination of them could remove this.


Contemporaneous exogeneity


Strict exogeneity

\(Cov (x_{is}, u_{it})=0)\)

This is stronger than contemporeous, all periods.

Shocks don’t affect future outcomes.

Sequential exogeneity

Sequential exogeneity: a bit looser than strict exogeneity. only holds when \(s\le t\).

So shocks can affect, but only in future.

Hidden Markov Models

Dynamic Bayesian networks