# Stochastic processes

## Introduction to processes

### Stochastic processes

In a stochastic process we have a mapping from a variable (time) to a random variable.

#### Discrete and continuous time

Time could be discrete, or continuous.

Temperature over time is a stochastic process, as is the number of cars sold each day.

#### Discrete and continous state space

The state space for temperature is continous, the number of people on the moon is discrete.

### Stochastic evolution

We can describe processes by their evolution.

$$p(x_t|x_{t-1}...)$$

### Autocovariance and autocorrelation

#### Autocovariance

$$AC(a,b)=cov(X_a, X_b)$$

#### Autocorrelation

The autocorrelation between two time periods is their covariance, normlised by their variances

$$AC(a,b)=\dfrac{E[(X_a-\mu_a)(X_b-\mu_b)]}{\sigma_a \sigma_b}$$

This is also called serial correlation.

### The Markov and Martingale properties

#### Markov property

For a process with the Markov property, only the current state matters for all probability distributions.

$$P(x_{t+n}|x_t)=P(x_{t+n}|x_t, x_{t-1}...)$$

#### Martingale property

For a process with the Martingale property, the expected value of all future variables is the current state.

This only restricts expectations.

$$E(X_{n+1}|X_0,...,X_n)=X_n$$

#### Taxonomy

A process can be Markov, Martingale, neither or both.

## Stationarity

### Weak- and wide-sense stationarity

Unconditional probabilities don’t change over time.

So GDP would not be stationary, but random noise would. A random walk is not stationary, because the variance increases over time.

#### Order of integration

How many differences to make it stationary?

#### Weak-sense stationary

Mean and autocovarinance don’t change over time.

#### Wide-sense stationary

All moments are the same.

### White noise

Variables at each time are indepdendent.

### Orders of integration

How many diffs do you need to do to get a stationary process?

If something is first order integrated it is $$I(1)$$.

### Trend stationary

If we can remove the trend as a function, eg linear or non-linear growth, and the rest is stationary, then the process is trend stationary

## Ergodic processes

### Ergodic processes

Sample moments must converge to generating momements. Not guaranteed.

Eg process with path dependence. 50 Generating average is £50, but sample will only convergen to £100 or £0

## Markov chains

### Finite state Markov chains

#### Transition matrices

This shows the probability for moving between discrete states.

We can show the probability of being in a state by multiplying the vector state by the transition matrix.

$$Mv$$

#### Time-homogenous Markov chains

For time-homogenous Markov chains the transition matrix is independent of time.

For these we can calculate the probability of being in any given state in the future:

$$M^nv$$

This becomes independent of v as we tend to infinity. The initial starting state does not matter for long term probabilities.

How to find steady state probability?

$$Mv=v$$

The eigenvectors! With associated eigenvector $$1$$. There is only one eigenvector. We can find it by iteratively multiplying any vector by $$M$$.

## Multiple time series

### Cointegration

If we have multiple variables, we can explore the order of integration of linear combinations.

If two series have time trends, a linear combination of them could remove this.

### Exogeneity

#### Contemporaneous exogeneity

$$Cov(x_{it},u_{it})=0$$

#### Strict exogeneity

$$Cov (x_{is}, u_{it})=0)$$

This is stronger than contemporeous, all periods.

Shocks don’t affect future outcomes.

#### Sequential exogeneity

Sequential exogeneity: a bit looser than strict exogeneity. only holds when $$s\le t$$.

So shocks can affect, but only in future.