Download PDF

Events, the probability function and the Kolgomorov axioms

Conditional probability and Bayes' theorem

Entropy

Variables

Expected value, conditional expectation and Jensen's inequality

Variance and covariance

Higher moments

Markov's inequality and Chebyshev's inequality

Characteristic functions

Single observation discrete distributions

Simple continous distributions

Independent and identically distributed variables

The weak law of large numbers

Levy's continuity theorem

The central limit theorem and the gaussian/normal distribution

Statistics

Order statistics

Repeated observations discrete distributions

Extreme value distributions

Mixture distributions

Stochastic processes and their moments

White noise, and weak- and wide-sense stationarity

Random walks

Martingale processes

Markov processes

Multivariate time series

Bayesian networks

Survival functions

Orders of integration

Auto-Regressive processes, Moving-Average processes and Wold's theorem

Vector Autoregression (VAR)

ARMAX

Partial Adjustment Model (PAM)

Error Correction Model

Wiener processes and Brownian motion

Stochastic differential equations

Rejection sampling

Markov chain Monte Carlo sampling

Sampling from processes

Creating pseudo-random numbers

Stochastic methods for integration

Stochastic optimisation

Calculus of stochastic processes

Lossy compression

For a process with the Martingale property, the expected value of all future variables is the current state.

This only restricts expectations.

\(E(X_{n+1}|X_0,...,X_n)=X_n\)