Download PDF

Events, the probability function and the Kolgomorov axioms

Conditional probability and Bayes' theorem

Entropy

Variables

Expected value, conditional expectation and Jensen's inequality

Variance and covariance

Higher moments

Markov's inequality and Chebyshev's inequality

Characteristic functions

Degenerate, Bernoulli and categorical distributions

Simple continuous distributions

Independent and identically distributed variables

The weak law of large numbers

Levy's continuity theorem

The central limit theorem and the gaussian/normal distribution

Statistics

Order statistics

Totals of independent draws: Binominal and Poisson distributions

Time between draws: geometric and exponential distributions

Extreme value distributions

The geometric distribution

Mixture distributions

Latent class analysis and the expectation-maximisation algorithm

The empirical distribution

Data cleaning

Summary statistics and visualisation for one variable

Testing population means with Z-tests and T-tests

Pivotal quantities

Jackknifing

Bootstrapping

Non-parametric estimation of probability distributions

Bayesian parameter estimation

Point estimates of probability distributions

Likelihood functions

The score, Fisher information and orthogonality

Quasi-likelihood functions

Maximum Likelihood Estimation (MLE)

Maximum A-Priori (MAP) estimation

The Method Of Moments (MOM)

Testing generative parameter estimates with Z-tests and T-tests

Choosing parametric probability distributions

Estimating population moments

Creating pseudo-random numbers

Stochastic methods for integration

Stochastic optimisation

Calculus of stochastic processes

Lossy compression

Non-cryptographic hashes

Rejection sampling

Please select a chapter from the left.

This is a live document, and is full of gaps, mistakes, typos etc.