Download PDF
Events, the probability function and the Kolgomorov axioms
Conditional probability and Bayes' theorem
Entropy
Variables
Expected value, conditional expectation and Jensen's inequality
Variance and covariance
Higher moments
Markov's inequality and Chebyshev's inequality
Characteristic functions
Degenerate, Bernoulli and categorical distributions
Simple continuous distributions
Independent and identically distributed variables
The weak law of large numbers
Levy's continuity theorem
The central limit theorem and the gaussian/normal distribution
Statistics
Order statistics
Totals of independent draws: Binominal and Poisson distributions
Time between draws: geometric and exponential distributions
Extreme value distributions
The geometric distribution
Mixture distributions
Latent class analysis and the expectation-maximisation algorithm
The empirical distribution
Data cleaning
Summary statistics and visualisation for one variable
Testing population means with Z-tests and T-tests
Pivotal quantities
Jackknifing
Bootstrapping
Non-parametric estimation of probability distributions
Bayesian parameter estimation
Point estimates of probability distributions
Likelihood functions
The score, Fisher information and orthogonality
Quasi-likelihood functions
Maximum Likelihood Estimation (MLE)
Maximum A-Priori (MAP) estimation
The Method Of Moments (MOM)
Testing generative parameter estimates with Z-tests and T-tests
Choosing parametric probability distributions
Estimating population moments
A pivotal quantity is a statistic whose distribution does not depend on the parameters of the underlying distribution.
For example, the z statistic if the underyling distribution is a normal distribution.