# Variance and covariance

## Introduction

### Variance

#### Definition

The variance of a random variable is given by:

$$Var(x)=E((x-E(x))^2)$$

$$Var(x)=E(x^2+E(x)^2-2xE(x))$$

$$Var(x)=E(x^2)+E(E(x)^2)-E(2xE(x))$$

$$Var(x)=E(x^2)+E(x)^2-2E(x)^2$$

$$Var(x)=E(x^2)-E(x)^2$$

#### Variance of a constant

$$Var(c)=E(c^2)-E(c)^2$$

$$Var(c)= c^2-c^2$$

$$Var(c)=0$$

#### Variance of multiple

$$Var(cx)=E((cx)^2)-E(cx)^2$$

$$Var(cx)=E(c^2x^2)-[\sum_i cx P(x_i)]^2$$

$$Var(cx)=c^2E(x^2)-c^2[\sum_i x P(x_i)]^2$$

$$Var(cx)=c^2[E(x^2)- E(x)^2]$$

$$Var(cx)=c^2Var(x)$$

$$E(x)^2+Var(x)=E(x)^2+E((x-E(x))^2)$$

$$E(x)^2+Var(x)=E(x)^2+E(x^2+E(x)^2-2xE(x))$$

$$E(x)^2+Var(x)=E(x)^2+E(x^2)+E(E(x)^2)-E(2xE(x))$$

$$E(x)^2+Var(x)=E(x)^2+E(x^2)+E(x)^2-2E(x)E(x))$$

$$E(x)^2+Var(x)=E(x^2)$$

#### Covariance

$$Var(x+y)=E((x+y)^2)-E(x+y)^2$$

$$Var(x+y)=E(x^2+y^2+2xy)-E(x+y)^2$$

$$Var(x+y)=E(x^2)+E(y^2)+E(2xy)-E(x+y)^2$$

$$Var(x+y)=E(x^2)+E(y^2)+E(2xy)-[E(x)+E(y)]^2$$

$$Var(x+y)=E(x^2)+E(y^2)+E(2xy)-E(x)^2-E(y)^2-2E(x)E(y)]$$

$$Var(x+y)=[E(x^2)-E(x)^2]+[E(y^2)-E(y)^2]+E(2xy)-2E(x)E(y)$$

$$Var(x+y)=Var(x) +Var(y)+2[E(xy)-E(x)E(y)]$$

We then define:

$$Cov(x,y):=E(xy)-E(x)E(y)$$

Noting that:

$$Cov(x,x)=E(xx)-E(x)E(x)$$

$$Cov(x,x)=Var(x)$$

So:

$$Var(x+y)=Var(x)+Var(y)+2Cov(x,y)$$

$$Var(x+y)=Cov(x,x)+Cov(x,y)+Cov(y,x)+Cov(y,y)$$

$$Cov(x,c)=E(xc)-E(x)E(c)$$

$$Cov(x,c)=cE(x)-cE(x)$$

$$Cov(x,c)=0$$

### Covariance matrix

With multiple events, covariance can be defined between each pair of events, including the event with itself.

The covariance between $$2$$ variables is:

$$Cov(x_i,x_j):=E(x_ix_j)-E(x_i)E(x_j)$$

Which is equal to:

$$Cov(x_i,x_j)=E{[x_i-E(x_i)][x_j-E(x_j)]}$$

We can therefore generate a covariance matrix through:

$$\sum =E[(X-E[X])(X-E[X])^T]$$