The variance of a random variable is given by:
\(Var(x)=E((x-E(x))^2)\)
\(Var(x)=E(x^2+E(x)^2-2xE(x))\)
\(Var(x)=E(x^2)+E(E(x)^2)-E(2xE(x))\)
\(Var(x)=E(x^2)+E(x)^2-2E(x)^2\)
\(Var(x)=E(x^2)-E(x)^2\)
\(Var(c)=E(c^2)-E(c)^2\)
\(Var(c)= c^2-c^2\)
\(Var(c)=0\)
\(Var(cx)=E((cx)^2)-E(cx)^2\)
\(Var(cx)=E(c^2x^2)-[\sum_i cx P(x_i)]^2\)
\(Var(cx)=c^2E(x^2)-c^2[\sum_i x P(x_i)]^2\)
\(Var(cx)=c^2[E(x^2)- E(x)^2]\)
\(Var(cx)=c^2Var(x)\)
\(E(x)^2+Var(x)=E(x)^2+E((x-E(x))^2)\)
\(E(x)^2+Var(x)=E(x)^2+E(x^2+E(x)^2-2xE(x))\)
\(E(x)^2+Var(x)=E(x)^2+E(x^2)+E(E(x)^2)-E(2xE(x))\)
\(E(x)^2+Var(x)=E(x)^2+E(x^2)+E(x)^2-2E(x)E(x))\)
\(E(x)^2+Var(x)=E(x^2)\)
\(Var(x+y)=E((x+y)^2)-E(x+y)^2\)
\(Var(x+y)=E(x^2+y^2+2xy)-E(x+y)^2\)
\(Var(x+y)=E(x^2)+E(y^2)+E(2xy)-E(x+y)^2\)
\(Var(x+y)=E(x^2)+E(y^2)+E(2xy)-[E(x)+E(y)]^2\)
\(Var(x+y)=E(x^2)+E(y^2)+E(2xy)-E(x)^2-E(y)^2-2E(x)E(y)]\)
\(Var(x+y)=[E(x^2)-E(x)^2]+[E(y^2)-E(y)^2]+E(2xy)-2E(x)E(y)\)
\(Var(x+y)=Var(x) +Var(y)+2[E(xy)-E(x)E(y)]\)
We then define:
\(Cov(x,y):=E(xy)-E(x)E(y)\)
Noting that:
\(Cov(x,x)=E(xx)-E(x)E(x)\)
\(Cov(x,x)=Var(x)\)
So:
\(Var(x+y)=Var(x)+Var(y)+2Cov(x,y)\)
\(Var(x+y)=Cov(x,x)+Cov(x,y)+Cov(y,x)+Cov(y,y)\)
\(Cov(x,c)=E(xc)-E(x)E(c)\)
\(Cov(x,c)=cE(x)-cE(x)\)
\(Cov(x,c)=0\)
With multiple events, covariance can be defined between each pair of events, including the event with itself.
The covariance between \(2\) variables is:
\(Cov(x_i,x_j):=E(x_ix_j)-E(x_i)E(x_j)\)
Which is equal to:
\(Cov(x_i,x_j)=E{[x_i-E(x_i)][x_j-E(x_j)]}\)
We can therefore generate a covariance matrix through:
\(\sum =E[(X-E[X])(X-E[X])^T]\)