\(Z=\sum_{i=1}^nY_i\)
\(\phi_Z(t)=E[e^{itZ}]\)
\(\phi_Z(t)=E[e^{it\sum_{i=1}^nY_i}]\)
\(\phi_Z(t)=E[e^{itY}]^n\)
\(\phi_Z(t)=\phi_Y(t)^n\)
\(Z=\sum_{i=1}^nY_i\)
\(Y=\dfrac{X}{n}\)
\(\phi_Z(t)=\phi_Y(t)^n\)
\(\phi_Z(t)=\phi_{\dfrac{X}{n}}(t)^n\)
\(\phi_Z(t)=\phi_X(\dfrac{t}{n})^n\)
\(\phi_X(t)=1+it\mu_X -\dfrac{(\mu_X +\sigma_X^2 )t^2}{2} +\sum_{j=3}^{\infty }\dfrac{E[X^j](it)^j}{j!}\)
\(\phi_X(\dfrac{t}{n})=1+i\dfrac{t\mu_X }{n}-\dfrac{(\mu_X +\sigma_X^2 )(\dfrac{t}{n})^2}{2} +\sum_{j=3}^{\infty }\dfrac{E[X^j](i\dfrac{t}{n})^j}{j!}\)
\(\phi_X(\dfrac{t}{n})=1+i\dfrac{t\mu_X }{n}-\dfrac{(\mu_X +\sigma_X^2 )t^2}{2n^2} +\sum_{j=3}^{\infty }\dfrac{E[X^j](i\dfrac{t}{n})^j}{j!}\)
We want \(\mu\) to be \(0\).
\(Z=\sum_{i=1}^nY_i\)
\(Y=\dfrac{X-\mu_X }{n}\)
\(\phi_Y(t)=1+it\mu_Y -\dfrac{(\mu_Y +\sigma_Y^2 )t^2}{2} +\sum_{j=3}^{\infty }\dfrac{E[Y^j](it)^j}{j!}\)
\(\mu_Y =E[\dfrac{X-\mu_X }{n}] ={\mu_X -\mu_X }{n}=0\)
\(\phi_Y(t)=1-\dfrac{\sigma_Y^2t^2}{2} +\sum_{j=3}^{\infty }\dfrac{E[Y^j](it)^j}{j!}\)
\(\sigma^2_Y =E[(\dfrac{X-\mu_X }{n})^2]\)
\(\sigma^2_Y =E[\dfrac{X^2+\mu^2_X-2X\mu_X }{n^2}]\)
\(\sigma^2_Y =\dfrac{E[X^2]+E[\mu^2_X]-E[2X\mu_X] }{n^2}]\) \(\sigma^2_Y =\dfrac{E[X^2]-\mu^2_X}{n^2}]\)
\(\sigma^2_Y =\dfrac{\sigma^2_X}{n^2}\)
\(\phi_Y(t)=1-\dfrac{\sigma_X^2t^2}{2n^2} +\sum_{j=3}^{\infty }\dfrac{E[(\dfrac{X-\mu}{n})^j](it)^j}{j!}\)
\(\phi_Z(t)=\phi_Y(t)^n\)
\(\phi_Z(t)=[1-\dfrac{\sigma_X^2t^2}{2n^2} +\sum_{j=3}^{\infty }\dfrac{E[(\dfrac{X-\mu}{n})^j](it)^j}{j!}]^n\)
\(\phi_Z(t)=[1-\dfrac{\sigma_X^2t^2}{2n^2}]^n\)
Eliminating \(\sigma^2\)
\(Z=\sum_{i=1}^nY_i\)
\(Y=\dfrac{X-\mu_X }{\sigma n}\)
\(\phi_Y(t)=1+it\mu_Y -\dfrac{(\mu_Y +\sigma_Y^2 )t^2}{2} +\sum_{j=3}^{\infty }\dfrac{E[Y^j](it)^j}{j!}\)
\(\mu_Y =E[\dfrac{X-\mu_X }{\sigma_X n}] ={\mu_X -\mu_X }{\sigma_X n}=0\)
\(\phi_Y(t)=1-\dfrac{\sigma_Y^2t^2}{2} +\sum_{j=3}^{\infty }\dfrac{E[Y^j](it)^j}{j!}\)
\(\sigma^2_Y =E[(\dfrac{X-\mu_X }{\sigma n})^2]\)
\(\sigma^2_Y =E[\dfrac{X^2+\mu^2_X-2X\mu_X }{\sigma^2 n^2}]\)
\(\sigma^2_Y =\dfrac{E[X^2]+\mu^2_X-2E[X]\mu_X }{\sigma^2 n^2}\)
\(\sigma^2_Y =\dfrac{E[X^2]-\mu^2_X}{\sigma^2 n^2}\)
\(\sigma^2_Y =\dfrac{\sigma^2_X}{\sigma^2 n^2}\)
\(\sigma^2_Y =\dfrac{1}{n^2}\)
\(\phi_Y(t)=1-\dfrac{t^2}{2n^2} +\sum_{j=3}^{\infty }\dfrac{E[(\dfrac{X-\mu}{\sigma n})^j](it)^j}{j!}\)
\(\phi_Z(t)=\phi_Y(t)^n\)
\(\phi_Z(t)=[1-\dfrac{t^2}{2n^2} +\sum_{j=3}^{\infty }\dfrac{E[(\dfrac{X-\mu}{\sigma n})^j](it)^j}{j!}]^n\)
\(\phi_Z(t)=[1-\dfrac{t^2}{2n^2}]^n\)
We know that
\([1+\dfrac{x}{n}]^n=e^x\)
As \(n \rightarrow \infty\).
With:
\(Z=\sum_{i=1}^nY_i\)
\(Y=\dfrac{X-\mu_X }{\sigma n}\)
We have:
\(\phi_Z(t)=[1-\dfrac{t^2}{2n^2}]^n\)
With:
\(Z=\sum_{i=1}^nY_i\)
\(Y=\dfrac{X-\mu_X }{\sigma \sqrt n}\)
We have:
\(\phi_Z(t)=[1-\dfrac{t^2}{2n}]^n\)
Which tends towards
\(\phi_Z(t)=e^{-\dfrac{1}{2}t^2}\)
The average of random variables, less their mean, and divided by their standard deviation multiplied by the square root of the sample size, follows a normal distribution as \(n\) increases.
What does this say about the actual distribution of sample averages?
\(Z=\sum_{i=1}^nY_i\)
\(Y_i=\dfrac{X_i-\mu_X }{\sigma_X \sqrt n}\)
\(\sum_{i=1}^nY_i\)
\(Y=\dfrac{X}{n}\)
Let’s create \(Q\).
\(Q=\dfrac{Z\sigma_X }{\sqrt n}+\mu_X\)
\(Q=\dfrac{(\sum_{i=1}^nY_i)\sigma_X }{\sqrt n}+\mu_X\)
\(Q=\dfrac{(\sum_{i=1}^n(\dfrac{X_i-\mu_X }{\sigma_X \sqrt n}))\sigma_X }{\sqrt n}+\mu_X\)
\(Q=\sum_{i=1}^n(\dfrac{X_i-\mu_X }{n})+\mu_X\)
\(Q=\sum_{i=1}^n(\dfrac{X_i-\mu_X }{n}+\dfrac{\mu_X}{n})\)
\(Q=\sum_{i=1}^n(\dfrac{X_i}{n})\)
This is the sample average.
\(\phi_Q(t)=\phi_{\dfrac{Z\sigma_X }{\sqrt n}+\mu_X}(t)\)
\(\phi_Q(t)=\phi_Z(\dfrac{t\sigma_X }{\sqrt n})e^{it\mu_X}\)
\(\phi_Z(\dfrac{t\sigma_X }{\sqrt n})=e^{-\dfrac{1}{2}(\dfrac{t\sigma_X }{\sqrt n})^2}\)
\(\phi_Z(\dfrac{t\sigma_X }{\sqrt n})=e^{-\dfrac{1}{2}\dfrac{t^2\sigma^2_X }{n}}\)
\(\phi_Q(t)=e^{-\dfrac{1}{2}\dfrac{t^2\sigma^2_X }{n}}e^{it\mu_X}\)
We name the normal distribution this function when \(n=1\)
\(N(\mu_X, \sigma^2_X)=e^{-\dfrac{1}{2}\dfrac{t^2\sigma^2_X }{n}}e^{it\mu_X}\)
\(N(\mu_X, \sigma^2_X)=e^{-\dfrac{1}{2}t^2\sigma^2_X }e^{it\mu_X}\)
\(\phi_X(t)=e^{-\dfrac{1}{2}t^2\sigma^2_X} e^{it\mu_X}\)
\(\phi_X(t)=e^{-\dfrac{1}{2}t^2\sigma^2_X}[\cos (t\mu_X )+i\sin (t\mu_X)]\)
Converges in probability
\(P(distance(X_n, X)>\epsilon )\rightarrow 0\)
For all \(\epsilon\).
\(X_n \rightarrow^P X\)
Little o notation is used to describe convergence in probability.
\(X_n=o_p(a_n)\)
mean that
\(\dfrac{X_n}{a_n}\)
Converges to \(0\) and \(n\) approaches something
Can be wrtiten:
\(\dfrac{X_n}{a_n}=o_p(1)\)
Big O notation is used to describe boundedness.
\(X_n=O_p(a_n)\)
means that:
If something is little o, it is big O.
\(X_n\) converges almost surely to \(X\) if:
\(d(X_n, X)\rightarrow 0\)
Where \(d(X_n, X)\) is a distance metric.
\(X_n\rightarrow^{as} X\)
\(f_x=\dfrac{1}{\sqrt {2\pi \sigma^2 }} e^{-\dfrac{(x-\mu)^2}{2\sigma }}\)
For univariate:
\(x \sim N(\mu, \sigma^2 )\)
We define the multivariate gaussian distribution as the distribution where any linear combination of components are gaussian.
For multivariate:
\(X \sim N(\mu, \Sigma )\)
Where \(\mu\) is now a vector, and \(\Sigma\) is the covariance matrix.
Density function is :
\(f_x=\dfrac{1}{\sqrt {(2\pi )^n|\Sigma |}} e^{-\dfrac{1}{2}(x-\mu )^T\Sigma^{-1}(x-\mu)}\)
For normal gaussian it is:
\(f_x=\dfrac{1}{\sqrt {2\pi |\sigma^2}} e^{-\dfrac{1}{2\sigma^2}(x-\mu )^2)}\)
This is the same wher \(n=1\).
Need det \(|\Sigma |\) and \(\Sigma^{-1}\). These rely on the covariance matrix not being degenerate.
If the covariance matrix is degenerate we can instead use the pseudo inverse, and the pseudo determinant.