We have a statistic:
\(S(x_1, x_2,...,x_n)\)
We may want to estimate moments for this statistic, but are unable to do so.
The jackknife is an approach for getting moments for statistics.
We start by creating \(n\) statistics each leaving out one observation.
\(\bar S_i(x_1,x_2,...x_{i-1},x_{i+1},...,x_n)\)
We define:
\(\bar S=\dfrac{1}{n}\sum_i\bar S_i\)
We want to know the variance.
\(Var \bar S=\dfrac{n-1}{n}\sum_i(\bar S_i-\bar S)^2\).
In the jackknife we calculate the statistic leaving one observation out.
This is the same as weighting observations and giving one a weighting of \(0\) and the others \(1\).
For the infintesimal jackknife we reduce the weight not to \(0\), but by an infintesimal amount.