Mode estimate
\(Arg max_\theta p(\theta | X)\)
Using Bayes theorem:
\(P(\theta | X)= \dfrac{P(X|\theta )P(\theta)}{P(X)}\)
So:
\(P(\theta | X)= \dfrac{P(X|\theta )P(\theta)}{P(X)}\)
\(Argmax_\theta p(\theta | X)=Argmax_\theta \dfrac{p(X|\theta )P(\theta)}{P(X)}\)
The denominator isn’t affected so:
\(Arg max_\theta p(\theta | X)=Arg max_\theta p(X|\theta )P(\theta)\)
If \(P(\theta )\) is a constant then this is the same as the MLE estimator.
\(Argmax_\theta p(\theta|X)\)
Mode estimate
\(p(\theta|X)= \dfrac{p(X| \theta)p(\theta )}{p(X)}\)
\(Argmax_\theta \dfrac{p(X| \theta)p(\theta )}{p(X)}\)
\(\theta\) doesn’t change denominator so can instead use:
\(Argmax_\theta p(X| \theta)p(\theta )\)
It is the same as maximum likelihood estimator if \(p(\theta )\) is a constant.