## Maximum Likelihood

The procedure of finding the value of one or more parameters for a given statistic which makes the known Likelihood distribution a Maximum. The maximum likelihood estimate for a parameter is denoted .

For a Bernoulli Distribution,

 (1)

so maximum likelihood occurs for . If is not known ahead of time, the likelihood function is

 (2)

where or 1, and , ..., .
 (3)

 (4)

 (5)

 (6)

For a Gaussian Distribution,

 (7)

 (8)

 (9)

gives
 (10)

 (11)

gives
 (12)

Note that in this case, the maximum likelihood Standard Deviation is the sample Standard Deviation, which is a Biased Estimator for the population Standard Deviation.

For a weighted Gaussian Distribution,

 (13)

 (14)

 (15)

gives
 (16)

The Variance of the Mean is then
 (17)

But
 (18)

so
 (19)

For a Poisson Distribution,

 (20)

 (21)

 (22)

 (23)

Press, W. H.; Flannery, B. P.; Teukolsky, S. A.; and Vetterling, W. T. Least Squares as a Maximum Likelihood Estimator.'' §15.1 in Numerical Recipes in FORTRAN: The Art of Scientific Computing, 2nd ed. Cambridge, England: Cambridge University Press, pp. 651-655, 1992.