Does the estimator itself converge to a non-degenerate normal distribution?

In the usual consistent case, the unscaled estimator concentrates at the true parameter; the centered, scaled error has the non-degenerate limit.

Is there a fixed sample-size threshold?

No. Accuracy depends on the model, parameter, and quantity being approximated.

Is this a Bayesian posterior statement?

No. It concerns repeated-sampling behavior, although related large-sample Bayesian results exist under their own assumptions.

AI Glossary / Probabilistic machine learning / Advanced

Asymptotic normality of the maximum likelihood estimator

Definition

Asymptotic normality of the maximum likelihood estimator is a large-sample result stating that, under suitable conditions, the estimator's centered and appropriately scaled sampling distribution approaches a normal distribution. In a regular, correctly specified model with independent, identically distributed observations and a fixed-dimensional parameter, the usual scaling is the square root of the sample size and the limiting covariance is the inverse Fisher information per observation.

What it means

A maximum likelihood estimate is a number computed from one dataset. Its sampling distribution describes how that number would vary across repeated datasets generated under the same conditions. Asymptotic normality concerns this repeated-sampling distribution, not the shape of the observed data.
The observations can be binary, counts, or other non-normal values while the estimator has an approximately normal sampling distribution at sufficiently large sample sizes. The theorem is a limiting statement, however. It does not specify one universally adequate sample size, and it does not make the approximation exact for a particular finite dataset.

How it works

For a scalar parameter theta_0, a standard result is sqrt(n)(theta_hat − theta_0) converging in distribution to N(0, 1/I(theta_0)), where I is Fisher information from one observation. Informally, theta_hat is then approximately normal with mean theta_0 and variance 1/[n I(theta_0)]. Vector parameters use the inverse information matrix.
A typical derivation expands the likelihood's score around the true parameter and combines a central limit theorem for the score with convergence of the likelihood curvature. Conditions include appropriate smoothness, identifiability, consistency, an interior parameter, and nonsingular information, with additional technical requirements depending on the theorem. This list is explanatory rather than a complete universal checklist.
In practice, unknown information is estimated, and its inverse supplies approximate standard errors.

Three practical examples

An analyst estimates a binary response probability from independent observations and uses a large-sample standard error to describe uncertainty in the fitted probability.
A researcher estimates a Poisson event rate. The raw counts are discrete, yet their sample mean can have an approximately normal sampling distribution when the relevant conditions and sample size support it.
A team fits a regular parametric regression model and obtains an estimated covariance matrix for its coefficients. It checks dependence, parameter boundaries, and numerical fitting issues before interpreting the resulting intervals.

Think of it this way

Imagine repeatedly weighing batches with the same measurement procedure and plotting the resulting estimates. The bell-shaped pattern concerns variation in the estimates across batches, not the shapes or weights of the individual objects. A theorem about that pattern cannot repair a biased scale or turn repeated measurements of one object into independent evidence.

Worked example

Consider an original hypothetical dataset containing 240 successes in 400 independent Bernoulli trials with a common probability p. The maximum likelihood estimate is p_hat = 240/400 = 0.6. For an interior Bernoulli probability, information per observation is I(p) = 1/[p(1−p)].
Substituting the estimate gives I(p_hat) = 1/0.24, approximately 4.1667. Total estimated information is 400/0.24, approximately 1666.6667. Its reciprocal is 0.0006, and the estimated standard error is sqrt(0.0006), approximately 0.024495.
A conventional approximate 95 percent Wald interval is 0.6 plus or minus 1.96 × 0.024495. Its endpoints are approximately 0.5520 and 0.6480. This is a large-sample construction, not an exact binomial confidence interval or a claim that this particular interval has a 95 percent probability of containing a fixed parameter.
If a separate study had 1600 trials and the same estimated proportion, the plug-in standard error would be sqrt(0.24/1600), approximately 0.012247. Quadrupling sample size halves this standard error under the same assumptions. It does not follow that collecting four times as many correlated observations would do the same. Near zero or one, or with small samples, the Wald approximation can perform poorly.

Illustrative case study

Fictional service-analysis company EstimateWorks treats every interaction in its records as an independent trial. Its report presents very narrow normal-approximation intervals. A reviewer notices that several interactions belong to the same customer and can reflect persistent customer characteristics.
The team revisits the sampling assumptions and uses an analysis appropriate to the dependence structure. It also checks whether the fitted model represents the process that generated the records. This fictional case illustrates why a large row count alone does not justify a standard error formula; it reports no observed company outcome or verified improvement.

Common misconceptions and mistakes

Asymptotic normality does not say that the data are normal, that the estimate is exactly unbiased, or that every finite-sample confidence interval has its nominal coverage. Convergence in distribution also does not by itself establish convergence of moments without further conditions.
Another mistake is using inverse Fisher information indiscriminately when the model is misspecified. Under suitable misspecification theory, a different limiting covariance, often a sandwich form, may be needed. Parameters on boundaries, non-identifiability, changing parameter dimension, and parameter-dependent support can also require different asymptotic results. Optimization software returning a number does not prove the required likelihood maximum or regularity conditions exist.

Business use

The result underpins many familiar uncertainty summaries in statistical software. Use it to understand where approximate standard errors come from and which assumptions support them. Document the observational unit, dependence structure, model specification, and fitting diagnostics. Check important conclusions with methods suited to the setting when the normal approximation is doubtful. Precision from a fitted model should not be confused with protection against systematic measurement or selection problems.

Check your understanding

How is per-observation information converted into an approximate estimator variance?

Answer: Per-observation information is multiplied by n before inversion to obtain the approximate estimator variance.

What was approximately normal in the example?

Answer: The sampling distribution of the estimated proportion, not each binary observation.

Why investigate customer clustering?

Answer: Dependence changes the information supplied by the recorded trials.

Related terms

Consistency
Central limit theorem
Standard error
Maximum likelihood estimation
Fisher information
score function
Wald interval
sandwich covariance

Learn next

Derive the score and Fisher information for a simple Bernoulli or Poisson model. Then compare asymptotic intervals with alternatives suited to finite samples and study examples where standard regularity conditions fail.

Sources

Fan, Stanford STATS 200, Lecture 14: Consistency and Asymptotic Normality of the MLE, 2016: https://web.stanford.edu/class/stats200/Lecture14.pdf

Previous: Asymptotic bounded optimality Next: Asymptotically unbiased estimator