Does a Wald test require fitting the restricted model in its usual form?

Often no; it uses the unrestricted estimate and covariance.

AI Glossary / Statistics and probability / Intermediate

Wald test

Definition

A Wald test evaluates a parameter restriction by measuring how far an estimated parameter lies from its null value relative to its estimated uncertainty. For one parameter, the familiar statistic is the estimate minus the hypothesized value, divided by its standard error. Squaring this statistic gives a one-degree-of-freedom Wald chi-square statistic under suitable large-sample conditions. More general Wald tests assess several restrictions together using their covariance matrix.

What it means

An estimated coefficient can differ from zero because of a real association, sampling variation or problems with the model and data. The Wald test asks whether the observed difference is large compared with the uncertainty predicted by the fitted model. It commonly uses the unrestricted model fit, making it convenient for testing selected coefficients. Its convenience does not guarantee reliable inference in small samples, near parameter boundaries or when the estimator's approximate normal distribution is poor.

How it works

Specify a null restriction, obtain parameter estimates and their estimated covariance matrix, and calculate the standardized departure from the restriction. For linear restrictions R theta = r, the quadratic statistic is (R theta-hat minus r) transpose times the inverse of R V-hat R transpose times (R theta-hat minus r). Under regular asymptotic conditions, compare it with a chi-square distribution whose degrees of freedom equal the number of independent restrictions. Some finite-sample linear-model settings use t or F references instead. Robust covariance estimates require their own validity conditions and do not repair an unidentified model.

Three practical examples

Example 1: A regression tests whether one predictor's coefficient is zero. Example 2: A model jointly tests whether several category coefficients vanish. Example 3: An AI evaluation model tests a planned contrast among parameter estimates, using a covariance estimate appropriate to its design. These examples concern model parameters. A statistically detectable coefficient is not automatically a meaningful effect, a causal effect or an improvement in out-of-sample prediction.

Think of it this way

Think of comparing a measured displacement with the uncertainty of the measuring system. A shift of one unit means something different when the measurement uncertainty is ten units than when it is one-tenth of a unit. The Wald statistic scales the departure by uncertainty, but its interpretation depends on the measuring model being appropriate.

Worked example

Suppose a regular large-sample model estimates a coefficient as 0.60 with standard error 0.20. Test the null that the coefficient is zero. The Wald z statistic is 0.60/0.20 = 3.0, and the squared Wald statistic is nine. With a standard normal reference, the two-sided p-value is approximately 0.00270; equivalently, compare nine with a chi-square distribution having one degree of freedom. A corresponding normal 95% Wald interval is 0.60 plus or minus 1.96 times 0.20, or 0.208 to 0.992. These calculations assume the estimated standard error and asymptotic approximation are appropriate. They do not reveal the coefficient's practical importance without its units and model interpretation, and the p-value is not the probability that the null is true.

Illustrative case study

In this fictional case study, Ash Analytics tests whether a usage feature is associated with task completion in a fitted model. Analysts report the coefficient, units, uncertainty interval and Wald result together. They check whether sparse outcomes and separation make the usual approximation unreliable, and compare an alternative inferential method when needed. Managers receive an explanation of the estimated association rather than a bare significance label, with causal claims limited by the study design.

Common misconceptions and mistakes

Common mistakes include treating the Wald test as exact in every model, using an inappropriate standard error, and assuming equivalent parameterizations always give identical finite-sample Wald results. Nonlinear transformations can change the approximation. Another mistake is testing many coefficients without acknowledging multiplicity. Boundary parameters, weak identification and separation can produce unreliable Wald inference. A large sample size does not automatically resolve every misspecification or dependence problem.

Business use

Businesses can use Wald tests for prespecified parameter questions within a well-supported model. Review effect magnitude and uncertainty alongside the p-value, and ensure the covariance estimate reflects clustering, heteroscedasticity or other relevant design features when justified. For difficult settings, consider likelihood-ratio, score, profile-likelihood or resampling approaches appropriate to the problem. Do not select whichever method happens to be significant. The decision should depend on the substantive effect and the credibility of the analysis, not a threshold alone.

Check your understanding

What is the z statistic in the example?

Answer: Three.

What is its squared Wald statistic?

Answer: Nine.

What is the approximate 95% interval?

Answer: 0.208 to 0.992.

What must be checked before trusting the p-value?

Answer: The model, standard-error estimate, identification and reference-distribution assumptions.

Related terms

Confidence interval
Score test
Related terms include standard error
likelihood-ratio test
covariance matrix and asymptotic normality

Learn next

Learn next how joint restrictions require covariance information rather than separate coefficient tests. Compare Wald, likelihood-ratio and score tests in a simple model, then examine a boundary or sparse-data example where their standard approximations fail. This helps distinguish computational convenience from inferential reliability.

Sources

Source: UCLA Statistical Consulting, How are the likelihood ratio, Wald, and Lagrange multiplier tests different and/or similar?, https://stats.oarc.ucla.edu/other/mult-pkg/faq/general/faqhow-are-the-likelihood-ratio-wald-and-lagrange-multiplier-score-tests-different-andor-similar/. This primary instructional reference explains the test comparison. The coefficient, p-value and interval calculations are original and independently checked; the usage-feature case is fictional.

Previous: Wald's maximin model Next: Wald - Wolfowitz runs test