Can there be two critical values?

Yes, for example in a two-sided equal-tail z-test.

Does a critical value always increase with sample size?

No. Its behavior depends on the statistic and reference distribution; a standard-normal cutoff stays fixed for a fixed alpha.

Can software calculate it?

Yes, but the user must still select the appropriate distribution, parameters, and decision rule.

AI Glossary / Statistics and probability / Intermediate

Critical value

Definition

A critical value is a boundary used in a statistical decision rule, commonly separating rejection and nonrejection regions for a hypothesis test. It is selected from the test statistic's reference distribution according to the significance level and alternative hypothesis. Different tests can require upper, lower, or multiple critical values.

What it means

A test statistic measures how the observed data compare with a null hypothesis. The critical value states how extreme that statistic must be before a prespecified rule rejects the null.

For a standard two-sided normal test at a 5% significance level, the familiar boundaries are approximately -1.96 and 1.96. These numbers belong to that particular reference distribution and rule. They are not universal cutoffs for every statistical analysis.

A critical value is also different from a practically important effect size. Crossing a test boundary says something about compatibility with a null model under the test assumptions; it does not automatically show that the observed difference is large enough to matter operationally.

How it works

Specify the null hypothesis, alternative, test statistic, reference distribution, and significance level alpha. For an upper-tail continuous test, select c so that the null probability of a statistic at least as large as c equals alpha. A lower-tail rule uses the corresponding lower quantile.

For a symmetric equal-tail two-sided rule, allocate alpha/2 to each tail. Other two-sided tests can use different rejection-region constructions, so follow the actual test definition.

Calculate the statistic and apply the chosen rule. For discrete distributions, attainable rejection probabilities may not equal the desired alpha exactly; a nonrandomized rule often has size no greater than alpha. Composite null hypotheses and estimated nuisance parameters can introduce further details. Critical values may come from tables, analytic calculations, simulation, or resampling under a justified model.

Three practical examples

An AI evaluation study compares a prespecified performance difference with a test boundary chosen before inspecting the results.
A manufacturing analyst uses an appropriate t critical value when estimating a mean with an unknown population standard deviation under the relevant assumptions.
A researcher uses simulated null statistics to determine a boundary for a test whose reference distribution is not conveniently available in a simple table.

Think of it this way

Think of a marked line on a measuring instrument. The line defines a decision under a particular procedure. It does not change the measurement itself, and using the same line on an instrument with a different scale would give a misleading result.

Worked example

Consider an original two-sided z-test illustration. Assume independent normally distributed measurements with known population standard deviation 10, sample size 25, and null mean 100. The standard error of the sample mean is 10/sqrt(25) = 2.

Choose alpha = 0.05 before observing the sample mean. The standard-normal boundaries are approximately ±1.96. On the original measurement scale, the rejection boundaries for the sample mean are 100 ± 1.96 × 2, or approximately 96.08 and 103.92.

Suppose the observed mean is 104.4. The z statistic is (104.4 - 100)/2 = 2.2, exceeding the upper boundary. The test rejects the null under the stated rule. Its two-sided p-value is approximately 0.0278.

The observed difference is 4.4 units. Whether that matters in practice depends on the application, costs, and uncertainty, not merely the rejection decision.

If the population standard deviation were unknown and replaced by a sample estimate, this exact known-variance procedure would no longer apply unchanged. A suitable t-test under normal assumptions would use a different reference distribution and degrees of freedom.

Illustrative case study

Fictional case: Harbor Search tests whether a revised retrieval system changes a task-quality score. Before evaluation, its team records the primary comparison and analysis rule.

After seeing a near-boundary result, a manager proposes switching from a two-sided test to a one-sided test because the result points in the desired direction. The analyst explains that this would change the decision rule after inspecting the evidence.

The report retains the planned analysis and presents the estimated effect and interval. It also lists exploratory findings separately. Harbor makes its product decision using the size and consequences of the observed change, while preserving an honest account of how the statistical conclusion was reached.

Common misconceptions and mistakes

A critical value is not the same as the observed test statistic or the p-value.
A 5% significance level is not the probability that the null hypothesis is true after observing the data.
Failure to cross a boundary does not prove the null or establish equivalence.
The value 1.96 is not appropriate for every 95% procedure.
Choosing the direction, significance level, or stopping time after seeing results can invalidate the advertised error rate.
Testing many hypotheses can require an adjusted procedure; applying a separate unadjusted boundary to each does not generally control the overall false-positive probability.

Business use

Critical values help make evaluation rules explicit and reproducible. Teams should document the planned statistic, reference distribution, direction, error level, and handling of multiple comparisons or interim looks.

Check that the data and sampling design justify the reference distribution. Repeated observations from the same users, selection effects, or unmodeled dependence can make a familiar boundary inappropriate. A larger sample does not automatically repair every design problem.

Report effect estimates and uncertainty alongside the decision. Avoid presenting nearly identical results on opposite sides of a threshold as radically different evidence. Statistical boundaries can support disciplined analysis without replacing judgment about implementation costs, user outcomes, and practical relevance.

Check your understanding

Answer: 1.

What are the example's standardized boundaries?

Answer: Approximately -1.96 and 1.96. 2.

Why are its raw-scale boundaries different numbers?

Answer: The statistic is translated using the null mean and standard error. 3.

Does rejection prove a useful business improvement?

Answer: No; magnitude, direction, uncertainty, and consequences need separate interpretation.

Related terms

Significance level
Rejection region
Quantile
P-value
Hypothesis test
statistical power
multiple testing

Learn next

Learn sampling distributions and quantiles, then compare one-sided and two-sided tests. Practice translating standardized boundaries into the original measurement scale. Study confidence intervals, power, and prespecified analysis plans to interpret decisions beyond a binary threshold.

Sources

Frank Harrell, Glossary of Statistical Terms, Critical value: https://hbiostat.org/glossary/

NIST/SEMATECH, Standard normal distribution table and critical values: https://www.itl.nist.gov/div898/handbook/eda/section3/eda3671.htm

The primary glossary and official table were read. The z-test calculation and fictional retrieval evaluation are original teaching examples with explicit assumptions.

Previous: Critical path method Next: Cromwell's rule