Must a sufficient statistic be one number?

No. It can be multidimensional.

Is the full dataset sufficient?

Yes, trivially, although it provides no reduction.

Does sufficiency prove the model is correct?

No. It is a property within the assumed model, which still requires evaluation.

AI Glossary / Statistics and probability / Advanced

Sufficient statistic

Definition

A sufficient statistic is a function of observed data that preserves all the information those data contain about a specified parameter within a specified statistical model. Formally, after conditioning on the statistic, the remaining distribution of the data does not depend on that parameter. Sufficiency is therefore relative to assumptions and an inference target. It does not mean the summary preserves every detail needed for every future question, audit, or model check.

What it means

Statistical summaries discard details. The important question is whether the discarded details contain additional information about the parameter under the assumed model. If they do not, the statistic is sufficient for that inference problem.

For independent Bernoulli observations sharing one success probability, the total number of successes is sufficient when the sample size is known. The order of successes and failures adds no information about that common probability under the model. However, order can matter for checking whether independence or a constant probability is plausible. This illustrates why sufficiency is a precise mathematical property rather than a blanket permission to discard raw data.

How it works

Specify the model family and unknown parameter before evaluating a proposed summary. A statistic must be calculated from the data without using the unknown parameter itself. It may be a number, vector, or another function of the observations.

One definition examines the conditional distribution of the full data given the statistic. If that conditional distribution is parameter-free, the statistic is sufficient. For common dominated models, the factorization theorem provides a practical route: write the data density or probability as a product of a term depending on the parameter and statistic, and a term that does not depend on the parameter.

Use the statistic for parameter inference while retaining the assumptions that justify the reduction. A sufficient statistic need not be minimal; the entire dataset is trivially sufficient, and a larger-than-necessary summary may also qualify. Minimal sufficiency asks for a reduction that removes redundant information in a specific formal sense.

Do not confuse sufficiency with unbiasedness, consistency, or efficiency. Those describe other properties of estimators. A sufficient statistic need not itself be an estimate of the parameter, and sufficiency alone does not prescribe an estimator or guarantee a useful small-sample result.

Three practical examples

1. A binary-event model uses the success count and known trial count for inference about a shared probability, while retaining sequence information for diagnostic checks.

2. A distributed analysis combines appropriate local summaries when their combined form is sufficient under the agreed model.

3. An analyst checks whether a proposed data-compression scheme preserves the likelihood information needed for a specific parameter rather than assuming all summaries are equivalent.

Think of it this way

Think of packing exactly the information needed for one specified journey. The package can be complete for that journey while omitting items needed for another destination. Sufficiency certifies completeness for a model-and-parameter task, not universal completeness for every possible use of the data.

Worked example

Consider an original example with four independent Bernoulli trials sharing an unknown probability p, with 0 < p < 1. Observe the sequence 1,0,1,0, which has total T = 2. Its probability is p squared times (1 − p) squared.

There are C(4,2) = 6 ordered sequences containing exactly two successes. Each has that same probability. The probability that T = 2 is therefore 6p squared times (1 − p) squared. Conditional on T = 2, the observed sequence has probability 1 / 6, independent of p. The same argument applies to each possible total, establishing sufficiency of T for this model.

For a numerical check, at p = 0.2 the sequence probability is 0.0256 and the total-count probability is 0.1536; their ratio is 1 / 6. At p = 0.7 the corresponding probabilities are 0.0441 and 0.2646, again giving 1 / 6.

The sequence 1,1,0,0 has the same count and likelihood for p, but a different run pattern. If success probabilities change over time or observations depend on preceding outcomes, the common independent-Bernoulli model no longer applies and the count alone may not be sufficient for the new parameters.

Illustrative case study

In a fictional company, Willow Metrics aggregates binary quality checks into success totals for a simple probability model. The team initially plans to delete the original ordering because the total is sufficient for the model's parameter. A reviewer points out that temporal clustering of failures is a separate operational concern and a useful model diagnostic. The company keeps the sequence under its retention policy while using the count for the specified inference. Mathematical reduction and broader evidence requirements are documented as separate decisions.

Common misconceptions and mistakes

Sufficient does not mean sufficient for every question. The property depends on the model and parameter, including what is assumed known.

A sample mean is not universally sufficient. Its role depends on the distribution family and whether other parameters, such as variance, are unknown.

Sufficiency does not imply that the statistic is unbiased, minimal, private, or free of sampling error. A summary can retain all model-based information about a parameter while still providing a highly uncertain estimate. It can also expose sensitive information. Claims about privacy, storage, or model validity require their own analysis rather than following automatically from sufficiency.

Business use

Sufficient statistics can simplify estimation, reduce repeated computation, and support aggregation when a well-defined model makes the reduction valid. They help explain why some analytical workflows can operate on counts or other summaries without using every record for each calculation.

Businesses should state the model, target parameter, sample-size information, and intended use before claiming a summary is sufficient. Consider what raw details are needed for auditing, subgroup evaluation, drift detection, or later questions. In AI systems, a compact representation learned for one objective should not be called statistically sufficient without a precise claim and supporting argument.

Check your understanding

For a fixed number of independent Bernoulli trials with one common unknown success probability, do two sequences with the same success count have different likelihoods for that probability?

Answer: No. Under that model, they have the same likelihood; their ordering can still matter for checking assumptions.

Related terms

Statistic
Likelihood
Parameter
Exponential family
Model checking
factorization theorem
minimal sufficiency
Bernoulli distribution

Learn next

Study the factorization theorem, then compare sufficiency for models with one unknown parameter and models with several unknown parameters.

Parameter

Sources

Will Fithian, Sufficiency, author lecture notes: https://www.stat.berkeley.edu/~wfithian/courses/stat210a/sufficiency.html . The notes provide the conditional-distribution definition and factorization theorem. The four-trial numerical checks and fictional retention discussion are original illustrations; the model-relative limitation is explicit throughout.

Previous: Sufficient dimension reduction Next: Sugeno integral