Can bias be positive or negative?

Yes. Under estimate-minus-target convention, positive bias means overestimation on average and negative bias means underestimation.

Is sample average error the true bias?

It estimates an average discrepancy under suitable conditions; a finite sample does not reveal the exact expectation automatically.

Is model bias the same as a neural-network bias parameter?

No. A bias parameter is an additive model term, while statistical bias concerns estimation error in expectation.

AI Glossary / Governance, ethics and safety / Advanced

Statistical bias

Definition

Statistical bias is a systematic difference between what an estimation or measurement procedure produces on average and the quantity it is intended to estimate. For an estimator T of a parameter theta, bias is E[T] - theta. The expectation refers to repeated samples under specified assumptions. In broader AI discussions, statistical bias can also describe systematic distortions arising from sampling, measurement or modeling choices.

What it means

A single wrong estimate does not establish bias. An unbiased method can produce values above or below the truth in any particular sample. Bias concerns the procedure's average tendency, while variability concerns how much its results fluctuate.
The target must be clear. A survey may accurately describe its respondents while providing a biased estimate of the wider customer population. Likewise, measurements may agree closely with each other but consistently differ from a valid reference value.
Statistical bias is not identical to unfair treatment. A procedure's unbiasedness for one numerical target does not establish fairness, and a deliberately biased statistical estimator may be useful when it substantially reduces variability.

How it works

Bias can arise when the records do not represent the target population, a measurement process has a systematic offset, or an estimation rule has a finite-sample tendency to miss the parameter. Different mechanisms require different investigations.
In the formal estimator setting, the analyst specifies a sampling process and derives the expectation of the rule. For measurement, repeated comparisons with an appropriate reference can provide evidence of systematic offset. In operational AI, evaluation also examines how labels, data collection and modeling assumptions relate to the intended outcome.
More observations can reduce random uncertainty without removing a systematic problem. Averaging many readings from an instrument with a stable offset may produce a very precise estimate of the wrong value. Correcting bias requires understanding the mechanism and checking that the proposed correction applies under the actual conditions.

Three practical examples

Customer research: A survey available only inside an app may underrepresent customers who cannot use the app. Increasing the number of app respondents does not automatically represent those missing customers.
Process measurement: A sensor repeatedly reads above a trusted reference. Calibration may address the offset, but the team also checks whether temperature or operating conditions affect it.
Model estimation: A regularized model deliberately shrinks some estimates. This can introduce bias while reducing variance, so the team compares overall predictive performance rather than assuming unbiasedness is always the only objective.

Think of it this way

Imagine weighing an object on a scale that consistently adds a small amount. Repeating the measurement can make the average stable without making it correct. By contrast, a scale whose readings scatter around the reference may have little average offset but substantial variability. Both properties matter.

Worked example

Consider two fictional estimation procedures for a fixed quantity of 100. Assume their sampling behavior is known.
Procedure A produces 98 or 102 with equal probability. Its expectation is (98 + 102) / 2 = 100, so its bias is zero. Its variance is the average squared deviation from its expectation: (4 + 4) / 2 = 4.
Procedure B produces 100 or 102 with equal probability. Its expectation is 101, so its bias is +1. Its variance is (1 + 1) / 2 = 1.
Mean squared error relative to the true value equals variance plus squared bias. A therefore has MSE 4 + 0 = 4, while B has MSE 1 + 1 = 2. Directly, B's squared errors are 0 and 4, also averaging 2.
B is biased but has lower MSE in this example. That does not make bias harmless in every application: systematic overestimation may have consequences not captured by this loss function. The example assumes the stated probabilities, rather than inferring bias from just two observations.

Illustrative case study

Fictional company Willow Services estimates average customer waiting time from completed online sessions. The report appears stable as more records accumulate. A reviewer notices that abandoned sessions are excluded, even though the intended measure concerns everyone who began requesting help.
The team checks which events are recorded and whether the missing sessions can be measured reliably. It revises the target definition and distinguishes completed-session waiting time from the broader customer experience.
This fictional case illustrates a population mismatch. It does not assume that the excluded sessions all had longer waits, or claim a correction without supporting evidence.

Common misconceptions and mistakes

“Every error is bias.” Random sample variation can produce an error even when the procedure is unbiased.
“A large dataset eliminates bias.” Systematic exclusion or measurement error can persist at large scale.
“Unbiased means accurate in every case.” It is an expectation property, not an individual guarantee.
“Biased estimators are always worse.” Performance also depends on variance, the chosen loss and the consequences of errors.
“Zero average error proves fairness.” Opposite errors may cancel across people or groups, and fairness involves more than a single statistical average.

Business use

Before interpreting a bias claim, identify the target, reference and process that generates the data. Ask whether the evidence concerns formal estimator bias, an observed average error or a broader representation problem. These descriptions should not be used interchangeably.
Evaluate uncertainty and relevant subgroup patterns alongside aggregate measures. If a correction is proposed, check its assumptions and test whether it improves the intended outcome on appropriate data. Keep a record of the target definition so that a convenient measure does not silently replace the business question.

Check your understanding

Why is procedure A unbiased?

Answer: Its expected estimate equals the target of 100.

Why does B have smaller MSE despite bias?

Answer: Its lower variance more than offsets its squared bias in this example.

Why inspect excluded records?

Answer: The observed population may differ from the population the estimate is intended to describe.

Related terms

Learn next

Compare estimator bias with observed prediction errors and representation problems. Then study how sampling design and loss functions affect the interpretation of statistical performance.

Sampling design

Sources

NIST/SEMATECH e-Handbook, Bias and Accuracy, measurement interpretation. https://www.itl.nist.gov/div898/handbook/mpc/section1/mpc113.htm Glossary of Forecasting Terms, biased estimator and unbiasedness. https://robjhyndman.com/mwh3/FG4.pdf NIST AI RMF 1.0, AI Risks and Trustworthiness, broader statistical and social bias context. https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/ The probability example, practical scenarios and Willow Services case are original teaching illustrations.

Previous: Statistical benchmarking Next: Statistical classification