AI Glossary / Causal inference and experimentation / Beginner
Z-score
Definition
A Z-score, or standard score, expresses how far a value lies above or below a reference mean in units of the reference standard deviation. It is calculated as z = (x - mean) / standard deviation, provided the standard deviation is positive.
What it means
A Z-score of 2 means that the value is two reference standard deviations above the mean. A score of -1 means it is one below. A score of zero means it equals the reference mean, which need not be the median or the most common value.
The original units cancel, making the result dimensionless. This can help describe relative positions across measurements with different units, but it does not make unlike quantities equally important or directly comparable for every decision.
The reference matters. The same observation can have different Z-scores against different populations, time periods or groups. Always identify which mean and standard deviation were used. In this glossary context, Z-score refers to statistical standardization, not another domain-specific formula bearing the same name.
How it works
Choose the reference observations or a specified population model. Calculate or obtain the reference mean and standard deviation, subtract the mean from the value and divide by that standard deviation.
When estimating spread from a sample, conventions differ. A sample standard deviation commonly uses an n - 1 denominator in its variance calculation, while many preprocessing tools use n. Both produce defined transformations, but their values differ, particularly with small samples. State the convention when it affects reproducibility.
No normal-distribution assumption is needed merely to calculate a Z-score. However, translating it into a normal-table percentile or probability requires additional distributional assumptions. Standardizing a skewed dataset does not make it bell-shaped.
Outliers can influence the mean and standard deviation and therefore influence their own apparent extremeness. A large absolute score warrants contextual investigation, not automatic deletion. If the reference standard deviation is zero, the ordinary formula cannot be used; software handling needs to be checked rather than silently treated as statistical evidence.
Three practical examples
- An analyst describes how an observed delivery duration compares with a defined historical reference for the same service type.
- A machine-learning team standardizes numerical inputs using training-data statistics before fitting a scale-sensitive model.
- A process analyst uses standardized deviations to identify observations for investigation while checking whether the reference conditions remain comparable.
Think of it this way
Think of measuring distance with a ruler whose unit is the typical spread of your reference data. Different original measurement units can be translated onto that ruler.
The ruler does not decide whether a value is good, bad or erroneous. It also changes if the reference changes. The analogy captures relative distance, not the probability or operational significance of an observation.
Worked example
One observation, two reference groups
An invented delivery record shows a duration of 50 minutes. For reference group A, the mean is 40 minutes and the standard deviation is 5 minutes. Its Z-score is (50 - 40) / 5 = 2.
For reference group B, the mean is 45 minutes and the standard deviation is 10 minutes. The same duration has score (50 - 45) / 10 = 0.5.
The observation has not changed; its relative position has. It is farther above the typical level of group A. That does not establish which comparison is appropriate. If the route belongs to group B, selecting group A simply to make the result look unusual would be misleading. Neither score alone supplies a percentile without further information about the distribution.
Illustrative case study
A fictional repair service, Elm Workshops, ranks job durations using a single company-wide Z-score. Complex rebuilds receive high scores and are labeled inefficient, while simple repairs look ordinary.
Review shows that the reference mixes fundamentally different jobs. The team separates descriptive comparisons by relevant job type and reports raw durations alongside standardized positions. It also investigates whether the historical reference contains unusual disruptions that inflate its estimated spread.
The revised analysis avoids treating a standardized number as an explanation of performance. It provides a clearer description of relative duration, while decisions about staffing and process improvement require additional evidence. The fictional case does not claim that within-group standardization removes every source of unfair comparison.
Common misconceptions and mistakes
A Z-score is not a probability or percentage. A score of 2 does not mean 2%, twice as likely, or twice the original value.
Negative scores are not necessarily bad; their meaning depends on the variable and the goal. A lower waiting time can be welcome, whereas a lower production quantity may have another interpretation.
Do not apply normal-distribution tail rules automatically to skewed, dependent or mixed data. Also distinguish a standardized individual observation from a statistical test statistic for a sample mean, whose denominator can involve a standard error rather than the raw observation spread.
Business use
Use Z-scores to communicate relative position and support consistent numerical processing. Keep the reference population, period, units and standard-deviation convention visible in reports.
When using a score for alerts, examine the expected data shape and operational cost of flags. Check unusual observations against their source records and context. When building a prediction pipeline, derive reference statistics from the fitting data and carry them forward consistently so future information does not leak into development assessment.
People also ask
Must the data be normally distributed?
No for calculating a standard score. Yes, or another justified model is needed, for specific normal-based probability interpretations.
Does a Z-score identify an outlier automatically?
It measures standardized distance. An outlier decision also depends on the distribution, sample size, context and assessment method.
Does standardization remove skewness?
No. Subtracting a mean and dividing by a positive constant changes location and scale, not the basic distribution shape.
Check your understanding
A value is 18, the reference mean is 24 and the standard deviation is 3. What is its Z-score, and what does it mean?
Answer: (18 - 24) / 3 = -2. The value is two reference standard deviations below the mean; it is not automatically an error or an undesirable outcome.
Related terms
- Mean
- Standard deviation
- Standardization
- z-score normalization
- Normal distribution
- Percentile
- Outlier
- Standard error
- reference population
Learn next
Calculate standard scores using two different reference groups and explain why they differ. Compare a histogram before and after standardization to see that its shape persists. Then study when a normal-based probability interpretation is justified and when a robust alternative may be useful.
More in Causal inference and experimentation
- Validation error - Validation error is the average loss a trained model incurs on a validation dataset that is separate from the examples used to fit it.
- Validation loss - Validation loss is the value of a specified loss function measured on validation data for a model checkpoint or candidate.
- Validation set - A validation set is a collection of examples used to assess candidate models during development, separately from the examples used to fit each candidate.
- Variable importances - Variable importances are scores that describe how input variables matter to a fitted model according to a particular assessment method.
- Verification and validation - Verification and validation, often abbreviated V&V, are complementary activities for assessing a system.
- Wasserstein loss - Wasserstein loss is an objective based on a Wasserstein distance between probability distributions.
Sources
Sources: SciPy, Z-score calculation and conventions: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.zscore.html Scikit-learn, standard scaling and outlier sensitivity: https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html NIST/SEMATECH, detection of outliers and distributional considerations: https://www.itl.nist.gov/div898/handbook/eda/section3/eda35h.htm Scikit-learn, avoiding preprocessing leakage: https://scikit-learn.org/stable/common_pitfalls.html
Previous: Youden's J statistic Next: z-score normalization