Is a q-value the same as a p-value?

No. It incorporates a multiple-testing false-discovery perspective.

Does q = 0.05 mean this finding is 95% likely to be true?

No. That is not the general interpretation.

Are all FDR-adjusted p-values adaptive Storey q-values?

No. The procedure and pi0 handling must be identified.

AI Glossary / Statistics and probability / Intermediate

Q-value (statistics)

Definition

A q-value is a measure of statistical significance designed for multiple hypothesis testing. It associates a finding with the smallest false-discovery-rate level attainable by a suitable rejection region that includes that finding. Storey's original formulation uses the positive false discovery rate, while practical q-value estimation commonly uses closely related false-discovery-rate calculations. A q-value concerns the reliability of a collection of selected findings; it is not simply the probability that one particular finding is false.

What it means

When many hypotheses are tested, an apparently small p-value can occur among numerous opportunities for false positives. A q-value places a result within that collection and its selection thresholds.

The false discovery rate is the expected fraction of false rejections among all rejections, with the fraction defined as zero when none are rejected. Positive FDR conditions on making at least one rejection. These quantities differ from the chance of making any false rejection and from a local FDR, which has an individual posterior-probability interpretation under an appropriate model. State which definition and estimation procedure are being used.

How it works

Define the family of hypotheses and calculate valid p-values using methods suited to the data and null hypotheses. Order the p-values from smallest to largest. At possible thresholds, estimate the fraction of false discoveries among the results that would be selected.

In a common plug-in calculation, the estimated number of false positives at p-value threshold t is approximately pi0 × m × t, where m is the number of tests and pi0 estimates the proportion of true null hypotheses. Divide by the number selected, then take suitable cumulative minima across thresholds so that the estimated q-values respect the ordering.

The estimate of pi0, treatment of dependence, finite-sample corrections, and calibration of the input p-values affect interpretation. Setting pi0 to one in the standard calculation produces the Benjamini - Hochberg adjusted values. That special case should not be confused with an adaptive estimate that learns pi0 from the data. Report the method and settings, and avoid treating every number labeled q as interchangeable.

Three practical examples

A discovery study prioritizes features while considering the expected proportion of false positives in the selected set.
An AI research team evaluates many candidate associations and documents its multiple-testing procedure.
An analyst distinguishes a raw p-value from a false-discovery-based significance summary.

Think of it this way

Imagine selecting a shortlist from many possible leads. A q-value describes the error-rate level associated with including a lead in such a shortlist, under the method's assumptions. It does not stamp that individual lead with a known probability of being wrong. The shortlist analogy also explains why the collection being searched matters: evaluating one candidate differs from searching through thousands and displaying only the most favorable results.

Worked example

In an original teaching calculation, five ordered p-values are 0.005, 0.012, 0.030, 0.200, and 0.600. Use the standard calculation with pi0 fixed to one and no positive-FDR finite-sample correction. For rank i, calculate 5p_i/i.

The resulting values are 0.025, 0.030, 0.050, 0.250, and 0.600. Taking the minimum from each rank through the final rank leaves these values unchanged because they already increase. They are the Benjamini - Hochberg adjusted values, also obtained from the q-value calculation under these explicit settings.

At a threshold of 0.05, the first three findings are selected. This is a statement about the selection procedure and its intended FDR control under suitable conditions. It does not imply that exactly 5% of these three findings are false, nor that each has a 5% posterior probability of being false. The example deliberately fixes pi0 rather than trying to estimate a true-null proportion reliably from only five tests. An adaptive q-value analysis would require its own assumptions and settings.

Illustrative case study

Fictional case study: Juniper Research screens many candidate indicators of customer churn. Its report includes effect estimates, raw p-values, and q-values calculated across the declared family of tests.

The team chooses a discovery threshold before interpreting the shortlist and reserves later data for follow-up. It explains that the shortlist may contain false positives and that statistical significance does not establish useful prediction or causality. This fictional case illustrates reporting discipline, not a real study or a guarantee about which indicators will replicate.

Common misconceptions and mistakes

A q-value is not an effect size, a measure of business importance, or a substitute for valid study design. It cannot repair biased p-values or an analysis that ignores how the data were selected.

Another mistake is calling a q-value the probability that its own null hypothesis is true. That interpretation belongs to a different quantity, local FDR, under a specified model. Selecting tests after seeing results and omitting the rest can also distort the relevant testing family. Finally, estimated FDR control is not a deterministic promise about the realized fraction of false discoveries in one dataset.

Business use

Q-values help organizations balance discovery against false leads when many statistical tests are considered. They can make a large exploratory analysis easier to communicate without pretending that every selected result is certain.

Keep the complete testing family, analysis settings, and software provenance. Report effect magnitudes and uncertainty alongside significance measures. Examine whether dependence, discreteness, or invalid null calibration challenges the method. Follow-up evaluation remains useful because a statistically detectable association may be unstable, practically trivial, or irrelevant to the intended AI application.

Check your understanding

Three findings pass a q-value threshold. Must a fixed known fraction of those three be false?

Answer: No. FDR describes an expected error proportion under a procedure and assumptions, not a known realized count for this shortlist.

Related terms

False discovery rate
P-value
Null hypothesis
Statistical significance
Effect size
positive FDR
local FDR
multiple testing
Benjamini–Hochberg procedure

Learn next

Learn the differences among false-positive rate, familywise error, and FDR. Then compare fixed and adaptive procedures, including the assumptions behind estimating the true-null proportion.

False positive rate

Sources

Storey and Bass's official qvalue vignette explains the quantities, interpretation, and diagnostic checks: https://www.bioconductor.org/packages/release/bioc/vignettes/qvalue/inst/doc/qvalue.pdf

The authors' implementation documents the pi0 = 1 special case and calculation options: https://raw.githubusercontent.com/StoreyLab/qvalue/master/R/qvalue.R

The five-test calculation and fictional case are original. The calculation uses explicitly stated settings rather than an estimated adaptive pi0.

Previous: Q-systems Next: Quadratic classifier