Does the boundary allow unlimited unplanned peeking?

No. The monitoring scheme and any flexibility must be covered by a valid design.

Is alpha spending the same as a p-value threshold?

No. It describes cumulative error allocation; boundaries are derived using sequential probabilities.

Is an O'Brien - Fleming-type spending design the original design?

Not necessarily. Report the exact boundary or spending function used.

AI Glossary / Statistics and probability / Advanced

O'Brien - Fleming boundary

Definition

An O'Brien - Fleming boundary is a group-sequential testing boundary that requires especially strong evidence for early rejection of a null hypothesis and becomes less stringent as information accumulates. It allows planned interim analyses while controlling the overall Type I error under the design's assumptions. The original boundary family and later O'Brien - Fleming-type alpha-spending approaches share this conservative-early behavior, but they are not identical constructions and should be identified precisely.

What it means

Repeatedly inspecting an ordinary fixed-sample p-value and stopping whenever it passes a familiar threshold can inflate the probability of a false positive. A group-sequential design addresses the sequence of possible analyses together.

O'Brien - Fleming boundaries make early stopping difficult unless the evidence is unusually strong, while preserving a final boundary relatively close to the corresponding fixed-design threshold. This is useful when an early decision should require compelling evidence. It is not a license to add arbitrary analyses or change outcomes after observing results.

How it works

Specify the hypothesis, error rate, test direction, information schedule, and number of planned looks. For a canonical standardized Z-statistic, the original O'Brien - Fleming shape has boundaries proportional to one divided by the square root of the information fraction. Calibrate the constant using the joint distribution of the correlated sequential statistics.

Information fraction measures accumulated statistical information relative to the planned maximum. It need not equal the fraction of calendar time elapsed, and in event-driven studies it can depend on observed events rather than enrollment alone.

A Lan - DeMets spending approach instead specifies how cumulative error expenditure grows with information and derives boundaries compatible with the analysis times. Its O'Brien - Fleming approximation is related to, rather than identical with, the original fixed boundary. Futility rules, binding assumptions, multiplicity across outcomes, and post-stopping estimation also need explicit treatment.

Three practical examples

A planned product experiment permits early success only when interim evidence crosses a conservative efficacy boundary.
A trial team uses an O'Brien - Fleming-type spending function when the precise information available at scheduled reviews can vary.
An AI benchmark study schedules several cumulative evaluations and accounts for their dependence instead of applying an unchanged fixed-sample cutoff repeatedly.

Think of it this way

Think of a committee that demands exceptionally persuasive evidence to decide early but allows a more ordinary evidence threshold after the full investigation. Its rules must account for every planned opportunity to decide. Simply relaxing the standard over time without calibrating the entire process would not provide the same error control.

Worked example

Consider the Lan - DeMets O'Brien - Fleming approximation with total error parameter alpha = 0.05 and information fraction t. Its cumulative spending function is a(t) = 2[1 − Phi(1.959964/sqrt(t))], where Phi is the standard normal cumulative distribution function.

At t = 0.25, cumulative spending is approximately 0.0000886. At t = 0.50, it is approximately 0.0055746. At t = 1, it equals 0.05. The amount added between the first two fractions is approximately 0.0054860, illustrating how little error is spent very early.

These are cumulative error-spending values, not ordinary p-value cutoffs for each look. Actual sequential Z boundaries must account for earlier stopping and correlations among analyses. The example checks the spending calculation only; it is not a complete trial design, and it does not substitute this approximation for the original boundary calibration.

Illustrative case study

Fictional case study: Cedar Experiments compares two AI-assisted search interfaces. The team wants a possibility of stopping early for a substantial improvement without repeatedly testing each day's results at the usual cutoff.

A statistician specifies an information-based review schedule, an O'Brien - Fleming-type efficacy rule, and a separate policy for operational problems. The dashboard distinguishes cumulative spending from nominal thresholds and records each planned analysis. The team also decides in advance how estimates and intervals will be reported after early stopping. This fictional case illustrates experiment governance rather than a claim that a particular boundary guarantees a good business decision.

Common misconceptions and mistakes

A common mistake is calling every conservative interim rule an O'Brien - Fleming boundary. Another is substituting cumulative alpha spending directly for a nominal p-value threshold. Correlation and previous stopping opportunities matter.

The final boundary is not necessarily exactly the ordinary fixed-sample boundary, and a chosen boundary shape does not determine power or maximum sample size by itself. Information fraction is not automatically elapsed time. Finally, rejecting early does not remove estimation uncertainty, and an unadjusted effect estimate after stopping may exaggerate the underlying effect. Statistical monitoring cannot compensate for biased outcomes or a broken randomization process.

Business use

Businesses running costly or lengthy experiments can use this design family when they value a credible early-success option while requiring strong early evidence. It can support disciplined monitoring of AI features, service processes, or other interventions with suitable experimental data.

Implementation should use validated sequential-design calculations and a documented analysis plan. Store the actual information at each look, the applicable boundaries, the stopping history, and any deviations. Evaluate practical effect size and other consequences alongside the formal rejection decision.

Check your understanding

If cumulative spending is 0.0056 at an interim look, can the analyst automatically reject whenever that look's ordinary p-value is below 0.0056?

Answer: No. The correct nominal boundary depends on the sequential design, including earlier looks and the joint distribution of the statistics.

Related terms

Type I error
Pocock boundary
Stopping rule
Group sequential design
interim analysis
alpha spending
information fraction
statistical power
sequential inference

Learn next

Study the joint distribution of sequential test statistics, then compare original boundary families with spending-function designs. Learn how stopping affects estimation and confidence intervals, and validate both operating characteristics and implementation before using a design in production.

Sources

The gsDesign author documentation distinguishes boundary families and design inputs: https://keaven.github.io/gsDesign/reference/gsDesign.html

Its Lan–DeMets documentation supplies the O'Brien–Fleming approximation used in the original calculation: https://keaven.github.io/gsDesign/reference/sfLDOF.html

The worked example is a spending-function calculation, not a calibrated complete design.

Previous: NYSIIS phonetic code Next: OAuth