Is design of experiments only for factories?

No. The principles also apply to software, services, agriculture, and many other settings.

Does replication mean measuring the same object repeatedly?

Not necessarily. Independent treatment applications are needed for experimental replication.

Can a factorial design detect interactions?

Yes, when the design and available data identify them; fractional designs may alias selected effects.

AI Glossary / Statistics and probability / Intermediate

Statistical design of experiments

Definition

Statistical design of experiments is the planned arrangement of experimental conditions, units, measurements, and analysis so that a study can answer its questions efficiently and credibly. It includes principles such as randomization, replication, and blocking, as well as designs such as factorial experiments and response-surface studies. The design determines which effects can be estimated and how uncertainty can be assessed. It is part of planning the evidence, not a statistical repair applied after an uncontrolled study has finished.

What it means

An experiment deliberately changes one or more factors and observes responses. Statistical design connects those changes to the comparisons needed for a decision. It helps prevent treatment effects from being confused with unrelated differences between units, times, or conditions.

Efficiency does not simply mean using the fewest runs. A small study that cannot separate important effects may waste its entire budget. A good design balances information, precision, operational constraints, and the consequences of a wrong conclusion. It also anticipates how interactions, measurement error, and unexpected missing observations will affect interpretation.

How it works

Define the objective, response variables, experimental units, factors, and factor levels. Specify the effects or contrasts that matter and the smallest differences worth detecting. Choose a design that makes those comparisons identifiable under plausible assumptions.

Randomize assignment or run order where feasible to reduce systematic confounding with uncontrolled influences. Use genuine replication to estimate variation between independently treated units. Block on known nuisance factors when comparing treatments within more homogeneous groups can improve precision. Repeated measurements of one treated unit are not automatically independent experimental replicates.

Factorial designs vary factors together and can reveal interactions. Fractional factorial designs reduce runs by accepting specified aliasing among effects, so their assumptions must be understood. Response-surface designs support local modeling and optimization. Plan analysis, diagnostics, stopping rules, and handling of deviations before examining outcomes. Afterward, verify promising findings in relevant operating conditions rather than assuming one experiment settles every future use.

Three practical examples

1. A manufacturing team studies temperature and mixing speed together to learn whether their combined effect differs from their separate effects.

2. A website team randomizes eligible users to alternative interfaces, defines the assignment unit, and plans metrics before launch.

3. A laboratory blocks experimental runs by day to compare formulations while accounting for day-to-day conditions.

Think of it this way

Think of arranging a fair set of races. The lanes, starting rules, repeated races, and changing weather all affect what can be learned about the runners. Statistical design sets up comparisons so that the result has an interpretable connection to the question.

Worked example

Consider an original two-factor experiment with factors A and B, each at low and high settings. Suppose the four cell means from a properly randomized, independently replicated study are: A low/B low = 10; A high/B low = 14; A low/B high = 12; and A high/B high = 22. These means are stipulated for illustrating contrasts; no sampling uncertainty is supplied.

The average response at A high is (14 + 22)/2 = 18, while at A low it is (10 + 12)/2 = 11. The marginal high-minus-low A effect is therefore 7. For B, the corresponding effect is (12 + 22)/2 - (10 + 14)/2 = 17 - 12 = 5.

But A's effect depends on B: at B low it is 14 - 10 = 4, while at B high it is 22 - 12 = 10. Their difference is 6, showing an interaction contrast. In the common two-level factorial convention, the AB effect is half that difference, 3; the coefficient of x_A x_B under -1/+1 coding is 1.5. Naming the convention avoids confusing these related numbers.

A one-factor-at-a-time study conducted only at B low would see A's effect as 4 and miss its larger effect at B high. Replicate-level data are still needed to assess uncertainty and distinguish a real interaction from experimental variation.

Illustrative case study

In a fictional materials company, engineers plan to optimize curing temperature and pressure. They initially propose changing temperature first and pressure afterward, using different material batches for each stage.

The team instead plans a factorial comparison with batch blocking and feasible randomized run order. It records actual conditions and measurement procedures, then examines both main effects and interaction. A promising combination is tested again in confirmation runs. The design makes the findings easier to interpret without assuming that the highest observed response is automatically the best or most reliable operating setting.

Common misconceptions and mistakes

Random assignment does not eliminate every source of bias or compensate for poor measurement. Random sampling and random assignment serve different purposes. Likewise, many measurements from one experimental unit do not replace replication at the treatment-assignment level.

Do not assume that every interaction is negligible or that a fractional design estimates all effects separately. Confounded effects cannot be untangled by more elaborate analysis alone without additional assumptions or data. Changing outcomes, exclusions, or stopping decisions after looking at results can alter inference. Optimization also requires attention to constraints, costs, and robustness, not simply maximizing a fitted response within an unvalidated model.

Business use

Designed experiments help businesses improve products, services, and processes with clearer evidence about changes. They can reduce expensive trial-and-error work by extracting several useful comparisons from a coordinated study. Involve operational experts and analysts during planning, preserve assignment and run records, and document departures. Evaluate whether the experimental conditions represent the intended deployment setting before generalizing results.

Check your understanding

A treatment is applied to two machines, and each machine produces 100 measurements. How many independently treated machines are present?

Answer: Two. The 200 measurements may provide within-machine information, but they do not create 200 independent treatment assignments.

Related terms

Replication (statistics)
Blocking
Factorial design
Confounding
Experimental unit
randomization
replication
interaction
response surface
confirmation experiment

Learn next

Learn randomization, replication, and blocking, then work through a two-factor factorial design. Study aliasing before using fractional designs and uncertainty before selecting an apparent optimum.

Sources

NIST/SEMATECH, glossary, Statistical design of experiments: https://www.itl.nist.gov/div898/handbook/glossary.htm

NIST/SEMATECH, Choosing an experimental design: https://www.itl.nist.gov/div898/handbook/pri/section3/pri3.htm

The primary handbook terminology and design guidance were consulted. The factorial means, contrast calculations, and fictional case are original teaching illustrations.

Previous: Statistical Data Protection (SDP) Next: Statistical disclosure