AI Glossary / Statistics and probability / Intermediate

Factorial design

Definition

A factorial design is an experimental arrangement that studies two or more factors together by combining their levels. In a full factorial design, every level of each factor is combined with every level of the other factors. This allows researchers to study individual factor effects and interactions, where the effect of one factor depends on another. Fractional factorial designs use selected combinations and require additional interpretation of which effects can be separated.

What it means

Suppose a team wants to compare two interfaces and two training formats. Studying both factors together creates four combinations. The design can reveal whether an interface works differently under the two training formats, something separate one-factor studies may miss.
“Factorial” describes the arrangement of conditions. It does not automatically establish random assignment, independent replication, adequate sample size, or a causal interpretation. Those features must be built into the study.

How it works

Define the factors, their levels, the response, and the experimental unit. If factor A has a levels and factor B has b levels, a full factorial has a × b distinct treatment combinations. With k two-level factors, the count is 2^k before replication.
Decide how units are assigned to combinations and in what order runs occur. Randomization helps separate treatment effects from uncontrolled influences. Blocking can account for known variation such as day or material batch. Hard-to-change factors may require restricted randomization and a split-plot analysis.
Plan independent replication to estimate variability and improve precision. Multiple measurements on one treated unit are not automatically independent replications. An unreplicated design can leave little or no direct error information for a fully specified model.
Estimate main effects and interactions. In a balanced two-level design, a main effect is the average response at the high level minus the average response at the low level, averaging across the other factor levels. An interaction reflects a difference between conditional effects.
Be explicit about coding. With factors coded −1 and +1, the fitted regression coefficient for a main effect is half its high-minus-low contrast. For a two-factor interaction, the conventional factorial interaction effect is half the difference between the two simple effects, and its regression coefficient is half that factorial effect.
Interpret interactions before relying on an overall main effect. Averaging can conceal that a factor helps in one condition and harms in another. Use the response scale, uncertainty, and substantive objective to explain the combinations.
As factors multiply, full factorial designs become expensive. A fractional design can reduce runs but may alias effects. Select the fraction according to which effects must be estimable and which assumptions are plausible. Do not describe omitted combinations as observed, or assume a fractional design estimates every interaction separately.

Three practical examples

• A manufacturer combines two temperatures with three material formulations to compare yield and interactions.
• A service experiment combines two onboarding messages with two reminder schedules.
• An AI evaluation crosses retrieval on/off with two prompt formats, using a predefined dataset and recording whether benefits depend on the combination.

Think of it this way

Think of testing every pairing on a small menu. You learn not only which ingredient tends to help, but whether two ingredients work especially well or poorly together.

Worked example

An invented balanced two-by-two experiment has mean responses 40 for low A/low B, 46 for high A/low B, 44 for low A/high B, and 58 for high A/high B.
A's average effect is (46 + 58)/2 − (40 + 44)/2 = 10. B's average effect is (44 + 58)/2 − (40 + 46)/2 = 8.
A's simple effect is 6 at low B and 14 at high B. Their difference is 8, showing an interaction in these observed means. The conventional factorial interaction effect is 8/2 = 4.
With −1/+1 coding, the model reproducing these cell means is ŷ = 47 + 5A + 4B + 2AB. Substituting all four combinations returns 40, 46, 44, and 58. Replicate-level variability is still required to quantify uncertainty; the arithmetic alone does not establish significance.

Illustrative case study

Fictional case: Maple Software tests two onboarding messages and two reminder schedules. The detailed results show that one message performs well only with the less frequent reminder. Instead of choosing the message from its average effect alone, the team examines the interaction, uncertainty, and user experience for each combination. This invented case illustrates the practical value of studying factors jointly.

Common misconceptions and mistakes

Do not confuse a factorial design with changing one factor at a time. The combinations are what make interaction assessment possible.
Do not confuse the number of treatment combinations with the total number of observations. Replication increases observations without adding new combinations.
Do not describe a main effect as universal when it varies across another factor. In unbalanced data, the averaging and model conventions also need attention; a single raw average may not represent the intended comparison.

Business use

Factorial experiments can answer several related questions within one coordinated study and reveal combinations that deserve attention. They are useful for process improvement, service design, and AI-system configuration. Teams should balance the value of interaction information against the number of units, implementation complexity, and precision needed for decisions.

People also ask

Must every factor have two levels?

No. Full factorial designs can combine different numbers of levels.

Does a full factorial require every combination?

Yes, at the specified levels.

Can an observational dataset have a factorial structure?

Yes, but the arrangement alone does not supply randomized causal evidence.

Check your understanding

How many conditions arise from two factors with three and four levels?

Answer: Twelve before replication.

What is the interaction difference in the example?

Answer: Eight response units between A's two simple effects.

Why is its coded AB coefficient two?

Answer: The conventional interaction effect is four, and −1/+1 coding halves that effect again.

Related terms

• Factor
• Factor level
• Main effect
• Aliasing (factorial experiments)
• Replication (statistics)
• interaction
• full factorial
• fractional factorial
• aliasing
• replication
• split-plot design

Learn next

Learn to calculate cell means and simple effects before interpreting main effects. Then study randomization, replication, and the alias structure of fractional designs.

More in Statistics and probability

• F-test of equality of variances - The F-test of equality of variances is a classical hypothesis test comparing the population variances underlying two independent normal samples.
• Face-to-face survey - A face-to-face survey collects responses through an interviewer and respondent meeting in person.
• Factor range - Factor range is the difference between the highest and lowest values of a quantitative factor used in an experimental design.
• Failure bias - Failure bias is an informal label for drawing general conclusions from visible failures while overlooking relevant successes or other cases that did not fail.
• Failure rate - Failure rate, in lifetime analysis, is the instantaneous rate at which units fail conditional on having survived to a given time.
• False discovery rate - The false discovery rate, or FDR, is the expected proportion of rejected null hypotheses that are actually true.

Sources

David Howell, Statistical Methods for Psychology glossary, “Factorial design”: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html NIST/SEMATECH, full factorial designs: https://www.itl.nist.gov/div898/handbook/pri/section3/pri333.htm NIST/SEMATECH, two-level full factorial designs: https://www.itl.nist.gov/div898/handbook/pri/section3/pri3331.htm

Previous: Factored representation Next: Factorial HMM