Is every orthogonal array saturated?

No; some leave capacity beyond the selected main effects.

Can mixed-level arrays be saturated?

Yes; count s - 1 for each factor.

Does saturation make a design useless?

No. It defines what its economical screening results can support.

AI Glossary / Statistics and probability / Advanced

Saturated array

Definition

A saturated array, in experimental design, is an orthogonal array whose factor main effects use all the available degrees of freedom after allowing for the overall mean. For N runs and factors with level counts s1, s2 and so on, saturation occurs when the sum of (sj - 1) equals N - 1. It describes the array's capacity for a specified main-effects model, not the absence of experimental error.

What it means

An orthogonal array balances combinations of factor levels across selected runs. A saturated array uses that economical arrangement to accommodate as many independent main-effect contrasts as its run count permits.

The economy comes with a limitation: if all columns represent factors and the intercept and all main effects are fitted, there are no residual degrees of freedom left. A perfect fit to the observed runs therefore does not establish that the model predicts well, that interactions are absent or that the experiment is noiseless.

How it works

Rows specify experimental settings and columns specify factors. In a strength-two orthogonal array, every pair of columns contains each possible pair of levels equally often. A factor with s levels contributes s - 1 independent main-effect contrasts.

Count those contrasts and add one for the intercept. When this equals the number of runs, the main-effects model is saturated. For two-level factors, each contributes one contrast, so N - 1 factors fill an N-run array.

The balance among main-effect columns does not prevent an interaction from sharing information with those columns. Before interpreting a fitted coefficient, examine which effects the chosen design can actually distinguish.

Three practical examples

1. Workflow screening: a team tests several binary settings with a small run budget. A saturated design can support an initial main-effects screen, provided the team states the assumptions needed to interpret it.

2. AI pipeline experiments: three settings, such as retrieval mode, prompt format and reranking, are arranged in four runs. The team checks whether combinations of settings could produce interactions before treating apparent individual effects as separate causes.

3. Equipment studies: an analyst combines factors with different numbers of levels. Counting columns alone is insufficient; a three-level factor uses two main-effect degrees of freedom.

Think of it this way

Think of allocating every seat in a room. Full allocation does not mean every issue has been covered. Saturation similarly leaves no spare estimation capacity.

Worked example

Consider three fictional two-level factors A, B and C, coded -1 and +1. Use four runs with settings (-1,-1,+1), (-1,+1,-1), (+1,-1,-1) and (+1,+1,+1).

Each factor has two low settings and two high settings. For any pair of factors, the four possible level combinations occur once each. The array is therefore pairwise balanced. Its three main-effect contrasts use 3 × (2 - 1) = 3 degrees of freedom, exactly N - 1 = 4 - 1.

Suppose the four responses, in that order, are 9, 7, 5 and 19. Fit y = b0 + bA A + bB B + bC C. The intercept is the response average: (9 + 7 + 5 + 19)/4 = 10.

Using the orthogonal coding, bA = (-9 - 7 + 5 + 19)/4 = 2. Similarly, bB = (-9 + 7 - 5 + 19)/4 = 3 and bC = (9 - 7 - 5 + 19)/4 = 4.

The fitted model is y = 10 + 2A + 3B + 4C. Substituting the four settings reproduces 9, 7, 5 and 19 exactly. The high-minus-low response contrasts are twice the coefficients: four for A, six for B and eight for C.

Four fitted parameters from four independent rows leave zero residual degrees of freedom. The residual sum of squares is zero, but dividing by zero degrees of freedom does not provide a usable residual variance estimate.

There is another limitation. In every chosen run, C equals A multiplied by B. Consequently, y = 10 + 2A + 3B + 4AB gives exactly the same four responses. These observations cannot distinguish C from AB.

At A = +1, B = +1, C = -1, the first model predicts 11, whereas the interaction model predicts 19. That setting was not among the four runs. The disagreement shows that agreement on observed runs does not determine responses elsewhere.

Illustrative case study

Fictional Alder Evaluation uses the worked array to screen three pipeline settings. Its first report calls C the strongest independently established influence because its fitted coefficient is largest.

The reviewer checks the design and finds C = AB. Alder changes the report to describe the observed contrast and its ambiguity. It plans additional settings that separate C from AB before choosing between the two explanations.

The team also distinguishes adding new settings from repeating existing ones. Independent replication can provide information about repeatability, but repeating only these same four settings preserves their C = AB relationship. This invented case demonstrates design interpretation, not a measured improvement in an AI system.

Common misconceptions and mistakes

"Saturated means every possible combination was tested." The example tests four of the eight possible combinations.

"Orthogonality removes all confounding." Pairwise balance among main-effect columns does not eliminate their relationship with interactions.

"Zero residuals prove zero noise." A saturated fit can absorb variation because no residual degrees of freedom remain.

"Replication automatically separates aliased effects." Repeating the same settings retains the same alias relation.

Business use

Use the intended model to evaluate a proposed experiment before paying for the runs. Record which factors and interactions matter to the decision, what the design can separate and how experimental variability will be assessed.

In an AI experiment, preserve the settings, responses and run identifiers. Randomize execution where appropriate to reduce systematic order effects. Do not let an automated report interpret an exact fit as independent validation. Reserve follow-up work for plausible interactions and for checking conclusions at additional settings.

Check your understanding

Why is the example saturated?

Answer: Three main effects and an intercept use four parameters in four runs.

Why is C confused with AB?

Answer: Their values agree on every selected row.

What do the two models predict at the untested setting?

Answer: Eleven and nineteen, respectively.

Related terms

Learn next

Add settings where C differs from AB and compare the two models' predictions. Separately consider replication for estimating repeatability.

Sources

Raghu N. Kacker, Eric S. Lagergren and James J. Filliben, Taguchi's Orthogonal Arrays Are Classical Designs of Experiments, particularly Sections 2 and 8: https://pmc.ncbi.nlm.nih.gov/articles/PMC4927234/

NIST/SEMATECH, Plackett-Burman designs, saturated main-effects designs and interaction limitations: https://www.itl.nist.gov/div898/handbook/pri/section3/pri335.htm

Previous: Satterthwaite solution Next: Saturated model