Is an orthogonal array always a full factorial?

No; many cover only lower-order combinations in a subset of full settings.

Is every covering array orthogonal?

No. Coverage can be complete at the chosen strength without equal frequencies.

Does a small balanced array prove safety?

No. It provides a defined testing structure, not a universal assurance claim.

AI Glossary / Statistics and probability / Intermediate

Orthogonal array

Definition

An orthogonal array is a structured table of experimental or test settings with a specified balance property. For any selected set of t columns, every possible combination of their levels occurs the same number of times. The number t is the array's strength. Rows represent runs and columns represent factors.

What it means

An orthogonal array spreads selected combinations evenly through a compact set of runs. A strength-two array balances every pair of factors, even if the full combinations of all factors are not exhausted.
This makes the table useful for planned experiments and configuration testing. Its balance is a mathematical property of the settings, not evidence that observed outcomes are independent or that every possible failure has been checked. The meaning of a factor and its levels must still be defined carefully.

How it works

For an array with N rows, k columns, and v levels per column, strength t means each t-column projection contains all v^t level combinations equally often. If each occurs lambda times, then N = lambda times v^t. This equal-level notation does not cover every mixed-level design without modification.
Select factors and levels relevant to the decision. Then choose or construct an array with sufficient strength and enough columns. Map its symbols to actual settings and verify the balance after that mapping.
An experimental analysis also needs a model. Pairwise balance can make appropriately coded main-effect columns orthogonal, but it does not ensure that every interaction is separately estimable. Some effects can be aliased with others in a reduced design.
Randomize run order where appropriate and consider blocking, replication, and measurement reliability. Balance alone does not remove time trends or supply an independent estimate of experimental error.
For software testing, translate each row into an executable test and specify how correctness will be judged. A table of inputs without an outcome check is an incomplete testing procedure.
A covering array has a weaker requirement: each required combination appears at least the specified number of times, rather than exactly equally often. This flexibility can be useful when equal repetition is unnecessary. Constraints that exclude impossible settings require careful redesign and coverage verification; simply deleting inconvenient rows can destroy the promised property.

Three practical examples

Product experimentation: an engineer distributes combinations of three component settings evenly before estimating selected effects under a stated model.
AI service testing: a team checks pairs of configuration settings, such as input format and deployment option, across a controlled suite.
Process development: a laboratory arranges treatment settings to reduce imbalance while retaining additional runs for important interactions and repeatability checks.

Think of it this way

Think of a schedule designed so that every pair of activity choices meets equally often. The schedule guarantees fair pairwise representation. It does not promise that every possible combination of three or more activities has occurred.

Worked example

Construct an original nine-run illustration with factors A, B, and C, each having levels 0, 1, and 2. List every pair of A and B, and set C equal to the remainder when A + B is divided by 3.
The runs are (0,0,0), (0,1,1), (0,2,2), (1,0,1), (1,1,2), (1,2,0), (2,0,2), (2,1,0), and (2,2,1).
For A and B, all nine possible pairs appear once by construction. For fixed A, cycling through B also cycles through every C level; therefore every A,C pair appears once. The same reasoning applies to B,C. Direct enumeration confirms all three pair projections contain nine distinct pairs, each with frequency one.
Thus N = 9, k = 3, v = 3, t = 2, and lambda = 1. The full three-factor factorial would contain 3^3 = 27 settings, so these runs represent one-third of them.
The triple (0,0,1) is absent. A failure that occurs only at that exact triple would be missed, despite perfect pairwise coverage. This is the distinction between strength-two balance and exhaustive coverage.

Illustrative case study

Illustrative fictional case: Juniper Analytics wants to compare settings for a document-processing prototype. Its engineers initially choose a compact orthogonal array and record processing accuracy and runtime for each row.
Before interpreting the main effects, they identify a plausible interaction between document format and preprocessing. They add targeted runs and replicated checks instead of assuming that the array can estimate every effect. The team records the original balance, the additions, and the revised analysis model. This fictional example shows design discipline rather than a measured commercial benefit.

Common misconceptions and mistakes

Do not equate orthogonal with exhaustive. Do not assume every orthogonal array has strength two or that every combination occurs only once; higher strengths and larger lambda values exist.
Do not interpret pairwise balance as proof that factors never interact. Avoid assigning invalid settings to a design and then silently dropping rows. Also distinguish an array's combinatorial balance from statistical independence of noisy outcomes.

Business use

An orthogonal array can help a business organize limited testing or experimental resources around an explicit coverage target. The practical value comes from matching the design to plausible failure mechanisms or estimable effects. Record what is covered, what is aliased or omitted, and which additional checks address consequential gaps.

Check your understanding

How often does each pair occur in the worked array?

Answer: Exactly once in each two-column projection.

Why can it miss (0,0,1)?

Answer: Strength two constrains pairs, while that condition specifies all three factors simultaneously.

Related terms

Factorial design
Fractional factorial design
Aliasing (factorial experiments)
Replication (statistics)
Covering array
experimental factor
interaction
aliasing
replication
combinatorial testing

Learn next

Next compare orthogonal arrays with covering arrays, then study alias structures in factorial experiments. Practice checking coverage independently after introducing real-world constraints or changing factor levels.

Sources

NIST/SEMATECH, Engineering Statistics Handbook glossary: https://www.itl.nist.gov/div898/handbook/glossary.htm — terminology context, describing the pairwise case. Kuhn, Kacker, and Lei, Practical Combinatorial Testing, NIST SP 800-142, Appendix A: https://nvlpubs.nist.gov/nistpubs/legacy/sp/nistspecialpublication800-142.pdf — general strength and equal-frequency definition and covering-array distinction. The nine-run construction and fictional case are original.

Previous: Original time series Next: Orthogonal array testing