AI Glossary / Statistics and probability / Advanced

Commonality analysis

Definition

Commonality analysis decomposes the variance accounted for by a regression model into components associated uniquely with individual predictors and components shared by specified sets of predictors. In the usual multiple linear regression setting, it partitions the model's R squared using results from predictor subsets. It helps explain overlapping predictive information, but it does not divide causal responsibility among variables or provide a universal ranking of importance.

What it means

Correlated predictors may account for some of the same outcome variation. Looking only at their individual correlations or standardized coefficients can obscure that overlap. Commonality analysis describes how the full-model fit relates to the fits of smaller models. A unique component measures the increment when a predictor is added after all the others. Shared components describe the algebraic overlap among subsets. They depend on the variables included, the sample and the model specification, so changing the predictor set changes the decomposition.

How it works

Fit the required subset regressions on the same observations using compatible specifications, ordinarily with an intercept and the same outcome. For two predictors A and B, let R_A squared and R_B squared be their separate fits and R_AB squared the joint fit. The unique A component is R_AB squared minus R_B squared; unique B is R_AB squared minus R_A squared. The shared component is R_A squared plus R_B squared minus R_AB squared. These sum to the joint R squared. With p predictors, a full decomposition contains up to 2-to-the-p minus one components, so complexity grows quickly. Quantify sampling uncertainty where interpretation depends on small differences.

Three practical examples

• Research interpretation: distinguish unique and overlapping associations of several related measures with an outcome.
• Feature review: examine whether two business indicators provide largely redundant or complementary linear predictive information.
• Measurement development: compare how related scales account for outcome variation without assuming that standardized coefficients tell the whole story.

Think of it this way

Think of several spotlights illuminating a surface. Some areas are lit by one spotlight and others by several. The analogy helps describe overlap, but it has a limit: regression decompositions can contain negative shared components under suppression, unlike literal areas of light. Those algebraic components need statistical interpretation.

Worked example

Suppose hypothetical regressions on the same observations give R squared values of 0.36 for A alone, 0.25 for B alone and 0.49 for both. Unique A is 0.49 - 0.25 = 0.24, unique B is 0.49 - 0.36 = 0.13, and the shared component is 0.36 + 0.25 - 0.49 = 0.12. Their sum is 0.49. Thus 24, 13 and 12 percentage points of total outcome variance form this decomposition; they are not percentages of the explained portion unless explicitly rescaled. The shared component represents 0.12/0.49, about 24.49%, of the model's accounted-for variance. It should not be assigned wholly to both predictors and then added, because that would count the same component twice.

Illustrative case study

Fictional case: Poplar Research models task quality using two related measures of employee experience. Both correlate with quality, but their regression coefficients change when entered together. Analysts use commonality analysis to describe their unique increments and overlapping explanatory information, then examine uncertainty and measurement quality. They avoid concluding that a small unique component makes a measure useless, since it may represent a valued construct also captured by the other measure. This invented case illustrates interpretation of correlated predictors, not a causal explanation of employee performance.

Common misconceptions and mistakes

Shared components are not necessarily positive. For example, separate R squared values of 0.04 and 0.09 with a joint value of 0.20 yield a shared component of -0.07, reflecting suppression rather than negative physical variance. Unique increments in ordinary nested least-squares fits on the same data are nonnegative, but that does not make every commonality component nonnegative. Mixing different samples or adjusted R squared values into the basic formulas changes their meaning. The analysis also does not remove confounding or establish what would happen if a predictor were intervened on.

Business use

Businesses can use commonality analysis to make overlapping information visible when interpreting a manageable regression model. It is most useful when the predictor set has a substantive rationale and the question concerns explanation of model fit. For large AI feature sets, the number of subsets and components can become prohibitive and the results difficult to communicate. Consider whether grouped predictors or a different interpretive method better fits the question. Validate predictive performance separately; an elegant decomposition of training fit does not demonstrate reliable deployment performance.

People also ask

Is commonality analysis the same as common source bias analysis?

No. Commonality partitions regression fit; common source bias concerns distortion from shared measurement sources.

Does a small unique component mean a predictor has no relationship with the outcome?

No. Its information may overlap substantially with other predictors.

Can a shared component be negative?

Yes. Suppression can produce negative commonality coefficients, which should be interpreted algebraically rather than as negative amounts of physical variance.

Check your understanding

Why would adding 0.36 and 0.25 overstate the joint fit in the worked example?

Answer: Each single-predictor value includes the shared component of 0.12. Adding them counts that overlap twice. Subtracting it once gives the joint R squared of 0.49.

Related terms

• Multiple regression
• Coefficient of determination
• Multicollinearity
• shared variance
• unique contribution
• suppression
• subset regression
• dominance analysis
• relative importance
• semipartial correlation

Learn next

derive the two-predictor decomposition from nested R squared values, then examine a suppression example. Compare commonality components with standardized coefficients and structure coefficients. Study bootstrap uncertainty and the dependence on predictor specification before presenting a decomposition as a stable account of the data.

• R-squared

More in Statistics and probability

• Combinatorics - Combinatorics is the branch of mathematics concerned with discrete structures, including how to count, arrange and select objects and how to establish that...
• Common cause - A common cause is a source of variation that forms part of the usual operation of a process.
• Common source bias - Common source bias is distortion in a measured relationship that arises because the variables are obtained from the same information source.
• Compact letter display - A compact letter display is a concise way to encode the outcomes of a set of pairwise statistical comparisons using letters beside group estimates.
• Compact model - A compact model is a small machine-learning model designed to operate on devices with limited computational resources, such as phones, tablets or embedded...
• Comparative trial - A comparative trial is a study that compares two or more treatment or intervention groups to evaluate differences in outcomes.

Sources

Nimon, Lewis, Kane and Haynes, An R package to compute commonality coefficients in the multiple regression case: https://doi.org/10.3758/BRM.40.2.457 Nimon, Oswald and Roberts, yhat package documentation: https://search.r-project.org/CRAN/refmans/yhat/html/yhat-package.html R yhat reference manual, commonalityCoefficients: https://stat.ethz.ch/CRAN/web/packages/yhat/refman/yhat.html

Previous: Common source bias Next: Commonsense knowledge (artificial intelligence)