Is a factor directly observed?

Usually not; it is inferred through a model of observed measurements.

Does a loading of 0.8 mean 80% of variance is explained?

Not in the standardized one-factor example: the shared variance contribution is the squared loading, 0.64, under its stated assumptions.

Does a good-fitting model prove the theory?

No. Fit is one piece of evidence, and alternative models or explanations may remain possible.

AI Glossary / Data science and analytics / Intermediate

Factor analysis

Definition

Factor analysis is a family of statistical methods that models relationships among observed variables using a smaller set of unobserved factors and variable-specific variation. It is often used to investigate whether several measurements reflect shared underlying dimensions. Factor loadings describe how observed variables relate to the factors under the model. A factor is a modeled construct, not automatically a discovered physical cause or a validated psychological trait.

What it means

Several survey responses may move together because they reflect a common aspect of experience, but they may also share wording, response habits, or other influences. Factor analysis offers a structured way to examine shared covariance. It distinguishes that shared part from variation specific to each measurement. Naming a factor, such as service confidence, requires substantive reasoning and supporting evidence; the software supplies a statistical pattern rather than a guaranteed interpretation.

How it works

Specify suitable observed variables and a factor model, estimate loadings and unique variances, and evaluate how well the implied relationships match the data. Exploratory factor analysis investigates possible loading patterns, whereas confirmatory approaches assess a specified measurement structure. The number of factors, estimator, and treatment of ordinal or continuous measurements matter. Rotations can make an exploratory loading pattern easier to interpret, with some rotations allowing correlated factors. Check stability and validity for the intended use rather than selecting a solution only because its labels sound appealing.

Three practical examples

Survey development: researchers examine whether items intended to measure several service dimensions exhibit a compatible shared structure.
Measurement research: a team studies correlated test scores while distinguishing common variation from measurement-specific variation.
Multivariate exploration: analysts investigate a low-dimensional covariance model for related indicators, then compare it with alternative explanations and models.

Think of it this way

Think of hearing several microphones record overlapping sounds and asking whether a few shared sound sources could explain some of their co-movement. Each microphone also has its own noise. The analogy conveys shared and specific variation. A statistical factor model does not necessarily identify literal sources, and multiple representations or explanations may fit similar observed relationships.

Worked example

Consider a toy one-factor model for two standardized measurements, X1 and X2. Let the factor F have variance 1. Suppose X1 = 0.8F + e1 and X2 = 0.6F + e2, with errors uncorrelated with F and with each other. Give e1 variance 0.36 and e2 variance 0.64. Then X1 has variance 0.8 squared + 0.36 = 1, and X2 has variance 0.6 squared + 0.64 = 1.
Their model-implied covariance is 0.8 times 0.6 = 0.48. Because both observed variances are 1, their implied correlation is also 0.48. The shared-factor contribution to X1's variance is 0.64, while its unique contribution is 0.36. This arithmetic illustrates the model; two observed variables alone are not presented as a sufficient dataset for uniquely estimating an unrestricted one-factor measurement model. The assumed error relationships are essential: allowing correlated errors would change the explanation of the observed covariance.

Illustrative case study

In a fictional customer-experience study, a team expects survey items about clarity and speed to represent separate dimensions. Exploratory results suggest that several negatively worded items cluster together. Rather than immediately naming a new business construct, the researchers consider whether wording or response style contributes to the pattern. They revise and pilot the instrument, then test the intended structure on additional data. The case illustrates interpretation and validation of a measurement model, not proof that a statistical grouping reveals a new customer trait.

Common misconceptions and mistakes

Factor analysis is not interchangeable with principal component analysis. PCA constructs components from observed variation, while a common factor model explicitly represents shared factors and unique variation. High loadings do not prove causation, and an attractive rotation does not validate a construct. Factor scores are estimates under a chosen model, not direct measurements of an invisible quantity. Adding or removing variables can change the solution. Poorly chosen items, inappropriate assumptions, and selective reporting can produce an apparently clean but misleading structure.

Business use

Businesses may use factor analysis when a measurement problem involves several related indicators and a defensible latent-variable interpretation. Define the intended decision before reducing a survey to a few scores. Examine content coverage, missing responses, subgroup comparability, and the consequences of treating a factor score as a performance measure. Preserve item wording, coding, estimator choices, and model specifications so scores remain interpretable across versions. A concise dashboard score is useful only when the underlying measurement supports its intended interpretation.

Check your understanding

In the toy model, why is the correlation 0.48 rather than 0.8 or 0.6?

Answer: The shared factor contributes covariance equal to the product of the loadings, 0.8 times 0.6, given unit factor variance and uncorrelated errors. Standardized observed variances make that covariance equal the correlation.

Related terms

Latent variable
Covariance
Construct validity
Principal component analysis
factor loading
communality
uniqueness
exploratory factor analysis
confirmatory factor analysis
rotation

Learn next

Learn covariance and standardization before working through a one-factor equation. Then compare factor analysis with PCA and study model identification. Explore rotation and factor-number selection next, followed by measurement validity and confirmation on new data rather than relying on a single appealing loading table.

Covariance

Sources

Scikit-learn, factor-analysis model reference: https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.FactorAnalysis.html . Statsmodels, factor results, loadings, uniqueness, and implied covariance: https://www.statsmodels.org/stable/generated/statsmodels.multivariate.factor.FactorResults.html . Primary project documentation excerpts were consulted. The numerical model and case are original illustrations rather than estimated research results; statistical and editorial review remains pending.

Previous: Factor Next: Factor graph