Does using one questionnaire prove common source bias?

No. It creates a potential concern whose importance depends on the measurement process and the relationship being estimated.

Will collecting the outcome from another person solve everything?

Not necessarily. The second source may share errors or measure a different construct.

Is common source bias always positive?

No. Its direction depends on how shared influences enter the measures and relate to the underlying variables.

AI Glossary / Statistics and probability / Intermediate

Common source bias

Definition

Common source bias is distortion in a measured relationship that arises because the variables are obtained from the same information source. For example, one respondent's general optimism may influence both their rating of AI use and their rating of personal performance. Shared-source measurement creates a potential route for correlated error; it does not prove that every relationship measured from one source is biased.

What it means

Researchers may want to understand a relationship between two underlying characteristics, but their measurements can also reflect how the source reports them. A manager's halo effect, a respondent's desire for consistency or a shared reporting process can influence several variables together. Common source bias overlaps with common method bias, but source and method are not identical: different sources can use the same measurement method, and one source can supply several kinds of records. The relevant issue is how measurement-related influences alter the relationship being estimated.

How it works

Map each variable to its source, collection time, question wording and measurement process. Identify plausible shared influences, such as social desirability, recall style or a tendency to rate everything favorably. Where appropriate, separate sources, use independently recorded outcomes or redesign collection to reduce those influences while preserving the intended construct. Statistical models can examine some method effects, but they rely on identification and measurement assumptions. No single diagnostic proves that a dataset is free of common source bias. Consider competing explanations and the possibility that combining different sources introduces other mismatches or errors.

Three practical examples

Employee survey: distinguish the relationship between self-reported assistant use and self-reported productivity from a relationship measured with independent task outcomes.
Organizational research: examine whether a manager's overall view of a department influences both leadership and performance ratings.
AI evaluation: check whether using the same automated judge for several constructs creates correlated scoring tendencies that affect the reported association.

Think of it this way

Think of viewing two objects through the same tinted lens. Their apparent colors can resemble each other partly because of the lens rather than the objects. Using another lens may help investigate the effect, but only if it still measures the colors you intended to compare.

Worked example

Consider a deliberately simplified measurement model. True variables X and Y are independent, each with variance 1 and mean zero. A shared reporting influence M is independent of both, also with variance 1 and mean zero. Observed scores are X-star = X + M and Y-star = Y + M. Their covariance is 1 because the shared M contributes to both, while each observed variance is 2. The observed correlation is therefore 1/sqrt(2 x 2) = 0.5, even though the true correlation is zero. If the second score instead contains minus M, the observed correlation becomes -0.5. This constructed example shows that shared measurement effects can create either direction of distortion under specified assumptions; it does not estimate bias in an actual survey.

Illustrative case study

Fictional case: Laurel Research surveys employees about an AI assistant and asks the same people to rate their own performance. The initial correlation looks encouraging, but analysts recognize that enthusiasm about the project could influence both answers. They add a separately defined task-performance measure where appropriate and compare what each source actually captures. Differences between the measures are investigated rather than treated as proof that one source is automatically correct. The report explains the remaining measurement limitations and avoids a causal productivity claim. This invented case illustrates source-aware research design.

Common misconceptions and mistakes

Common source data are not automatically worthless, and self-reports can be the appropriate source for experiences such as perceived usefulness. Conversely, a large sample does not remove systematic shared measurement error. Common method variance is a description of variation attributable to measurement; bias concerns the distortion of a particular estimate. Shared influences can inflate or attenuate relationships depending on their structure. A one-factor test or a low correlation is not conclusive evidence that the problem is absent. Independent sources can also share incentives, information or errors, so independence must be substantive rather than just nominal.

Business use

Businesses researching AI adoption should choose measurement sources based on the question. Perceived productivity, logged activity and independently assessed output quality are related but different constructs. Combining them thoughtfully can improve interpretation, whereas replacing a subjective construct with an unrelated administrative metric can change the question entirely. Document who supplied each variable and how it was measured. Use sensitivity analyses and design improvements to assess plausible distortions, and communicate the limits of associations based entirely on one survey or scoring pipeline.

Check your understanding

Why is the observed correlation 0.5 in the worked example despite independent true variables?

Answer: Both observed scores contain the same M term. That term contributes covariance 1 and raises each variance to 2, producing correlation 1/2. The association is generated by the specified measurement mechanism rather than a true X - Y relationship.

Related terms

Measurement error
Construct validity
Latent variable
Common method bias
common method variance
correlated error
halo effect
social desirability bias
self-report
triangulation

Learn next

distinguish a construct from its measurement and trace how shared errors enter a covariance. Compare multiple-source and multiple-method designs, then study their assumptions before applying statistical corrections. For AI evaluation, audit whether judges share training, prompts or information that could create correlated scoring errors across supposedly separate measures.

Sources

Podsakoff, MacKenzie, Lee and Podsakoff, Common method biases in behavioral research: https://pubmed.ncbi.nlm.nih.gov/14516251/ George and Pandey, We Know the Yin—But Where Is the Yang? Toward a Balanced Approach on Common Source Bias in Public Administration Scholarship: https://pmc.ncbi.nlm.nih.gov/articles/PMC5633037/ IARC, Statistical Methods in Cancer Research Volume V, Dependent error: https://www.ncbi.nlm.nih.gov/books/NBK612863/box/box2_6/

Previous: Common random numbers Next: Commonality analysis