Skip to content
← Blog

Harman's single-factor test: why it is not enough to rule out common method bias

Author:

In short: Harman's single-factor test only picks up common method variance when it is extreme. It misses milder method effects that still bias results, and it cannot tell you which relationships are affected. A result such as "the first factor explains less than 50% of the variance" is therefore no evidence that your results are free of bias.

In research on work, we collect most variables (autonomy, workload, engagement, attitudes to AI) with a single questionnaire from the same person. How much of the relationship between them is created by the method itself is therefore a question almost every doctoral thesis in management and HRM has to answer. Harman's test is the weakest of the common ways to answer it.

Video: Harman's single-factor test is not enough (Methodology mini-glossary, in Slovak)
Methodology mini-glossary: Harman's single-factor test (3:12, in Slovak with Slovak subtitles). The video plays via YouTube after you click.

How the test works

You enter all items from all scales in the questionnaire into a single exploratory factor analysis. You use no rotation and extract a single factor (or look at the first factor of the unrestricted solution). If a common method dominated, one general factor should explain most of the shared variance. In practice, a 50% threshold has become established: if the first factor stays below it, the author concludes that common method bias is not a problem.

In SPSS: Analyze → Dimension Reduction → Factor, items from all scales, Extraction: principal components or principal axis factoring, Fixed number of factors = 1, Rotation: None. Then read the percentage of variance explained by the first factor.

Why it is not enough

The test is not reliable. Podsakoff et al. (2003) described it as insensitive. In a simulation, Fuller et al. (2016) found that the test signals a problem only when the common method accounts for roughly 70% or more of the variance. In their view, this is also the level at which CMV starts to inflate relationships substantially, so they consider concerns about CMB often overstated. Yet they too showed that the test produces both false positives and false negatives. Baumgartner et al. (2021), by contrast, showed analytically that measurement unreliability does not necessarily offset bias due to a common method and that the inflationary effect of CMV is largest for weaker relationships. They strongly recommend against using the test, as do Aguirre-Urreta and Hu (2019).

The result depends on what you put into the analysis. The more scales you include and the less related they are, the harder it is for the first factor to exceed 50%. The same method effect can therefore "pass" in one model and fail in another. Baumgartner, Weijters and Pieters (2021) also show analytically why the share of variance explained by the first factor is a poor measure of how biased the relationships are.

It cannot tell you where or how much. Even when the test detects a problem, you get one percentage for the whole questionnaire. It does not tell you which relationships are biased or by how much.

Example: the test passed even with problematic data

As an illustration, we use a random subset of data from an online panel survey of employees in Czechia and Slovakia conducted in 2026: 600 completed questionnaires and four reflective scales (labelled A to D for this article). The subset contained 40 respondents (6.7%) with repeated response patterns. They answered following the same pattern regardless of the content of the questions, as we describe in Repeated response patterns. We excluded another 28 for careless responding, leaving a final sample of 532 respondents.

Before cleaning (n = 600) After cleaning (n = 532)
First factor: principal component analysis 31.6% 30.8%
First factor: principal axis factoring 29.5% 28.7%
Full collinearity VIF (constructs A to D) 1.13 to 2.46 1.09 to 2.59

In both cases, Harman's test would have "confirmed" that common method bias is not a problem. Yet the first factor barely changed after we removed 40 respondents who answered independently of the question content. The test did not even register a group whose answers were, by definition, driven by the way they filled in the questionnaire rather than by content.

Full collinearity VIF (Kock, 2015), a common alternative in PLS-SEM, did not flag anything either: all values are below the recommended threshold of 3.3. It is a better indicator than Harman's test, but it is also only indirect. What matters most is what you do before collecting the data. That is the topic of the article on common method bias.

How to report it

If you report the test (for example because a reviewer or journal asks for it), present it as a supplementary check and do not overstate it. Put the main weight on procedural remedies and a stronger statistical check.

To reduce the risk of common method bias, we [describe procedural remedies, e.g., temporal separation, anonymity, separation of item blocks]. As a statistical check, we report full collinearity VIFs (Kock, 2015); all values were below 3.3. We report Harman's single-factor test (the first factor explained 30.8% of the variance) only as a supplementary check, because it rules out an extreme level of common method variance but does not reliably detect more moderate levels (Podsakoff et al., 2003; Aguirre-Urreta & Hu, 2019).

Common mistakes I see as a reviewer

  • "CMB was not a problem" based on Harman's test alone. The most common and most easily challenged statement. The correct wording is "the test did not indicate an extreme level of…".
  • No information on what went into the analysis. State whether all items were included, which extraction method you used and that no rotation was applied.
  • Harman's test without any procedural remedies. If you did nothing when designing the questionnaire, no post hoc test can make up for it. It is better to acknowledge this in the limitations.
  • Applying the 50% threshold to other methods. The result of Harman's test says nothing about VIF, HTMT or the quality of the measurement model.

Cheat sheet: if you already have Harman's test or want to report it

  1. Report it as a supplementary check, with the extraction method and the percentage for the first factor.
  2. State what the test shows (no extreme CMV) and what it does not show (that relationships are unbiased).
  3. Put procedural remedies and a stronger check next to it: full collinearity VIF in PLS-SEM, ideally a marker variable or a latent method factor if your design allows it.

Sources

  • Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879
  • Fuller, C. M., Simmering, M. J., Atinc, G., Atinc, Y., & Babin, B. J. (2016). Common methods variance detection in business research. Journal of Business Research, 69(8), 3192–3198. https://doi.org/10.1016/j.jbusres.2015.12.008
  • Aguirre-Urreta, M. I., & Hu, J. (2019). Detecting common method bias: Performance of the Harman's single-factor test. The DATA BASE for Advances in Information Systems, 50(2), 45–70. https://doi.org/10.1145/3330472.3330477
  • Baumgartner, H., Weijters, B., & Pieters, R. (2021). The biasing effect of common method variance: Some clarifications. Journal of the Academy of Marketing Science, 49, 221–235. https://doi.org/10.1007/s11747-020-00766-8
  • Kock, N. (2015). Common method bias in PLS-SEM: A full collinearity assessment approach. International Journal of e-Collaboration, 11(4), 1–10. https://doi.org/10.4018/ijec.2015100101