Is it wrong to form a hypothesis after seeing data?

No. The issue is presenting it as if it preceded those data.

Does preregistration prevent every problem?

No. Design quality, complete reporting, and transparent changes still matter.

Can an exploratory finding be useful?

Yes. It can suggest a valuable explanation or next experiment while remaining appropriately tentative.

AI Glossary / Data science and analytics / Intermediate

Harking

Definition

HARKing means hypothesizing after the results are known and presenting the resulting hypothesis as though it had been proposed before examining those results. The term focuses on how the origin of a hypothesis is represented. Discovering an unexpected pattern and openly describing it as exploratory is different from claiming that the same pattern was a planned prediction.

What it means

A hypothesis formed in advance takes a risk: the observations may fail to support it. A hypothesis tailored to an observed result has already benefited from seeing what happened. Both can be useful, but they provide different kinds of evidence. If a report hides that distinction, readers may believe a theory successfully predicted something that actually helped create the theory. In business analytics, the same problem can occur when a retrospective dashboard insight is presented as a successful forecast.

How it works

HARKing can arise when an analyst explores several outcomes, groups, or relationships, notices one pattern, and rewrites the original question around it. The final narrative may omit the unsuccessful expectations or the sequence of exploratory decisions. A practical response is to preserve the original research question and analysis plan, record changes, and clearly identify newly generated hypotheses. Preregistration can make the timing of a plan more visible, but it does not guarantee a sound design or accurate reporting. A new hypothesis can then be examined with suitable fresh data or an appropriately protected holdout sample.

Three practical examples

Product testing: a team plans to assess overall task completion, finds no clear improvement, then discovers a possible benefit for a subgroup. It should describe the subgroup idea as arising during exploration.
Customer analytics: a seasonal pattern noticed after inspecting sales charts can motivate a future test, but should not be portrayed as a forecast made beforehand.
Model evaluation: an unexpected strength on one benchmark category can generate a research question without retroactively becoming the model's original design hypothesis.

Think of it this way

Think of drawing a target around an arrow after it has landed and then claiming the archer aimed at that target. The analogy captures the misleading chronology. It does not mean that looking at where arrows land is useless: those observations may suggest an interesting target for a genuinely new attempt.

Worked example

A fictional team records a plan to test whether a redesigned help page increases overall successful task completion. After analyzing the experiment, it finds no clear overall improvement. It then explores 20 different subgroup comparisons and notices one with a p-value below 0.05. The team develops an explanation involving the subgroup's prior experience.
The explanation may be worth investigating, but presenting it as the original prediction would conceal the exploration. As a simplified illustration, if all 20 null hypotheses were true and the tests were independent with a 5% false-positive rate, the probability of at least one false positive would be 1 minus 0.95 to the twentieth power, approximately 64%. Real subgroup tests can be dependent, so this is not a calculation of the actual experiment's error rate. It illustrates why selecting a result from many opportunities differs from one prespecified test. The honest report states the original outcome, describes the exploratory search, and proposes a separately planned follow-up.

Illustrative case study

In a fictional AI training study, a team expects shorter lessons to improve average quiz performance. The average result is inconclusive, but an analyst notices a pattern among participants who already use coding tools. During report preparation, a colleague suggests making that subgroup effect the study's opening prediction. The team instead preserves the original question and labels the subgroup interpretation as exploratory. It documents how the subgroup was identified and designs a follow-up study. No claim is made that the new effect will replicate; the useful outcome is a clearer account of what the evidence does and does not show.

Common misconceptions and mistakes

HARKing is not the same as all post hoc reasoning. Exploration and theory development are legitimate when reported transparently. It is also distinct from p-hacking, although searching across analyses and rewriting hypotheses can occur together. A preregistration does not excuse selective reporting of planned results or unexplained deviations. Nor does a compelling explanation turn an observed association into a causal effect. Avoid assigning intent to an individual from an unclear report alone; the relevant editorial task is to clarify the chronology and evidential claim.

Business use

For business experiments, record the decision question, primary measure, target population, and analysis approach before examining outcomes. Keep a dated decision log when the plan changes. Reports should distinguish planned findings from ideas generated during analysis and include important results that did not support expectations. This helps decision-makers judge whether they are acting on a tested prediction or a promising lead. Reserve enough independent evidence for follow-up when a costly launch decision depends on an unexpected pattern.

Check your understanding

A subgroup hypothesis is developed after examining results, and the report explicitly says so. Is that the misleading practice described here?

Answer: No. Transparent hypothesis generation is different from claiming the subgroup prediction was made in advance. The finding still needs evidence appropriate to its exploratory origin.

Related terms

p-hacking
Replication (statistics)
Exploratory analysis
confirmatory research
preregistration
multiple testing
selective reporting
replication
holdout dataset

Learn next

Compare an original analysis plan with a final report and mark which claims were planned, revised, or newly generated. Then learn how multiple comparisons and repeated access to a holdout set affect inference. Practice writing an unexpected result as a follow-up question without overstating its confirmation.

Sources

Norbert L. Kerr, original 1998 article abstract defining HARKing: https://pubmed.ncbi.nlm.nih.gov/15647155/ Center for Open Science, preregistration and planned versus exploratory work: https://www.cos.io/initiatives/prereg Center for Open Science, reporting plans, changes, and exploratory analyses: https://www.cos.io/initiatives/prereg-more-information

Previous: Hard Sigmoid Function Next: Harmful bias