AI Glossary / Causal inference and experimentation / Intermediate
F 1
Definition
F1, sometimes written F 1 or F1 score, combines precision and recall through their harmonic mean. For a designated positive class, it equals 2PR/(P + R), or equivalently 2TP/(2TP + FP + FN) when the denominator is nonzero. Higher values indicate a stronger balance between finding relevant cases and avoiding false positive predictions. F1 does not include true negatives directly and is not a complete measure of a classifier's operational value.
What it means
Precision asks how many predicted positives are actually positive. Recall asks how many actual positives were found. A system can score highly on one by being very selective or very permissive, while doing poorly on the other. F1 is high only when both are reasonably high. It summarizes a tradeoff, but the equal role of precision and recall in the formula does not mean false positives and false negatives have equal real-world costs.
How it works
Choose the positive class and decision threshold, then count true positives, false positives, and false negatives on evaluation data. Calculate precision and recall, or use the count formula directly. For multiple classes or labels, declare the averaging method: macro F1 averages class-level F1 values equally, weighted F1 weights them by support, and micro F1 pools relevant counts before calculation. State the treatment of classes with no positive examples or predictions. Different averaging conventions can lead to different conclusions, even on the same predictions.
Three practical examples
- Message filtering: summarize precision and recall for identifying messages that require review, while keeping the underlying counts visible.
- Information extraction: evaluate identified entities or relations under an explicitly defined matching rule.
- Multiclass routing: use macro F1 when each class should contribute equally to the summary, with class-level results alongside it.
Think of it this way
Think of two necessary ingredients in a recipe: having plenty of one cannot fully compensate for very little of the other. The harmonic mean behaves similarly, pulling the combined score toward the weaker component. The analogy explains the balance, but it does not determine how an organization should value different kinds of errors or allocate limited review capacity.
Worked example
Suppose a detector identifies 30 true positives and 10 false positives, while missing 20 actual positives. Precision is 30/(30 + 10) = 0.75, and recall is 30/(30 + 20) = 0.6. F1 is 2 times 0.75 times 0.6 divided by 1.35, which equals 2/3, approximately 0.6667. The count formula gives the same result: 60/(60 + 10 + 20) = 60/90.
Now compare two hypothetical systems with precision-recall pairs (0.9, 0.5) and (0.5, 0.9). Both have F1 = 0.9/1.4, approximately 0.6429, despite very different behavior. One makes relatively few false positive predictions but misses more positives; the other finds more positives with a noisier set of predictions. The shared F1 does not make them operationally equivalent. Adding correctly classified negative examples without changing TP, FP, or FN leaves the binary positive-class F1 unchanged, illustrating why true-negative performance needs separate attention when it matters.
Illustrative case study
A fictional support-routing team celebrates a rise in weighted F1 after an update. It then inspects category-level results and discovers that a frequent category improved while a rare specialist category deteriorated. Macro F1 and the individual class scores make that tradeoff visible. The team reviews the affected requests and compares the consequences with its service requirements. It keeps weighted F1 for one reporting purpose but stops treating it as proof that every category improved.
Common misconceptions and mistakes
F1 is not ordinary accuracy and not the arithmetic mean of precision and recall. It can hide important differences between two error patterns. A high score does not establish well-calibrated probabilities, because F1 normally uses thresholded predictions. Macro F1 is the mean of class F1 values, not generally the F1 calculated from macro precision and macro recall. In ordinary single-label multiclass classification over all classes, micro F1 equals accuracy; that equivalence should not be generalized to every multilabel or restricted-label setting.
Business use
Use F1 when precision and recall both matter and a compact summary is helpful. Report the positive class, matching rule, threshold, averaging, and underlying precision and recall. Select thresholds on development data, then evaluate the chosen procedure separately. If missing positives and producing false positives have very different consequences, consider an explicitly justified F-beta measure or a cost-based decision analysis. Keep capacity, error severity, and subgroup performance visible instead of relying on a single harmonic mean.
People also ask
What is the best possible F1?
One, achieved when precision and recall are both one for the evaluated target and the quantities are defined.
What happens when there are no positives or positive predictions?
The formula can be undefined; software applies a configurable convention that must be documented.
Can two models have the same F1 but different usefulness?
Yes. Different precision-recall combinations and error types can produce the same score.
Check your understanding
Why does adding more true negatives leave the binary F1 unchanged in the worked example?
Answer: The formula uses true positives, false positives, and false negatives only. Correctly rejected negatives do not enter it directly, so they need another measure if their performance is important.
Related terms
- Precision
- Recall
- Harmonic mean
- Confusion matrix
- F-beta score
- macro averaging
- micro averaging
- weighted averaging
- decision threshold
- class imbalance
Learn next
Calculate F1 from both precision-recall values and confusion counts. Then compare macro and micro results on a small imbalanced multiclass example. Study precision-recall curves and threshold selection next so the chosen operating point reflects the workflow rather than an unexplained default threshold.
More in Causal inference and experimentation
- Exact match - Exact match is an evaluation rule that gives credit only when a prediction equals an accepted reference answer under the specified comparison procedure.
- Experiment - An experiment is a planned investigation in which a researcher deliberately changes one or more conditions and observes the resulting outcomes.
- Extrinsic evaluation - Extrinsic evaluation assesses an AI component or output by measuring its effect on a downstream task or wider application.
- False negative rate - False negative rate is the proportion of actual positive cases that a binary classifier incorrectly labels negative.
- False positive rate - False positive rate is the proportion of actual negative cases that a binary classifier incorrectly labels positive.
- Fowlkes-Mallows scores - Fowlkes-Mallows scores compare two partitions of the same observations by examining which pairs are grouped together.
Sources
Scikit-learn, f1_score: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html Scikit-learn, precision_recall_fscore_support: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_fscore_support.html Scikit-learn, metrics and scoring guide: https://scikit-learn.org/stable/modules/model_evaluation.html
Previous: Eye tracking Next: F distribution