AI Glossary / Natural language and speech / Intermediate
F-measure
Definition
F-measure is a family of evaluation scores that combine precision and recall into one number. The best-known member, F1, is their harmonic mean. A more general version, F-beta, changes their relative emphasis through a positive parameter beta, with larger beta favoring recall and smaller beta favoring precision.
What it means
Precision asks how many selected or predicted-positive items were correct. Recall asks how many of the relevant or actual-positive items were found. A system can do well on one while doing poorly on the other.
F-measure provides a compact way to summarize that trade-off. The harmonic mean prevents one very high component from fully compensating for a very low one.
The score does not explain every consequence of a decision. It does not directly include true negatives in its binary count formula, and it is not a complete expression of financial cost, workload, fairness, or reliability. The task, positive class, decision threshold, and averaging convention must therefore accompany the number.
How it works
Count true positives, TP, false positives, FP, and false negatives, FN, under a stated matching rule. Precision is TP / (TP + FP), and recall is TP / (TP + FN), when those denominators are nonzero.
F1 can be calculated as 2PR / (P + R), or directly as 2TP / (2TP + FP + FN). For positive beta, F-beta is (1 + beta²)TP / [(1 + beta²)TP + FP + beta²FN].
For multiple classes, micro averaging pools counts before scoring, while macro averaging calculates class scores first and then gives them equal weight. These can answer different questions.
If there are no actual or predicted positives, the relevant denominator is zero. Software needs an explicit convention for that case; an unexplained default can distort comparisons.
Three practical examples
- An information-extraction team combines precision and recall for correctly identified organization mentions under an exact-span matching rule.
- A document-search experiment uses an F-measure to summarize which relevant documents were retrieved at a chosen cutoff.
- A text-classification team reports macro F1 alongside class-specific results so common categories do not completely dominate the summary.
Think of it this way
Think of balancing two necessary capabilities: selecting useful items and finding the useful items that exist. The combined score falls when either capability is weak. The analogy stops at evaluation: the mathematical balance does not automatically represent the real cost of every missed or incorrect item.
Worked example
In an invented document-review test, 30 documents are relevant. A system selects 25 documents, of which 20 are relevant. Thus TP = 20, FP = 5, and FN = 10.
Precision is 20 / 25 = 0.8, while recall is 20 / 30, approximately 0.667. F1 is 40 / (40 + 5 + 10) = 40 / 55, approximately 0.727.
With beta = 2, F2 is 100 / (100 + 5 + 40) = 100 / 145, approximately 0.690. With beta = 0.5, F0.5 is 25 / (25 + 5 + 2.5) = 25 / 32.5, approximately 0.769.
Because recall is lower than precision here, emphasizing recall lowers the combined score, while emphasizing precision raises it. None of these changes alters the predictions themselves.
Adding many correctly rejected irrelevant documents would leave these F scores unchanged because TP, FP, and FN stay the same. It could change accuracy, illustrating why the two metrics are not interchangeable.
Illustrative case study
In a fictional project, Birch Library compares classifiers that identify requests needing specialist routing. The team initially sees one attractive F1 score but discovers that a rare request category performs poorly.
Reviewers inspect the class-specific precision and recall and compare macro and micro summaries. They also examine how different thresholds change the number of requests sent for review.
The imagined workflow treats F-measure as one part of an evaluation report. It does not assume that a higher F1 guarantees a lower workload or that a chosen beta precisely encodes the organization’s operational costs.
Common misconceptions and mistakes
F1 is not the arithmetic average of precision and recall. For unequal positive values, the harmonic mean is lower than their arithmetic mean.
F-measure is not accuracy and does not directly reward true negatives in the binary formula.
“F1” alone is incomplete for a multiclass task unless the averaging method and included classes are clear.
Finally, beta should not be interpreted as a universal monetary exchange rate between error types. Its mathematical role is defined by the formula, and a business decision may require explicit cost analysis in addition to the score.
Business use
F-measure is useful for comparing systems when both missed targets and incorrect selections matter. It often appears in search, extraction, and classification reports, but comparisons require consistent test data and correctness rules.
Report precision, recall, support counts, and the chosen threshold alongside the combined measure. These reveal whether a similar F score hides very different behavior.
Choose the evaluation objective before inspecting final test results. Repeatedly selecting a threshold or beta to make the test score look favorable undermines the test’s independence and makes the reported result less informative about future performance.
People also ask
Does F1 equal one only when precision and recall are both one?
For a defined positive-class evaluation, yes; neither false positives nor false negatives remain.
Is macro F1 the F1 calculated from macro precision and macro recall?
Generally no. Macro F1 averages the class-level F1 values directly.
Does changing beta retrain the model?
Not by itself. It changes the evaluation calculation, although a team may later use that objective during model or threshold selection.
Check your understanding
A system has TP = 8, FP = 2, and FN = 6. What is its F1?
Answer: F1 = 2 × 8 / (16 + 2 + 6) = 16 / 24, approximately 0.667. The true-negative count is not needed for this calculation.
Related terms
- Precision
- Recall
- Harmonic mean
- Confusion matrix
- Classification threshold
- Accuracy
- F1 score
- macro averaging
- micro averaging
- information retrieval evaluation
Learn next
Learn precision and recall from a confusion matrix, then calculate F1 and F-beta by hand on a small example. Next, compare micro and macro averages across uneven class sizes. Finally, connect the metrics to the actual costs and review capacity of the intended workflow.
More in Natural language and speech
- Example-based machine translation - Example-based machine translation, or EBMT, translates new input by retrieving relevant bilingual examples and adapting their target-language counterparts.
- Explicit semantic analysis - Explicit semantic analysis, usually abbreviated ESA, represents text as a weighted vector of named concepts drawn from a knowledge collection.
- Extraposition - Extraposition is a grammatical construction in which a clause or phrase appears later in a sentence than in a corresponding more basic arrangement.
- Factoid questions - Factoid questions seek a specific factual answer that can usually be expressed briefly, such as a person’s name, a place, a date, or a quantity.
- Filled pause - A filled pause is a vocalized hesitation, such as “uh” or “um” in English, occurring within the flow of speech.
- Filtered-popping recursive transition network - A filtered-popping recursive transition network, abbreviated FPRTN, is a recursive transition network that permits a return from a subroutine only when the...
Sources
Official scikit-learn F-beta documentation, formulas, averaging, and zero-division conventions: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.fbeta_score.html. Manning, Raghavan, and Schütze, Introduction to Information Retrieval, evaluation of unranked retrieval: https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-unranked-retrieval-sets-1.html. Independent review checked the official formula, micro/macro definitions and zero-division policy.
Previous: F-logic Next: F-test of equality of variances