AI Glossary / Causal inference and experimentation / Intermediate
Exact match
Definition
Exact match is an evaluation rule that gives credit only when a prediction equals an accepted reference answer under the specified comparison procedure. Each evaluated item usually receives either full credit or no credit, and the overall score is the fraction of items that match. Exactness may apply to raw text, normalized text, structured fields, or a complete set of labels. The comparison rules must be stated because “exact” does not always mean identical raw characters.
What it means
Some tasks require one unambiguous answer, such as a category code or an extracted identifier. Exact match is attractive because it is simple and easy to reproduce. Other tasks permit several equivalent forms or open-ended explanations, where strict matching can reject useful answers. Normalization can remove irrelevant differences, but can also erase meaningful ones. The metric is therefore best understood as agreement with a declared answer representation, not an automatic judgment of semantic correctness.
How it works
Define the evaluated unit and accepted references. Apply any permitted normalization consistently to both prediction and reference, then compare the resulting values. Assign 1 if the prediction matches an accepted reference and 0 otherwise, and average over eligible items. For multiple references, the protocol may accept a match to any one of them. A structured-output task must define field ordering, numerical representation, and whether the entire object or individual fields are compared. Missing outputs and unanswerable questions also need explicit scoring rules.
Three practical examples
- Extraction: check whether a predicted invoice identifier exactly matches the reviewed identifier without removing meaningful punctuation.
- Question answering: compare short answers against accepted references using the benchmark's declared normalization.
- Multilabel classification: require the complete predicted label set to match the reference set, rather than awarding partial credit for individual labels.
Think of it this way
Think of a key that must fit a particular lock. A nearly matching shape may still fail, just as a nearly correct identifier can be unusable. The analogy works for strict outputs, but language often allows several legitimate keys: different expressions can convey the same answer, so the accepted representations must reflect the actual task.
Worked example
Consider five short-answer pairs under a teaching rule that trims outer spaces and ignores letter case, but changes nothing else. Reference “Paris” with prediction “ paris ” matches after normalization. Reference “42” with prediction “42” matches. Reference “red blue” with prediction “blue red” does not match because word order remains significant. Reference “A-17” with prediction “A17” does not match because punctuation remains significant. Reference “5” with prediction “five” does not match because numeric and written forms are not converted.
The score is 2/5 = 40%. If a different task explicitly treats “5” and “five” as equivalent, an expanded reference set or documented conversion could raise the score to 3/5 = 60%. That is a change in the evaluation rule, not a change in model output. In SQuAD-style text evaluation, normalization includes operations such as lowercasing and removal of punctuation and English articles. Those benchmark conventions should not be copied blindly into identifier extraction, where the hyphen in “A-17” may be meaningful.
Illustrative case study
A fictional form-processing team uses whole-record exact match to assess five required fields per form. It discovers that many failed records contain only one incorrect field, while a smaller group has several serious errors. The team retains whole-record matching because its downstream import requires a completely correct record, but adds field-level analysis to guide improvements. It also records accepted date and number formats. This keeps the strict operational requirement visible while making the failure report useful for diagnosis.
Common misconceptions and mistakes
Exact match is not always a raw string comparison, and normalization is not automatically harmless. Removing punctuation can merge distinct identifiers; ignoring order can change a sentence's meaning. A correct paraphrase may score zero if it is absent from the accepted references. Conversely, a matching answer can be obtained through memorization or an unintended shortcut. In multilabel evaluation, complete-set matching is stricter than Hamming-style per-label scoring. An exact-match score alone does not explain the severity or source of failures.
Business use
Use exact match when the task has a clear accepted representation or when complete correctness is operationally necessary. Publish the normalization and reference policy with the score. Inspect failures to distinguish genuine mistakes from missing acceptable variants, then revise the protocol carefully and version it. Pair strict matching with diagnostic measures where partial correctness matters. Avoid selecting normalization rules solely because they improve the score; the rules should follow the task's meaning and the downstream system's requirements.
People also ask
Can multiple different answers receive full credit?
Yes, if the protocol provides several accepted references or a justified canonical representation.
Does exact match give partial credit?
Not for the evaluated unit; partial-credit measures must be reported separately.
Is exact match appropriate for long creative responses?
Usually it is a poor standalone measure because many valid responses can differ from any supplied reference, though specific constrained components can still be checked exactly.
Check your understanding
Why is “A17” rejected against “A-17” in the worked example?
Answer: The declared rule does not remove punctuation, and the hyphen may carry meaning in an identifier. Removing it without a task-based justification would change what correctness means.
Related terms
- Normalization
- Reference answer
- string equality
- canonicalization
- subset accuracy
- partial credit
- token F1
- structured output
- answer equivalence
- benchmark protocol
Learn next
Create a short list of predictions with case differences, punctuation, alternate number forms, and paraphrases. Score it first using raw equality and then a task-justified normalization rule. Review every changed result. Next study token-level F1 and semantic evaluation to understand when strict matching is useful and when complementary evidence is necessary.
More in Causal inference and experimentation
- Evaluation - Evaluation is the systematic assessment of an AI model, output, or complete system against defined purposes and criteria.
- Evaluation function - An evaluation function is a rule or program that assigns a value to something being assessed.
- Evaluation metric - An evaluation metric is a defined measure used to summarize an aspect of a model's or system's performance.
- Experiment - An experiment is a planned investigation in which a researcher deliberately changes one or more conditions and observes the resulting outcomes.
- Extrinsic evaluation - Extrinsic evaluation assesses an AI component or output by measuring its effect on a downstream task or wider application.
- F 1 - F1, sometimes written F 1 or F1 score, combines precision and recall through their harmonic mean.
Work in finance? Get the free CFO AI Prompt Pack: 7 prompts for finance leaders.
Sources
SQuAD, original version 2 evaluation implementation: https://github.com/rajpurkar/SQuAD-explorer/blob/master/evaluate-v2.0.py Scikit-learn, accuracy_score and multilabel subset accuracy: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html Google BIG-bench, task metric documentation: https://github.com/google/BIG-bench/blob/main/docs/doc.md
Previous: Exact cell decomposition Next: Exact test