AI Glossary / Natural language and speech / Beginner
Word error rate
Definition
Word error rate, or WER, measures the word-level differences between a speech-recognition transcript and a reference transcript. It counts substitutions, deletions and insertions, then divides their total by the number of reference words.
What it means
A substitution replaces a reference word with another word. A deletion omits a reference word. An insertion adds a word not present in the reference. Lower WER indicates fewer such errors under the chosen scoring rules.
The comparison requires alignment: after one missing word, later words should not all be treated as wrong simply because their positions shifted. An edit-distance procedure finds an appropriate minimum-cost alignment.
WER is a transcript comparison, not a complete measure of understanding or task success. A single mistaken account number may matter more operationally than several missing conversational fillers. The reference also needs quality control, because an incorrect reference can make a correct recognizer appear wrong.
How it works
With S substitutions, D deletions, I insertions and N reference words, WER = (S + D + I) / N. Multiplying by 100 expresses the result as a percentage.
The scoring protocol must specify text preparation, including case, punctuation, numbers and tokenization. “Twenty one” and “21” may be treated differently depending on that protocol. Comparisons are meaningful only when the rules and test material are comparable.
For a corpus-level score, sum the error counts and reference-word counts across the included recordings before dividing. This weights recordings by their number of reference words. A simple average of per-recording WERs answers a different question and can give short recordings disproportionate influence.
Three practical examples
- A transcription team compares two recognizers on the same held-out audio and reference transcripts.
- A developer investigates whether substitutions or deletions explain poor recognition of specialist vocabulary.
- A service team reports overall WER alongside error rates for names, numbers and task-critical phrases.
Think of it this way
Think of proofreading a typed copy against a reference document and counting replacements, omissions and additions. Alignment prevents one missing word from making every later position appear wrong.
The analogy is limited because spoken language can contain hesitations, incomplete words and ambiguous boundaries. The reference and scoring conventions determine how those features are represented. The metric is not a direct reading of the speaker's intention.
Worked example
Calculating errors and aggregation
Use the invented reference “send the blue box today”, which contains five words. The recognizer outputs “send a blue box”. A natural minimum-edit alignment has one substitution, a for the, and one deletion, today. There are no insertions.
WER is (1 + 1 + 0) / 5 = 0.4, or 40%.
Now consider a separate two-recording test. Recording A contains ten reference words and two errors; recording B contains ninety reference words and nine errors. Their individual WERs are 20% and 10%.
The pooled score is (2 + 9) / (10 + 90) = 11%. The unweighted mean of the two percentages is 15%. Both calculations are arithmetically valid, but they summarize different units of analysis.
Finally, a one-word reference “hello” compared with “hello extra extra” has two insertions and WER 200%. WER can exceed 100% because inserted words increase the numerator without increasing the reference denominator.
Illustrative case study
A fictional meeting-transcription provider evaluates a recognizer on recordings from several customer teams. Its initial summary reports a low aggregate WER, but reviewers find recurring mistakes in project identifiers.
The provider keeps WER for overall comparison and adds a focused check for those identifiers. It also documents how numbers and abbreviations are normalized, so a formatting change does not appear to be an acoustic improvement.
The team inspects errors by recording condition and speaker group, rather than relying only on the pooled score. It tests whether corrections improve the intended workflow without overstating what one metric establishes. This is an illustrative evaluation process, not a claimed product benchmark.
Common misconceptions and mistakes
- Dividing by the number of recognized words instead of reference words.
- Averaging percentages without saying whether recordings or words receive equal weight.
- Assuming WER cannot exceed 100%.
- Comparing scores from different datasets or normalization rules as if they were directly equivalent.
- Treating every error as equally consequential for the application.
- Describing one minus WER as a reliable probability that a whole sentence is correct.
Business use
WER provides a useful common measure for transcript quality, but purchasing and deployment decisions require context. Specify the audio domain, language, recording conditions and reference conventions alongside the score.
Use a test set separate from material used to tune the system. Preserve enough examples to understand the errors instead of reporting only a single percentage. If names, dates or quantities drive the business task, evaluate those explicitly and inspect complete-task outcomes.
Changes of a few percentage points should be assessed against dataset size and variation across recordings. Report absolute and relative improvements carefully: reducing WER from 20% to 15% is five percentage points, or a 25% relative reduction. Neither statement establishes a 25% improvement in every downstream workflow.
People also ask
Does WER measure speaker identification?
No. Standard WER compares word sequences. Correct speaker attribution requires additional evaluation.
What happens when the reference has no words?
The ordinary formula divides by zero. The evaluation protocol must define how empty-reference cases and inserted output are handled.
Is character error rate the same metric?
It uses analogous edit operations at character level. Its values and tokenization implications differ from word-level scoring.
Check your understanding
A test has 200 reference words, 12 substitutions, 5 deletions and 3 insertions. What is WER?
Answer: (12 + 5 + 3) / 200 = 20 / 200 = 10%. The denominator is the reference-word count, including words omitted by the recognizer.
Related terms
- Automatic Speech Recognition
- Edit distance
- sequence alignment
- substitution
- insertion
- deletion
- character error rate
- reference transcript
Learn next
Read edit distance and speech-recognition evaluation. Align two short transcripts by hand, then calculate pooled and per-recording average WER on unequal-length examples. Explain why the summaries differ and which matches the evaluation goal.
More in Natural language and speech
- Weight tying - Definition: Weight tying makes two or more parts of a neural network use the same trainable parameters.
- Wh-pronoun - Definition: A wh-pronoun is a pronoun from the family of forms such as who, whom, what and which that can be used in questions and related constructions.
- Wikification - Definition: Wikification is the process of identifying relevant terms or mentions in a text and linking them to the appropriate Wikipedia articles.
- Word list - A word list is an organized collection of words selected for a stated purpose.
- Word sense - Definition: A word sense is a particular meaning associated with a word.
- Word sense disambiguation - Definition: Word sense disambiguation, or WSD, is the task of deciding which meaning of a word is intended in a particular context.
Sources
NIST, OpenASR21 challenge report, metrics section: https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=934557. NIST Speech Recognition Scoring Toolkit: https://github.com/usnistgov/SCTK. These primary resources document WER calculation and scoring tools. All transcript examples, aggregation figures and the provider scenario are original illustrations.
Previous: Word embedding Next: Word list