Word Error Rate (WER) Explained: How to Measure Speech Recognition Accuracy

Word error rate (WER) is the standard metric for speech-to-text accuracy. It measures how many words an ASR system got wrong relative to a human reference transcript: WER = (S + D + I) / N, where S is substituted words, D is deleted words, I is inserted words, and N is the total number of words in the reference. A WER of 10% means roughly one word in ten was wrong; lower is better, and because insertions count, WER can exceed 100%.

WER is simple, comparable, and universally reported. It is also one of the most misused numbers in the speech industry. The same model can score 3% or 18% depending on the test audio, and two vendors' "WER" figures are frequently not comparable at all because of different text normalization. This guide covers the formula with a worked example, how to compute it properly, the pitfalls, and what typical WER bands actually look like in 2026.

How to calculate word error rate: formula and worked example

WER is based on minimum edit distance (Levenshtein distance) at the word level: align the hypothesis (ASR output) to the reference (ground truth) so that the total number of edits is minimized, then count each edit type.

Reference: the cat sat on the mat Hypothesis: the cat sat in the hat today

Optimal alignment:

Two substitutions (on→in, mat→hat), one insertion (today), zero deletions, N = 6 reference words:

WER = (2 + 0 + 1) / 6 = 0.50 = 50%

Two details people miss: N always comes from the reference, never the hypothesis, and the alignment must be optimal, since greedy left-to-right matching can overcount errors. Related metrics from the same alignment: word accuracy (1 − WER, when clamped) and CER, the same computation at the character level.

How to compute WER in practice: jiwer

The de facto Python tool is jiwer, which handles alignment, error counting, and (critically) normalization:

import jiwer

transform = jiwer.Compose([
    jiwer.ToLowerCase(),
    jiwer.RemovePunctuation(),
    jiwer.RemoveMultipleSpaces(),
    jiwer.Strip(),
    jiwer.ReduceToListOfListOfWords(),
])

out = jiwer.process_words(
    reference, hypothesis,
    reference_transform=transform,
    hypothesis_transform=transform,
)
print(out.wer, out.substitutions, out.deletions, out.insertions)

jiwer.visualize_alignment(out) prints the word-by-word alignment, which is the fastest way to debug why a WER number is higher than expected. For research-grade comparisons, NIST's sclite and the Whisper-style normalizers (published by OpenAI and used by the Hugging Face Open ASR Leaderboard) are common alternatives.

Normalization pitfalls that make WER comparisons meaningless

WER compares strings, so any formatting difference counts as an error unless you normalize it away. The classic traps:

• Casing: "Turn Left" vs "turn left" counts as two substitutions if you forget ToLowerCase.
• Punctuation: "let's go." vs "lets go". Punctuation glued to words changes tokens.
• Numerals: reference says "twenty five", hypothesis says "25". One system's inverse text normalization (ITN) produces digits, another spells numbers out; without a shared convention this alone can add several points of WER.
• Contractions, hyphens, fillers: "cannot"/"can't", "e-mail"/"email", and whether "uh"/"um" are scored or stripped.
• Compounds and spelling variants in languages like German, or transliteration variance in Hindi and Arabic.

The consequence: never compare WER numbers computed with different normalization pipelines. A vendor benchmark that lowercases, strips punctuation, and normalizes numbers will beat an identical system scored raw by a wide margin, with zero difference in real accuracy. When you benchmark, fix one normalizer (e.g., the Whisper English normalizer) and run it on every system's output.

CER: when words aren't the right unit

For languages written without spaces between words (Chinese, Japanese, Thai), "word" error rate depends on a tokenizer, which adds its own errors. The convention is character error rate (CER): identical formula, computed over characters. CER is also standard for agglutinative or morphologically rich languages (Korean often uses syllable-level error rates) and useful as a secondary metric everywhere, since it distinguishes "recognized the word almost right" from "produced something unrelated." Expect CER values to be numerically lower than WER on the same data, which is a reason never to compare a CER to a WER directly.

Typical WER bands: what the numbers look like in 2026

There is no single "state of the art WER", only WER on a given distribution of audio. Honest reference points, as ranges:

For calibration: Whisper large-v3 reports roughly 2-3% on LibriSpeech test-clean, yet the same model typically lands in the high single digits to double digits on real-world conversational English, and much higher on conversational telephone corpora like CallHome. The gap between read and spontaneous speech is the single most underestimated factor in speech-to-text accuracy planning. Spontaneous speech brings disfluencies, overlaps, fragments, and channel noise that read-speech benchmarks never exercise (more on this in conversational vs. scripted speech data).

WER's blind spots

WER treats every word as equally important, which produces systematic blind spots:

• Semantic severity. Dropping "not" is one deletion, the same cost as dropping "um", but it inverts the sentence's meaning.
• Named entities and rare terms. Getting a drug name, person name, or account number wrong is often the only error that matters in a workflow, yet it moves WER by a fraction of a point. Entity-level error rates are a necessary supplement.
• No error weighting by frequency of harm. A 12% WER system that nails entities can be more useful than an 8% WER system that garbles them.
• Gaming via normalization, as covered above.

Newer supplements (entity WER, keyword recall, and semantic similarity scores) don't replace WER, but any production evaluation should include at least one of them.

A checklist for trustworthy WER benchmarking

Before you quote or trust a WER number, verify:

1. Same test set. Numbers from different corpora are not comparable, full stop.
2. Same normalization. One shared normalizer applied to references and every system's hypotheses; state which one.
3. Reference quality. Human transcripts have their own error rate (often 2-5% disagreement between annotators on spontaneous speech); a noisy reference puts a floor under every system's measured WER. Check transcription conventions (how the reference handles fillers, partial words, and numbers) before scoring against it.
4. Enough audio. WER on ten utterances is noise. Use hours, not minutes, and report per-condition breakdowns (channel, accent, speaker) rather than one blended number that hides regressions.
5. No test-set leakage. Public benchmarks like LibriSpeech are in many models' training data; unseen, licensed test sets are the only clean read on a foundation model's real accuracy.
6. Statistical significance for A/B claims. For close comparisons, bootstrap confidence intervals over utterances before declaring a winner on a 0.3-point WER gap.

Why your test set matters more than your metric

The most common WER mistake isn't the math. It's evaluating on audio that doesn't resemble production. A model validated at 5% WER on read speech can ship into a call-center product and deliver 25% WER on day one, because the test set never contained telephone codecs, overlapping speakers, or spontaneous disfluencies.

The fix is boring and effective: evaluate on held-out audio drawn from the same distribution as production traffic, meaning same channel, same speaking style, same demographics, same languages. Since most teams can't ethically or legally carve test sets out of customer calls, they license conversational test sets instead: real spontaneous conversations with accurate time-aligned transcripts and a clean consent chain. Our datasets, including US English, Hindi, and Arabic (MSA), are dual-channel spontaneous conversations built exactly for this, and our guide to buying speech data covers what to check before you license (transcript quality conventions matter as much for scoring as for training). The same logic applies to training data volumes; see how much audio data you need to train ASR.

Need training data?

Meaningful WER numbers require test audio that matches production: real conversations, accurate transcripts, and clear licensing. Browse our conversational speech datasets in 60 languages, or contact us to set up an evaluation set for your target market.