Objective transcription benchmarks.

01

Golden set

Curated test audio with verified human reference transcripts across languages, accents, and noise levels. Some public read-speech corpora (LibriSpeech, FLEURS, Common Voice) may appear in model training data, which can flatter the models trained on them; we counter this by also scoring across domain audio — legal, financial, and conversational — that is unlikely to be in standard training sets.

02

Same audio, vendor-recommended settings

Every model runs the identical test set — no cherry-picked clips. Our rule for calls: each model uses its provider's recommended settings — the configuration real users would run — not a lowest-common-denominator call that penalizes a model for its vendor's best practices (e.g. OpenAI's gpt-4o models get server-side VAD chunking).

03

Event-driven & automated

Benchmarks re-run automatically on every model or pricing change, with no human bias. Every provider gets the same test audio.

04

Scoring

Overall score is a weighted composite: 50% accuracy (WER), 30% speed, 20% cost efficiency. Category leaders show a confidence interval; where clips are too few to separate models, we say so rather than crown a winner.