Speech recognition scores look excellent until the speaker does not sound like the test data. Hugging Face's Open ASR Leaderboard, one of the most watched public rankings of transcription models, has added two new private test sets that target exactly that gap: conversational English spoken with Egyptian and Nigerian accents, supplied by the data company Liva AI.
The new recordings are short. The Egyptian set runs to 1.34 hours and the Nigerian set to 1.55 hours, both spontaneous conversation transcribed with punctuation, capitalisation and disfluencies such as false starts left in. They join 11 private splits added a few months ago from Appen and DataOceanAI, covering scripted and conversational speech in five accents, and a later private set of Indian-accented English from Voice Arena.
The private part is the point. Public benchmarks leak into training data over time, and models can be tuned to them, a practice the leaderboard's maintainers call benchmaxxing. Held-out audio that developers cannot download is harder to game, so a model's score on it says more about how it will handle real callers. With the new accents, the leaderboard's private tab now reports two comparisons: US speakers against everyone else, covering the UK, Australia, Canada, India, Nigeria and Egypt, and the inner circle of native-English countries against India, Nigeria and Egypt, using the linguist Braj Kachru's three-circles model of English.
For anyone choosing a transcription model from our reviews, such as Qwen3-ASR, Whisper Large v3 Turbo or Parakeet, this is a useful second opinion. A model that leads on clean American read speech can fall back sharply on fast, accented conversation, and that is the audio many call centres, clinics and courts actually deal with.
The limits are plain. Under three hours of new audio, from one provider, is a small sample; small score gaps between models on these splits should not be treated as meaningful. Two accents do not represent the range of English spoken across Africa and the Middle East, and the maintainers say they will split the outer and expanding circles further only as more accents arrive. Private data also cuts both ways: it resists gaming, but outsiders cannot inspect it to check its quality or balance. Liva AI collects its data through consumer products, which the post presents as natural speech, but no details on consent or speaker counts are given.