Skip to content

BuzzASR

100+ monolingual speech recognizers, one per language

We fine-tune Whisper-large-v3 into a separate model for each of 102 FLEURS languages, with two recipes: plain fine-tuning (SFT) and full fine-tuning with a native tokenizer (FFT). This site is a short technical overview of the project: the two recipes, why the tokenizer matters, and what the models achieve.

Read the paper (arXiv)  ·  Get the models (Hugging Face)  ·  Code

89 / 102
languages where the best BuzzASR model beats Whisper-large-v3 zero-shot (FLEURS-test, CER)
31 / 102
lowest CER among open systems on FLEURS-test
41 / 102
lowest CER on the combined FLEURS + CommonVoice set

How to read the SOTA counts

"Lowest CER among open systems" means the best of SFT or FFT (picked per language on validation, never on test) beats Omni-1B/7B, MMS-1B, Qwen3-ASR, and Cohere on CER. We report it on FLEURS-test (31) and on the combined FLEURS + CommonVoice set (41).

The two recipes

FFT's advantage comes from its tokenizer. Whisper's tokenizer was built for English-heavy data, so it splits other languages into many small pieces. FFT trains a native tokenizer that keeps whole morphemes. That one change makes FFT more accurate, and it also makes decoding faster.

Decode latency is token count times time-per-token. Whisper-ZS and SFT share a tokenizer and emit the same number of tokens, so they tie. FFT's native tokenizer emits fewer, so it finishes first.
How FFT swaps in a native tokenizer without retraining from scratch: learn a tokenizer, warm-start its embeddings from Whisper's, then fine-tune.

Where to start

  • Technical overview

    What BuzzASR is, and how the two recipes (SFT and FFT) differ.

    Read the overview →

  • The tokenizer

    Why a native tokenizer helps, and a bug worth knowing about.

    Tokenizer →

  • Results

    How accurate the models are, and why FFT decodes faster.

    See results →

  • Models

    All 102 per-language weights are live on Hugging Face under the BuzzASR org.

    Get the models →

All 102 models are available now on Hugging Face under the BuzzASR organization — one model per language. See Models.

Acknowledgments

Developed in collaboration with EleutherAI, whose compute and support made these training runs possible.

EleutherAI logo

Citation

Paper: arXiv:2609.09554 · Findings of EMNLP 2026.

@misc{buzzasr2026,
  title         = {BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models},
  author        = {Shivam Singh and Aditya Yadavalli and Catherine Arnett and Alex Warstadt},
  year          = {2026},
  eprint        = {2609.09554},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  note          = {Findings of the Association for Computational Linguistics: EMNLP 2026},
  url           = {https://arxiv.org/abs/2609.09554}
}