Factors affecting ASR performance in Indic languages (2026)

This summary is based on the paper’s abstract and the parts of the text we could access at the time of writing. Exact figures appear in the paper’s plots, which we have not reproduced; where we are not sure, we say “see paper”.

In one paragraph

Basu and colleagues at IISc take several open ASR systems, run them zero-shot (no fine-tuning) on the test splits of public Indian-language datasets, and then ask what explains the errors. They look at linguistic and speaker-level factors across five languages (Hindi, Bengali, Kannada, Telugu, Marathi): average word length, speaking rate in words per minute, and utterance duration. For Hindi they add audio-level experiments, re-encoding the same test audio through telephone codecs, reducing bit depth, resampling with classical and neural methods, and adding white and natural background noise at a range of signal-to-noise ratios. The result is a map of which conditions a deployment should worry about, and which it can ignore.

Key idea

Treat WER as a function of measurable conditions rather than a single number per model. Findings reported in the paper, qualitatively:

Word length: WER is lowest for short-to-medium words and rises for long words, consistent with the tokenization cost of long Devanagari words.
Speaking rate: the effect is non-monotonic and differs by language; for Hindi, WER falls as speaking rate increases, while other languages degrade at high rates.
Utterance duration: very short utterances have high WER; long ones degrade gradually.
Bit depth: stable down to 10 to 12 bits, sharply worse at 8 bits and below.
Telephone codecs: GSM (2G) consistently hurts; narrowband and wideband AMR (3G, 4G) stay close to the original 16 kHz result; Opus (5G) causes only marginal degradation.
Resampling: classical upsampling (linear, soxr) causes moderate loss; neural restoration (VoiceFixer, AudioSR) makes WER worse, not better.
Noise: white and natural background noise raise WER at low SNR and recover as SNR rises; under background speech, Whisper-based models were more robust than Conformer-based ones.

Why it matters for vaak

Several of vaak’s practical recommendations come straight from this list. The audio page tells you to keep PCM16 and avoid “enhancement” before recognition; this paper is the evidence that 16-bit is more than enough, that 8-bit telephony is a real loss, and that neural audio restoration should not be in the pipeline. If a client must compress audio before it reaches the streaming endpoint, the finding that Opus is nearly harmless while GSM is not is the relevant guidance. The models tested overlap with vaak’s backends (IndicConformer, Whisper large-v3), so the robustness ordering matters when choosing between indic-conformer:hi and faster-whisper:large-v3 for noisy call audio. The datasets used include three that vaak benchmarks on (Kathbath, Common Voice, FLEURS), so the paper’s per-dataset observations can be compared with vaak bench output, remembering that its numbers are published rather than reproduced.

Numbers to remember

The abstract and accessible text state the directions of the effects but not the WER values, SNR levels or exact bit-depth thresholds, which are in the figures. See paper for those. Facts we are confident of:

Five languages: Hindi, Bengali, Kannada, Telugu, Marathi.
Audio-level experiments on Hindi only, using the FLEURS and Kathbath Hindi test sets, with Indic-Conformer-hi and a Vaani-fine-tuned Whisper large as the two models.
Models in the speaker-level study include Indic Conformer, data2vec-aqc, the Vakyansh toolkit, Vaani Whisper, Voxtral Mini, Shrutam-HindiASR-1.0 and OpenAI Whisper large-v3.

Citation

Agneedh Basu, Pavan Kumar J, Pranav Bhat, Sujith Pulikodan, Visruth Sanka, Nihar Desai, Prasanta Kumar Ghosh. “Factors affecting ASR performance: A study using state of the art ASR models in Indic Languages.” arXiv:2606.09335, June 2026. https://arxiv.org/abs/2606.09335