Research: how to read this section
This section summarises the papers that explain how the models in vaak work and how they should be evaluated. It is not a literature review. Each page is 350 to 600 words with the same five headings, so you can skim all of them in half an hour or read one when the tutorial sends you there.
The headings
Section titled “The headings”- In one paragraph. What the paper did, for someone who has read the tutorial.
- Key idea. The single mechanism that made it work. Usually one equation or one diagram’s worth.
- Why it matters for vaak. Which backend, benchmark or design decision in vaak traces back to this paper.
- Numbers to remember. Only figures the paper itself reports. If a figure is missing here, it is because we were not confident of it; the paper is one click away. Numbers are not re-normalised, so they are not comparable with vaak’s own benchmark table, which is labelled separately.
- Citation. Authors, venue, year, and the arXiv link (or the publisher’s link for papers that predate arXiv use in this field).
For the two 2026 Indic papers we had access to the abstract and limited parts of the text at the time of writing. Those pages say what we know and mark the rest “see paper”.
Timeline
Section titled “Timeline”| Year | Paper | What it introduced | vaak backend |
|---|---|---|---|
| 2006 | CTC | Training sequence models without frame alignments | indic-conformer, wav2vec2, the toy model |
| 2012 | RNN Transducer | Streaming model that conditions on output history | parakeet (ancestor) |
| 2015 | Listen, Attend and Spell | Attention encoder-decoder for speech | faster-whisper, hf-whisper (ancestor) |
| 2020 | Conformer | Convolution-augmented Transformer encoder | indic-conformer, parakeet |
| 2020 | wav2vec 2.0 | Self-supervised pretraining from raw audio | wav2vec2 |
| 2021 | XLS-R | wav2vec 2.0 at 128 languages and 436k hours | wav2vec2 Hindi checkpoints |
| 2022 | Whisper | 680k hours of weak supervision, multitask tokens | faster-whisper, hf-whisper |
| 2023 | FastConformer | 8x subsampling, linearly scalable attention | parakeet |
| 2023 | TDT | Transducer that predicts token durations and skips frames | parakeet |
| 2024 | IndicVoices | 7 348 h of spontaneous speech in 22 Indian languages; IndicASR / IndicConformer | indic-conformer |
| 2026 | Vaani Benchmark V1.0 | Multi-reference Hindi benchmark from 102 districts | benchmark methodology |
| 2026 | Factors affecting Indic ASR | Speaker and audio factors behind Indic WER | benchmark methodology |
Vosk is not covered by a paper here. It is a Kaldi-based hybrid system whose lineage (HMMs, WFST decoding, n-gram language models) predates all of the above. The decoding page covers the parts that still matter.
Three threads to follow
Section titled “Three threads to follow”The alignment thread. CTC (2006) removes the need for alignments by marginalising over them. The transducer (2012) keeps that and adds a prediction network. Attention (2015) replaces the explicit marginalisation with learned soft alignment. TDT (2023) makes the transducer predict how long each token lasts. Every output head in vaak is one of these.
The data thread. wav2vec 2.0 (2020) shows that unlabelled audio can replace most labelled data. XLS-R (2021) does it across languages. Whisper (2022) takes the opposite route: weakly labelled internet audio at a scale that makes self-supervision unnecessary. IndicVoices (2024) is the labelled-data answer for India.
The evaluation thread. The 2026 papers show that a single reference transcript overstates Hindi error rates and that speaker rate, word length, codecs and bit depth each move WER measurably. vaak’s choice to report CER next to WER, to normalise both sides identically, and to label every number by its source comes from this thread.
Start with CTC if you are reading in order, or with Whisper if you want to understand the default backend first.