7. Transducers: RNN-T and TDT

CTC streams but ignores output history. Attention decoders use history but need the whole audio. The transducer, proposed by Graves in 2012, does both. It is the architecture behind most on-device dictation systems and behind NVIDIA’s Parakeet models, which vaak exposes as parakeet:<hf-id>.

Three networks

audio frames  ->  encoder (transcription network)  ->  f_t   (one vector per frame)
tokens so far ->  prediction network               ->  g_u   (one vector per emitted token)
joint network: h_{t,u} = tanh(W_f f_t + W_g g_u + b);  P(k | t, u) = softmax(W_o h_{t,u})

The encoder is any of the encoders from page 4. The prediction network is a small autoregressive model over tokens, originally an LSTM and in modern systems often a single LSTM layer or even a stateless embedding of the last one or two tokens. The joint network combines one encoder frame and one prediction state and outputs a distribution over the vocabulary plus blank.

The key difference from CTC: the output at frame t depends on u, how many tokens have been emitted so far, and on what they were. The conditional independence assumption is gone.

The lattice

Think of a grid with time t on one axis and token count u on the other. From cell (t, u) you can either emit blank, which moves to (t+1, u) (advance one frame, no output), or emit token k, which moves to (t, u+1) (output a token, stay on the same frame). Any path from (0, 0) to (T, U) is a valid alignment of the transcript to the audio. The loss is the negative log of the sum over all such paths, computed with a forward-backward algorithm over the (T x U) grid, just as CTC does over (T x 2U+1).

The cost is the joint network. It is evaluated at every (t, u) cell during training: a (batch, T, U, vocab) tensor. With T = 250, U = 60 and a vocabulary of 1 024 that is 15 M logits per utterance. This is why RNN-T training is memory hungry and why implementations fuse the joint and the loss (NeMo’s fused_batch_size, or the warp-rnnt kernels).

Why it streams

Nothing in the three networks looks ahead. If the encoder is causal (or chunked with a fixed lookahead), then at time t the model has everything it needs to decide whether to emit a token or a blank. The decoder loop is:

def transducer_greedy(enc_out, pred_net, joint, blank=0, max_symbols=5):
    tokens = []
    state = pred_net.initial_state()
    g = pred_net.step(None, state)                # start symbol
    for t in range(enc_out.shape[0]):
        for _ in range(max_symbols):              # several tokens may share a frame
            k = joint(enc_out[t], g).argmax()
            if k == blank:
                break
            tokens.append(int(k))
            g, state = pred_net.step(k, state)
    return tokens

This is greedy decoding: at each frame emit tokens until the joint says blank, then move on. It runs as audio arrives and the partial transcript only grows. Contrast with the attention decoder, which cannot start until the encoder has seen the end of the window.

Token-and-Duration Transducer (TDT)

The RNN-T loop visits every frame and asks “token or blank”. Most answers are blank. With 80 ms frames (FastConformer) and typical English speech, around two thirds of frames emit nothing. TDT changes the joint network to output two distributions: one over tokens, and one over a small set of durations such as {0, 1, 2, 3, 4} frames. When the model emits a token it also says how many frames that token covers, and the decoder jumps ahead by that many frames. When it emits blank, it also predicts how many frames to skip.

Consequences:

• Fewer joint evaluations at inference. The TDT paper reports up to 2.82x faster inference for ASR with equal or better accuracy.
• The duration output gives a crude timestamp for every token for free.
• Training still sums over all paths, now over a lattice with duration-labelled edges. NeMo implements the loss.

Parakeet

NVIDIA’s Parakeet family pairs a FastConformer encoder (8x subsampling, page 4) with either a CTC head or a TDT head, trained on tens of thousands of hours of English. The parakeet-tdt-0.6b-v2 checkpoint in particular combines:

• 600 M parameter FastConformer encoder.
• TDT decoder with durations 0..4.
• Greedy decoding with frame skipping, so a long file is decoded in one pass with no 30 s windowing.
• Segment and word timestamps from the duration output.

In vaak:

from vaak import get_backend, load_audio

audio = load_audio("meeting.wav")
backend = get_backend("parakeet:nvidia/parakeet-tdt-0.6b-v2")
result = backend.transcribe(audio)      # English only; language is ignored
print(result.rtf)                        # typically well below Whisper on the same GPU

Parakeet is English only. There is no Hindi Parakeet as of this writing. IndicConformer (indic-conformer:hi) is the closest equivalent for Indic languages: a Conformer encoder with a CTC head and, in the newer multilingual checkpoints, an RNN-T head as well.

Choosing between the three families

	CTC	Transducer	Attention decoder
Uses output history	no	yes	yes
Streams	yes	yes	no (window-based)
Decoding	parallel	sequential, cheap	sequential, expensive
Implicit language model	none	weak	strong
Hallucination risk	very low	low	real
Timestamps	free, coarse	free, coarse	from attention, needs work
Training memory	low	high (T x U joint)	moderate

For a self-hosted server that must never invent text (medical, legal, call analytics), transducer or CTC is the safer choice. For maximum accuracy on clean long-form speech with punctuation, Whisper still wins in many languages. vaak’s benchmarks put numbers on this for Hindi and English.

Next: Tokenization and multilingual text.