Convolutional subsampling
Strided convolutions shrink the audio sequence (often 4–8×) before the expensive transformer layers run.
Spectrogram frames arrive at ~100 per second. Far denser than needed. A small stack of strided convolutions downsamples time resolution several-fold and lifts features into the model width, cutting transformer cost quadratically. Practically every ASR encoder (Conformer, Whisper, Parakeet) starts this way.
Adoption over time
Share of new models that have included convolutional subsampling over time.
See it in real models
Open any of these on hfviewer to find this block in the interactive architecture graph.
nvidia/parakeet-ctc-1.1bautomatic-speech-recognition · ↓ 1.9M · ♡ 58Open in visualizer
nvidia/nemotron-3.5-asr-streaming-0.6bautomatic-speech-recognition · ↓ 1.3M · ♡ 1kOpen in visualizer
CohereLabs/cohere-transcribe-03-2026automatic-speech-recognition · ↓ 841k · ♡ 1kOpen in visualizer
nvidia/parakeet-tdt-0.6b-v3automatic-speech-recognition · ↓ 494k · ♡ 1kOpen in visualizer
google/gemma-3n-E2B-itimage-text-to-text · ↓ 391k · ♡ 319Open in visualizer
ai-sage/GigaAM-v3automatic-speech-recognition · ↓ 361k · ♡ 140Open in visualizer
internlm/Intern-S1-Proimage-text-to-text · ↓ 318k · ♡ 279Open in visualizer
google/medasrautomatic-speech-recognition · ↓ 231k · ♡ 353Open in visualizer
nvidia/nemotron-speech-streaming-en-0.6bautomatic-speech-recognition · ↓ 131k · ♡ 604Open in visualizer
nvidia/canary-qwen-2.5bautomatic-speech-recognition · ↓ 35k · ♡ 453Open in visualizer
nvidia/omni-embed-nemotron-3bsentence-similarity · ↓ 19k · ♡ 128Open in visualizer
internlm/Intern-S2-Previewimage-text-to-text · ↓ 16k · ♡ 118Open in visualizer
Browse all 24 models with this in the catalog →
Related concepts
hfviewer renders the full architecture of 3,500+ Hugging Face models as interactive graphs. Hover any block to see what it does, with this model’s real numbers.
Browse all model graphs →