Audio

Mel spectrogram front-end

Converts raw audio into a time-frequency image on a perceptual (mel) scale. The standard input for speech models.

Mel spectrogram: a picture of soundaudio becomes a time-frequency image, with finer bands where hearing is finerraw audiomel bands: fine low, coarse high

Raw waveforms are too long and too low-level for transformers, so audio models first compute a spectrogram: short-time Fourier transforms binned onto the mel scale, which spaces frequencies the way human hearing does. The result is effectively an image (time × mel bins) that convolutional or transformer encoders can process like any other 2-D input.

Adoption over time

Share of new models that have included a mel spectrogram front-end over time.

1%202220232024202520261%20222023202420252026

See it in real models

Open any of these on hfviewer to find this block in the interactive architecture graph.

coqui/XTTS-v2 architecture graphcoqui/XTTS-v2text-to-speech · ↓ 8.7M · ♡ 4kOpen in visualizer laion/clap-htsat-fused architecture graphlaion/clap-htsat-fusedaudio-classification · ↓ 7.3M · ♡ 125Open in visualizer charactr/vocos-mel-24khz architecture graphcharactr/vocos-mel-24khz↓ 3.1M · ♡ 42Open in visualizer Qwen/Qwen3-TTS-12Hz-1.7B-Base architecture graphQwen/Qwen3-TTS-12Hz-1.7B-Base↓ 3.0M · ♡ 484Open in visualizer Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice architecture graphQwen/Qwen3-TTS-12Hz-1.7B-CustomVoicetext-to-speech · ↓ 2.2M · ♡ 2kOpen in visualizer ResembleAI/chatterbox architecture graphResembleAI/chatterboxtext-to-speech · ↓ 2.1M · ♡ 2kOpen in visualizer Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice architecture graphQwen/Qwen3-TTS-12Hz-0.6B-CustomVoicetext-to-speech · ↓ 1.5M · ♡ 175Open in visualizer CohereLabs/cohere-transcribe-03-2026 architecture graphCohereLabs/cohere-transcribe-03-2026automatic-speech-recognition · ↓ 841k · ♡ 1kOpen in visualizer laion/clap-htsat-unfused architecture graphlaion/clap-htsat-unfusedfeature-extraction · ↓ 714k · ♡ 79Open in visualizer nvidia/parakeet-tdt-0.6b-v3 architecture graphnvidia/parakeet-tdt-0.6b-v3automatic-speech-recognition · ↓ 494k · ♡ 1kOpen in visualizer Qwen/Qwen3-TTS-12Hz-0.6B-Base architecture graphQwen/Qwen3-TTS-12Hz-0.6B-Basetext-to-speech · ↓ 494k · ♡ 276Open in visualizer Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign architecture graphQwen/Qwen3-TTS-12Hz-1.7B-VoiceDesigntext-to-speech · ↓ 429k · ♡ 387Open in visualizer

Browse all 46 models with this in the catalog →

Related concepts

Convolutional subsampling Conformer block RVQ codebooks

hfviewer renders the full architecture of 3,500+ Hugging Face models as interactive graphs. Hover any block to see what it does, with this model’s real numbers.

Browse all model graphs →