Mixture of experts

Grouped expert routing

Experts are organized into groups; the router first picks groups, then experts inside them. Friendlier to multi-GPU serving.

First page of DeepSeek-V3 Technical ReportIntroduced inDec 2024DeepSeek-V3 Technical ReportDeepSeek-AI · arXiv 2412.19437 ↗
Grouped routing: pick a group, then expertsgroups map to devices · each token’s experts stay on a couple of GPUstokenGPU 0GPU 11. pick the group   2. pick experts inside it

With hundreds of experts spread across devices, letting a token pick any k experts scatters traffic everywhere. Grouped routing first selects the best expert groups (which map to devices), then the best experts within those groups. DeepSeek introduced this ‘node-limited’ routing to cap cross-device communication; you will see it in graphs as a two-stage top-k.

Adoption over time

Share of new models that have included grouped expert routing over time.

3%202220232024202520263%20222023202420252026

See it in real models

Open any of these on hfviewer to find this block in the interactive architecture graph.

nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 architecture graphnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4text-generation · ↓ 2.3M · ♡ 420Open in visualizer nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 architecture graphnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16text-generation · ↓ 956k · ♡ 811Open in visualizer nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 architecture graphnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16text-generation · ↓ 865k · ♡ 417Open in visualizer nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4 architecture graphnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4text-generation · ↓ 628k · ♡ 174Open in visualizer nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 architecture graphnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8text-generation · ↓ 624k · ♡ 357Open in visualizer nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 architecture graphnvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16text-generation · ↓ 448k · ♡ 324Open in visualizer nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4 architecture graphnvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4text-generation · ↓ 431k · ♡ 128Open in visualizer nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 architecture graphnvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4text-generation · ↓ 327k · ♡ 301Open in visualizer internlm/Intern-S1-Pro architecture graphinternlm/Intern-S1-Proimage-text-to-text · ↓ 318k · ♡ 279Open in visualizer nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 architecture graphnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8text-generation · ↓ 189k · ♡ 273Open in visualizer inclusionAI/LLaDA2.0-mini architecture graphinclusionAI/LLaDA2.0-minitext-generation · ↓ 150k · ♡ 70Open in visualizer nvidia/Nemotron-Cascade-2-30B-A3B architecture graphnvidia/Nemotron-Cascade-2-30B-A3Btext-generation · ↓ 49k · ♡ 523Open in visualizer

Browse all 51 models with this in the catalog →

Related concepts

MoE router Shared expert MoE experts

hfviewer renders the full architecture of 3,500+ Hugging Face models as interactive graphs. Hover any block to see what it does, with this model’s real numbers.

Browse all model graphs →