DeepSeekMoE
The MoE block style emphasized in DeepSeek-V4, where expert routing is a central architecture and serving choice rather than a small add-on.
Dense feed-forward blocks waste compute when only a few specialized transformations are useful for one token. DeepSeekMoE keeps sparse expert capacity, but the V4 framing makes routing and parallel serving part of the module story instead of an afterthought.
At a glance
Optimizes
- Parameter Capacity
- Compute Efficiency
- Routing Specialization
Example models
What It Is
DeepSeekMoE is the DeepSeek-specific MoE block family described in the V4 report. It still follows the broad MoE idea of routing tokens to a subset of experts, but it is presented as a mature backbone choice rather than an experimental add-on.Why It Exists
The module aims to raise parameter capacity while keeping per-token active compute much lower than a dense feed-forward block of the same total size.How It Works
A router scores the token state, selects experts, and combines the chosen expert outputs. The V4 story then adds serving machinery so those expert choices do not turn into a deployment bottleneck.Token stateRouterSelected expertsToken state to RouterRouter to Selected experts