DeepSeek-V4-Pro
The larger DeepSeek-V4 reasoning model, built around long-context attention, MoE routing, and coupled post-training and serving choices.
DeepSeek-V4-Pro is useful to study because the model page points to a whole stack, not just a bigger checkpoint. Its identity comes from how architecture, post-training, and long-context serving are combined.
At a glance
- Family
- deepseek
- Source type
- Research
- Released
- June 2026
- Authors
- DeepSeek-AI
- Modalities
- Text
- Parameter count
- 671 billion total parameters
- Context length
- 1,048,576 tokens
- Precision
- fp8, fp4
What It Is
DeepSeek-V4-Pro is the larger reasoning-oriented model in the DeepSeek-V4 release. It is a text model with a transformer-style backbone, sparse expert blocks, and an unusually large advertised context length.Inputs And Outputs
The model takes text tokens and produces text tokens. The practical difference from an ordinary chat model is not modality. It is that the stack is designed to preserve useful work over very long prompts and long reasoning traces.Architecture
The model keeps a transformer skeleton, but the paper highlights manifold-constrained hyper-connections, compressed sparse attention, heavily compressed attention, DeepSeekMoE routing, MLA-style cache compression, and supporting serving systems rather than one single signature block.Output
ProbabilitiesSoftmaxOutput
LinearTransformer architectureAttention sublayer containerSparse expert sublayer containerN×RMSNormCompressed
Attention
(CSA + HCA)RMSNormDeepSeekMoEEmbeddingRoPEInput
EmbeddingInput
TokensSoftmax to Output
ProbabilitiesOutput
Linear to SoftmaxRMSNorm to Output
LinearDeepSeekMoE to RMSNormRMSNorm to DeepSeekMoERMSNorm to RMSNormCompressed
Attention
(CSA + HCA) to RMSNormEmbedding to Compressed
Attention
(CSA + HCA)Embedding to RMSNormInput
Embedding to EmbeddingRoPE to EmbeddingInput
Tokens to Input
Embedding