FP4 Quantization-Aware Training
A DeepSeek-V4 training step that prepares the model for very low precision by exposing it to quantization effects during training.
A model that was never trained to survive very low precision often breaks when it is compressed too far. FP4 quantization-aware training bakes that low-precision pressure into training so the deployed model loses less of itself when it finally runs at FP4-like constraints.
At a glance
What It Is
This is a quantization-aware training step aimed at extremely low precision. Instead of treating quantization as a last-minute export trick, the model sees precision loss during training.Why It Exists
It targets low-bit deployment quality, memory savings, and practical serving cost.How It Works
The training loop injects low-precision behavior into the optimization path so the model learns weights and activations that remain more stable after compression.FP4 Quantization-Aware Training flow
Float modelFP4-aware training stepLow-precision deploymentFloat model to FP4-aware training stepFP4-aware training step to Low-precision deployment
Request and weight flow