Qwen3-0.6B
The smallest dense Qwen3 open-weight checkpoint: a 0.6-billion-parameter causal language model with 32,768-token context, Apache 2.0 licensing, and post-training for reasoning and instruction following.
Qwen3-0.6B is the smallest dense open-weight checkpoint in the Qwen3 family. Official sources describe it as a 0.6-billion-parameter causal language model released under Apache 2.0, with grouped-query attention, a 32,768-token context window, and post-training that adds reasoning and instruction-following behavior on top of the pre-trained Qwen3-0.6B-Base checkpoint.
At a glance
- Family
- qwen
- Source type
- Open weights
- Released
- April 2025
- Authors
- Qwen Team
- Modalities
- Text
- Parameter count
- 0.6 billion parameters
- Context length
- 32,768 tokens
What It Is
Qwen3-0.6B is a dense open-weights causal language model from the Qwen Team and the smallest dense checkpoint in the Qwen3 open-weight release. The Qwen3 release blog lists it among six dense Qwen3 checkpoints open-weighted under Apache 2.0, and the Hugging Face model card describes it as a 0.6-billion-parameter model with both pretraining and post-training stages. This page treats the post-trained `Qwen/Qwen3-0.6B` checkpoint as the canonical reader route; the pre-trained-only `Qwen/Qwen3-0.6B-Base` checkpoint is cited separately for comparison. For larger Qwen family operating points, the published Qwen 3.6 model pages cover the dense Qwen3.6-27B and sparse-expert Qwen3.6-35B-A3B checkpoints.Inputs And Outputs
Qwen3-0.6B accepts text tokens and produces text tokens. The Hugging Face model card lists a 32,768-token context length and describes hybrid thinking and non-thinking modes controlled through the chat template. This page keeps the published context figure and text-only modality because those are the conservative operating points repeated across the primary Qwen-controlled sources.Architecture
Official Qwen3 materials describe Qwen3-0.6B as a dense decoder-only stack rather than a mixture-of-experts model. The Qwen3 release blog table lists 28 layers, 16 query and 8 key-value attention heads for grouped-query attention, tied embeddings, and a 32K context length for this size class. The diagram below shows the familiar dense decoder flow—token input, embeddings, repeated attention and feed-forward blocks, and output logits—without implying sparse expert routing.Output
ProbabilitiesSoftmaxOutput
LinearTransformer architectureAttention sublayer containerDense feed-forward sublayer containerN×RMSNormGrouped-Query
AttentionRMSNormDense
Feed
ForwardEmbeddingRoPEInput
EmbeddingInput
TokensSoftmax to Output
ProbabilitiesOutput
Linear to SoftmaxRMSNorm to Output
LinearDense
Feed
Forward to RMSNormRMSNorm to Dense
Feed
ForwardRMSNorm to RMSNormGrouped-Query
Attention to RMSNormEmbedding to Grouped-Query
AttentionEmbedding to RMSNormInput
Embedding to EmbeddingRoPE to EmbeddingInput
Tokens to Input
Embedding
Important Modules
The registry-backed modules below are the smallest set worth opening first when connecting this page to the site's module reference. Grouped-query attention, rotary position encoding, and RMS normalization are named directly in the public Qwen3 model materials.Training
The Hugging Face model card for `Qwen/Qwen3-0.6B` lists the training stage as pretraining and post-training, while the `Qwen/Qwen3-0.6B-Base` card lists pretraining only. The Qwen3 release blog describes the post-trained dense checkpoints as receiving instruction tuning and reinforcement-learning stages after the shared Qwen3 pretraining pipeline, which is how Qwen3 adds reasoning and instruction-following behavior without making benchmark ranking the point of this page. This page cites both checkpoints so readers can separate the base weights from the post-trained operating point without treating the base checkpoint as the canonical route.Training regimes
No training regimes listed yet.
Linked papers
No linked paper pages listed yet.