Qwen3.6-35B-A3B
The sparse-expert Qwen 3.6 open-weights model with 35 billion total parameters, about 3 billion active per token, multimodal inputs, and a 262,144-token native context window.
Qwen3.6-35B-A3B is the mixture-of-experts member of the first Qwen 3.6 open-weights release. It keeps a large total parameter budget but routes each token through a smaller active expert subset, which is the main practical contrast with the dense 27B operating point in the same family.
At a glance
- Family
- qwen
- Source type
- Open weights
- Released
- April 2026
- Authors
- Qwen Team
- Modalities
- Text, Image, Video
- Parameter count
- 35 billion total parameters
- Context length
- 262,144 tokens
What It Is
Qwen3.6-35B-A3B is a sparse-expert open-weights model from the Qwen Team and the MoE checkpoint in the first published Qwen 3.6 release. Public release materials describe it as a 35-billion-parameter mixture-of-experts causal language model with a vision encoder, released alongside the dense Qwen3.6-27B variant as a stability-focused coding and agentic-workflow upgrade.Inputs And Outputs
The model accepts text, image, and video inputs and produces text outputs. The Hugging Face model card includes Video Input examples alongside text and image prompts, and lists a 262,144-token native context length with longer extension claims noted separately in the same source. This page keeps the native context figure because it is the conservative operating point repeated across official model metadata.Architecture
As a mixture-of-experts operating point, Qwen3.6-35B-A3B stores many expert weights but routes each token through a smaller active subset rather than activating the full 35-billion-parameter stack on every step. Official model materials describe a deep decoder with repeated attention and sparse expert blocks, rotary position encoding, and RMS normalization. The public Hugging Face model card also names hybrid attention paths inside the stack; this page's diagram stays at the familiar decoder level—attention, normalization, and a sparse expert routing block—so it teaches the MoE flow without turning unexpanded block names into extra claims.Output
ProbabilitiesSoftmaxOutput
LinearTransformer architectureAttention sublayer containerSparse expert sublayer containerN×RMSNormDecoder
AttentionRMSNormExpert
RoutingEmbeddingRoPEInput
EmbeddingInput
TokensSoftmax to Output
ProbabilitiesOutput
Linear to SoftmaxRMSNorm to Output
LinearExpert
Routing to RMSNormRMSNorm to Expert
RoutingRMSNorm to RMSNormDecoder
Attention to RMSNormEmbedding to Decoder
AttentionEmbedding to RMSNormInput
Embedding to EmbeddingRoPE to EmbeddingInput
Tokens to Input
Embedding
Important Modules
The registry-backed modules below are the smallest set worth opening first when you want to connect this MoE page to the site's module reference. Mixture-of-experts routing is the defining feed-forward behavior here, while rotary position encoding and RMS normalization anchor the high-level stack shown above.Training
Official sources describe both pre-training and post-training for the released checkpoint. The Qwen 3.6 release blog emphasizes agentic coding workflows and preserving reasoning context across turns, but this page does not restate training-corpus or recipe details beyond what those primary sources publish.Training regimes
No training regimes listed yet.
Linked papers
No linked paper pages listed yet.