← Back to models

Qwen3 (original April 2025 release)

Alibaba Cloud / Qwen Team (China) April 28, 2025

Parameters

Dense: 0.6B, 1.7B, 4.0B, 8.2B, 14.8B, 32.8B total; MoE: 30.5B total / 3.3B active and 235B total / 22B active (including embeddings)

Context Window

32,768 tokens native for all sizes; 131,072 with explicitly enabled YaRN for post-trained 4B, 8B, 14B, 32B, 30B-A3B and 235B-A22B; 0.6B/1.7B and released Base cards specify 32,768; post-trained configs default to 40,960, while 8B-Base config is 32,768

License

Apache License 2.0 (all 14 confirmed original post-trained/Base checkpoints)

Key Features

Original text-only, decoder-only Qwen3 family: six dense sizes (0.6B/1.7B/4B/8B/14B/32B) and two mixture-of-experts models (30B-A3B/235B-A22B). All eight have post-trained hybrid reasoning checkpoints without an -Instruct suffix; publicly downloadable Base counterparts are confirmed only for 0.6B, 1.7B, 4B, 8B, 14B and 30B-A3B, not 32B or 235B-A22B. Uses grouped-query attention, QK normalization, bias-free QKV projections, SwiGLU, RoPE and pre-RMSNorm; MoE layers have 128 routed experts, eight active per token and no shared experts. Pre-trained on approximately 36 trillion tokens spanning 119 languages and dialects, including web/document text and synthetic mathematics/code; three stages move from general 4K training through reasoning-focused data to 32K long-context training. Post-training combines long-chain-of-thought cold start, reasoning RL, thinking/non-thinking fusion and general RL; smaller models additionally use strong-to-weak distillation. Hybrid checkpoints default to thinking, with enable_thinking=False as a hard chat-template switch and /think or /no_think as soft turn-level controls; inference-time thinking budgets trade reasoning quality against latency. The Qwen3 Technical Report evaluates the original flagship in thinking mode at 85.7 on AIME 2024, 81.5 on AIME 2025, 70.7 on LiveCodeBench v5 and 70.8 on Berkeley Function Calling Leaderboard v3, with a CodeForces rating of 2,056 (developer evaluations, not interchangeable with non-thinking scores). Official cards warn that greedy thinking-mode decoding can degrade performance and cause endless repetitions, excessive repetition penalties can cause language mixing, and static YaRN can hurt shorter-text performance. Native 32K, configured 40,960 and YaRN-extended 128K are distinct; the launch blog lists 4B at 32K while its contemporaneous model card explicitly validates 128K with YaRN. Excludes later 2507 Instruct/Thinking refreshes, Qwen3-Coder, Next, VL and Omni. April 28 is provisional availability-by in UTC, supported by the official public-weight release README published that evening; the publisher labels the launch April 29, consistent with China time, and exact first public unlock remains unverified.

Paper / Source

https://arxiv.org/abs/2505.09388

Official Links