25.2B total / 3.8B active (MoE, marketed as A4B)
Up to 256K tokens
Apache-2.0
Official open-weight Gemma variant that replaces standard token-by-token decoding with discrete text diffusion for exceptionally fast multimodal generation. Google DeepMind created DiffusionGemma by fine-tuning the Gemma 4 Mixture-of-Experts model with 25.2B total and 3.8B active parameters, retaining its text and image understanding, thinking mode, and long-context capabilities. The model uses block-autoregressive multi-canvas sampling: prompt context is processed into a KV cache, after which a decoder with bidirectional attention iteratively denoises 256-token generation canvases in parallel before appending them to the output. Its compute-efficient two-stage training pipeline used fewer than 10% of the starting autoregressive model's training-token budget. The first stage applies supervised fine-tuning to teach bidirectional denoising, while the second combines reinforcement learning with sampler distillation to jointly improve generation quality and inference efficiency. The technical report says DiffusionGemma generates approximately 20 tokens per forward pass and averages roughly 1,500 output tokens per second on a single NVIDIA H100, establishing a new capability-versus-speed Pareto frontier against autoregressive models using state-of-the-art speculative decoding. Despite its diffusion fine-tuning, the model can still perform conventional autoregressive generation with only minor reported performance degradation, offering a path toward hybrid diffusion-autoregressive decoding. The instruction-tuned checkpoint also supports native system prompts, optional thinking triggered with the `<|think|>` token, text-to-text and image-to-text tasks, and contexts up to 256K tokens.