~18B total / ~2B active (MoE)
5,000,000 tokens (experimental config)
MIT
Official open-weight early preview checkpoint in SILX AI's Quasar Foundation Model series, released as the first public architecture-scale Quasar model for research, decentralized distillation, and Bittensor SN24 training rather than as a finished production release. The model card describes Quasar-Preview as a Quasar Long Hybrid Transformer with roughly 18B total parameters and an approximately 2B active sparse Mixture-of-Experts path, built around hybrid Quasar, Raven, and GLA branches with GQA compatibility attention, Safe NoPE / DrOPE-style long-context staging, and a Loop Transformer scaffold that is present but configured conservatively in this checkpoint. Published technical details list 20 layers, 256 experts with 8 experts active per token, 16 attention heads, 4 KV heads, 157,184 vocabulary size, 64 Raven slots with top-32 routing, and an exposed maximum context configuration of 5,000,000 tokens. Release notes emphasize that the checkpoint is still immature: trained on more than 1T but less than 1.5T tokens overall, with less than 1B tokens of long-context extension training so far, loop execution set to a single pass by default, and Engram memory code included but not active by default. The card also provides an early benchmark snapshot rather than final claims, including reported scores of 68.4 on MMLU, 33.2 on MMLU-Pro, 25.6 on GPQA, 63.0 on ARC Challenge, 80.1 on ARC Easy, 81.9 on PIQA, 74.0 on HellaSwag, 47.0 on OpenBookQA, and 71.4 on MATH-500.