9B (dense)
Apache 2.0
First 9B model trained with Sparse-Linear Attention (SALA) hybrid architecture; 75% linear attention (Lightning Attention) for global efficiency + 25% sparse attention (InfLLM-v2) for high-fidelity long context modeling; 1M+ token context via HyPE (Hybrid Positional Embedding); 3.5x inference speed and significantly lower KV-cache overhead vs dense baselines; first model to achieve million-token context inference on consumer GPUs (RTX 5090); maintains 81.6 score at 2048K context despite training on only 520K tokens; NoPE configuration in sparse attention layers for length extrapolation without YaRN; average score 76.53 across standard benchmarks; outperforms Qwen3-8B and Falcon-H1R-7B; strong in knowledge, code, and math