72B (dense)
Apache 2.0
Large open base language model trained from scratch on 1.1T English tokens with a permissionless collaborative-training setup; model card describes 20+ globally distributed participants coordinated over Bittensor and SparseLoCo communication-efficient optimization; LLaMA-style architecture with 80 layers, 64 attention heads (8 KV heads), and 262,144 vocabulary; companion instruction-tuned checkpoint (Covenant-72B-Chat) is released alongside the base model family.