Understand this story
Reflection announces a $10,000 Hub credit for the Beam project. The news was released on October 16, 2026. At the same time, Hugging Face released the Beam model: a 501B-parameter sparse MoE open-source model aimed at long-context reasoning and agent tasks with a 1M context, activating only about 23B parameters per token. The model uses 52 layers of hybrid attention, of which 43 layers use a 512-token sliding window and 9 layers use global attention; the KV cache grows by 36 KiB per token, about one-sixth that of a full-attention scheme. Official checkpoints are provided in BF16, FP8, and NVFP4; NVFP4 can compress the weights from about 1 TB to about 280 GB, and day-0 support is already available in transformers, vLLM, and SGLang. There are currently no subsequent developments.
AI-generated from public reports
Latest development · Report
Hugging Face Releases Beam: A 501B Open Source Model Supporting 1M Context and Self-HostingDevelopments
1 confirmed developments
- Source:Hugging Face
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 16
Hugging FaceFirst-partySelected
Hugging Face Releases Beam: A 501B Open Source Model Supporting 1M Context and Self-Hosting
Hugging Face has released Beam, a 501B-parameter sparse MoE open source model aimed at long-text reasoning and agent tasks with 1M context, activating only about 23B parameters per token. The model uses 52 layers of hybrid attention, with 43 layers using a 512-token sliding window and 9 layers using global attention; the KV cache grows by 36 KiB per token, about one-sixth that of a full-attention approach. The official release provides three checkpoints—BF16, FP8, and NVFP4; NVFP4 can compress the weights from about 1 TB to about 280 GB, and day-0 support is already available in transformers, vLLM, and SGLang.