InternLM 开源 ArchSpace：把 LLM 架构探索变成社区可复用知识
GitHub - InternLM/archspace: Turning LLM architecture exploration into reusable knowledge for the community. 🌳 · GitHub
InternLM 推出开源实验项目 ArchSpace，将社区提交的 LLM 架构假设放入可追溯、可复现的训练与评测流程，并把成功结论、负面结果和设计取舍沉淀为共享知识。获批提案按与全开放 Olmo 3 模型流程对齐的完整训练工作流验证，覆盖 1B、3B、8B 三种规模，经历预训练、中期训练和指令微调，全流程约 6.2T tokens，训练与评测日志发布在 W&B。
正文
当前显示原文，中文全文译文尚未就绪。
InternLM; archspace
Repository navigation
NotificationsYou must be signed in to change notification settings; Fork9 (9); Star87 (87)You must be signed in to star a repository
Turning LLM architecture exploration into reusable knowledge for the community. 🌳
Repository files navigation
README; Apache-2.0 license
English | 简体中文
🌳 Turning LLM architecture exploration into reusable knowledge for the community. 🌳
ArchSpace is an open experiment for large language model (LLM) architecture innovation. We place architecture hypotheses proposed by the community into transparent, traceable, and reproducible training and evaluation workflows, then turn successful findings, negative results, and design trade-offs into shared knowledge assets.
Each generation of open foundation models brings new architectural ideas. Yet their actual gains across a full training lifecycle, their operating boundaries, and the lessons from failed attempts are rarely reusable by the broader field. ArchSpace provides a public experimental ground: from a proposal issue on GitHub through peer review, implementation, training, evaluation, and publication of conclusions.
What We Do
Open proposals: Community members can submit architecture ideas worth validating; Peer review: A review committee assesses proposals for novelty, feasibility, and experimental rigor; Full validation: Approved proposals enter a unified workflow covering pretraining, instruction tuning, and reinforcement learning; Open records: Training and evaluation logs are continuously recorded and published through platforms such as Weights & Biases (W&B); Shared knowledge: Implementations, experiment records, and conclusions are made available whether results are positive, negative, or conditional
How to Contribute
You can participate in ArchSpace by:
Submit an Issue using the architecture proposal template to discuss an architecture innovation proposal; Contributing implementations, tests, evaluations, or documentation for approved proposals; Sending pull requests with incremental implementations and experiment updates; Help reproduce results, verify implementation correctness, and discuss public experiment results
⭐ Accepted proposals may receive joint appointment or internship collaboration opportunities from the lab.
🏆 Point architecture innovations adopted by the next-generation InternLM architecture will receive open-source contribution recognition from the lab and the community.
Approved proposals are validated using a full training workflow aligned with the fully open Olmo 3 model flow:
Model sizes: 1B, 3B, and 8B; Training stages: pre-training, mid-training, and instruction tuning; Training scale: approximately 6.2T tokens in total across the complete workflow; Process records: training and evaluation logs are recorded and published on W&B; Outputs: implementations, configurations, metrics, experiment records, and conclusions, including architecture trade-offs and operating conditions where possible
We value negative results. An architecture attempt that does not deliver the expected gain is still valuable when it has been clearly and rigorously validated, because it helps the community avoid redundant trial and error.
References
Turning LLM architecture exploration into reusable knowledge for the community. 🌳