智源 FlagEval：覆盖 800+ 模型的大模型评测系统
GitHub - FlagOpen/FlagEval · GitHub
智源（BAAI）2023 年推出的 FlagEval 大模型评测系统已覆盖全球 800 多个开源与闭源模型，包含推理、数学、任务求解等 40 多个能力维度、五大任务和四类指标。FlagEval Report 已更新至第 2 期（2024-12-30），并在 FlagScale 上开展 nightly 持续基准测试，附带自动 PR 徽章与回归告警。
正文
当前显示原文，中文全文译文尚未就绪。
FlagOpen; FlagEval
Repository navigation
NotificationsYou must be signed in to change notification settings; Fork3 (3); Star13 (13)You must be signed in to star a repository
Repository files navigation
README
FlagEval, launched by BAAI in 2023, is a comprehensive large model evaluation system that encompasses over 800 open-source and closed-source models from around the globe. It features more than 40 capability dimensions, including reasoning, mathematical skills, and task-solving abilities, along with five major tasks and four categories of metrics.
🧠 FlagEval Report
作者：FlagEval
The FlagEval Report series provides in-depth insights into the evolving landscape of large-scale model evaluation. Each issue delivers a comprehensive analysis of model capabilities across diverse tasks and metrics, enabling researchers and developers better to understand the strengths and limitations of leading AI models.
Issue 2 (2024-12-30 Updated) pdf
Issue 1 (2024-07-13 Updated) pdf
Continuous Benchmarking: nightly runs on FlagScale with automated PR badges and regression alerts; Community Challenges: quarterly leaderboard sprints to surface emerging research directions
🤝 Contributing
We welcome issues & PRs! Please check each project’s CONTRIBUTING.md and adhere to its license terms.
📄 Citation
If you use any component of the ecosystem, please cite the corresponding paper listed in that project’s README.
🛡️ License
This meta‑repository is released under Apache‑2.0. Individual projects may apply different licenses—see their respective READMEs.
Maintained by the FlagEval team · Last updated: 2025‑04‑23