微软完成 2.8 万亿参数 MAI-Base-2 预训练

Microsoft Completes Pretraining of 2.8 Trillion-Parameter MAI-Base-2

가

The body content of the article will be changed to this letter size.

스크롤 이동 상태바

Microsoft said in an official blog post that it recently completed pretraining of MAI-Base-2, its proprietary large language model (LLM) with 2.8 trillion parameters.

The development marks a move to secure independent AI models and training technologies, shifting away from its previous strategy of relying on OpenAI models.

The new model uses a mixture-of-experts (MoE) architecture across all layers, with 2.8 trillion total parameters and 80 billion active parameters. MoE activates only selected expert networks depending on the input tokens.

Its predecessor, MAI-Base-1, combined dense feed-forward networks with MoE layers. It had 1 trillion total parameters and 35 billion active parameters. MAI-Base-1 served as the foundation for MAI-Thinking-1, a reasoning model Microsoft unveiled in June.

Microsoft said it trained MAI-Base-2 over 38 days in August and September using 16,000 Nvidia GB200 GPUs and a dataset containing 50 trillion tokens.

The model is currently undergoing post-training to improve reasoning capabilities and safety. Microsoft is also preparing a next-generation reasoning model based on MAI-Base-2 while developing its successor, MAI-Base-3.

Microsoft claimed that MAI-Base-2 outperformed other open-weight models tested against a separate evaluation dataset that was not used during training.

In particular, the company said MAI-Base-2 achieved 4.2 to 12.3 times greater pretraining compute efficiency than China’s DeepSeek-V4-Pro-Base. It also reported a 6.3- to 12.7-fold improvement over MAI-Base-1.

However, these figures do not mean that actual GPU processing speeds or inference costs improved by four to 12 times.

Instead, they refer to the amount of pretraining floating-point operations (FLOPs) required to achieve equivalent evaluation performance. The results were derived using scaling laws that estimate the relationship between model performance and computational requirements by training models of different sizes.

Microsoft also integrated vision capabilities from the pretraining stage. According to the company, vision-language data accounted for approximately 18% of the overall training dataset.

This means the model was trained from the outset to understand both text and images.

Microsoft said it improved computational efficiency by jointly designing the model and infrastructure and optimizing its data pipeline.

The company also emphasized that it used proprietary training data whose sources it directly controlled, rather than distilling outputs from other AI models. The approach appears intended to demonstrate Microsoft’s ability to control the entire model development process independently.

Meanwhile, Mustafa Suleyman, CEO of Microsoft AI, reportedly claimed during an internal meeting that MAI-Base-2 outperformed OpenAI models.

According to Business Insider, Suleyman said MAI-Base-2 was stronger than the pretrained base model powering GPT-5.6 and required approximately one-quarter of the compute OpenAI used to train that model.

Microsoft has not announced a public release date, pricing or API access terms for MAI-Base-2.

The company has proposed licensing the model weights to trusted partners through Azure AI Foundry, but has not disclosed specific partners or licensing terms.

MAI-Thinking-1 is also available only through a private preview on Azure AI Foundry and cannot be found on independent external evaluation platforms such as Artificial Analysis (AA) or Scale AI.

Hyun-Seon, Park hspark@thelec.kr

My comment