AI News

前 OpenAI 工程师称 DeepSeek 等中国实验室用 Blackwell 芯片训练模型

indigo更新于 11月13日 09:44

前 OpenAI 工程师发帖声称通过模型权重初始化指纹分析,发现 DeepSeek、Moonshot、MiniMax 和阿里等中国 AI 实验室使用被美国出口禁令限制的 NVIDIA Blackwell 芯片训练最新模型。

AI 导读

前 OpenAI 工程师 will depue 发帖称,通过模型权重初始化指纹分析发现 DeepSeek、Moonshot、MiniMax 和蚂蚁集团的最新中国模型在 NVIDIA Blackwell 芯片上初始化训练。他搜索超过六万亿种芯片布局与随机种子组合,称 DeepSeek-V4-Pro、Kimi K3、MiniMax-M3 等六个模型匹配 Blackwell,其中四个对应 GB200 和 GB300,并在真实 B300 与租用 GB200 上复现了初始权重。他估计 Kimi K3 和 DeepSeek-V4-Pro 各使用 2000 至 3000 块 Blackwell GPU,单集群采购成本约 1.5 亿至 2 亿美元,并称结果获 OpenAI 和 Anthropic 资深研究员独立验证。

推荐理由

精选AI 评分 80

这条转述呈现了一种基于模型权重指纹的取证方法,读者可据此了解出口管制争议中的技术论证路径。

主题

标签

引用

will depue

@willdepue

我有一个重大结果要分享。 我找到了确凿的取证证据,表明中国排名前四的AI实验室——DeepSeek、Moonshot、MiniMax和蚂蚁集团——用NVIDIA最先进的Blackwell芯片构建了它们最新的模型,而这些芯片是被禁止出口到中国的。 这听起来很疯狂,但我发现几乎每个AI模型都带有训练它的GPU的隐藏特征。当模型首次建立时,其权重的初始值会以一种取决于具体GPU的模式排列。每种GPU与种子数的组合都会产生一种独特的模式——一种用于创建该模型的硬件的指纹。这种模式的痕迹会在训练过程中保留下来,并进入已发布的模型。 再次强调,这一证据并非基于任何内部信息或泄露。它基于对各公司自己发布的模型权重的取证分析。 我搜索了超过六万亿种芯片布局与种子数的组合,并将它们与开放权重模型进行匹配。在此过程中,我找到了确凿证据,表明六个中国AI模型——包括DeepSeek-V4-Pro、Kimi K3和MiniMax-M3——是在NVIDIA Blackwell系统上初始化的。其中四个模型匹配Grace Blackwell,即NVIDIA旗舰级液冷机架式系统中的GB200和GB300部件。 随后我在真实硬件上验证了这些结果。使用我恢复出的种子运行未经修改的PyTorch,一块Blackwell B300逐位复现了DeepSeek-V4-Flash未经改动的初始权重,而租用的一块GB200复现了Kimi K3的初始权重,相关性达到0.999995。一块Hopper H100未能复现DeepSeek-V4-Flash的任何初始权重,而将种子改变一个就会破坏匹配。(顺便说一句,DeepSeek的种子是953049583。)该方法还被验证能将24个已知的开放权重模型与开发者声称使用的硬件相匹配。 这些结果是巧合的可能性极低。在六万亿种组合中,最佳的偶然匹配应该只能达到约8 sigma(即比随机偶然产生的结果高出八个标准差)。真实匹配的范围从49到1,385 sigma,即使是最弱的单项测试也达到20。 这些是数据中心规模的训练运行。通过对发布时间表和模型细节的一些分析,我估计最大的Kimi K3和DeepSeek-V4-Pro各使用了2,000到3,000块Blackwell GPU。这种规模的集群每套采购成本约为1.5亿到2亿美元,仅这些实验室加起来就会构成超过5亿美元的受出口管制GPU,还不包括实验性算力或其他方面。我们很可能看到的是数十亿美元级别的受出口管制GPU支出。 我对这一结果高度自信,并且它已得到OpenAI和Anthropic资深研究人员的独立验证。你可以在https://t.co/YOtzND0Jqk阅读完整报告。 通过其他类似的检查点取证形式,我还发现了关于这些模型的更多信息,很快会有更多内容分享。

查看原文

I have a major result to share. I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China. This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model. Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves. I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems. I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used. It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20. These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs. I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at https://t.co/YOtzND0Jqk. There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.

查看原帖

正文

前 OpenAI 工程师发帖声称通过模型权重初始化指纹分析,发现 DeepSeek、Moonshot、MiniMax 和阿里等中国 AI 实验室使用被美国出口禁令限制的 NVIDIA Blackwell 芯片训练最新模型。

其方法依赖 GPU 类型和随机种子产生的独特权重布局模式,这些模式在训练后部分保留,经数万亿组合搜索和真实 Blackwell硬件验证,与多个中国模型高度匹配。

分析显示这些实验室可能使用了数千块Blackwell GPU,涉及数亿美元规模,暗示出口管制存在漏洞,作者称结果获 OpenAI和 Anthropic 资深研究员独立验证。鬼故事越来越多 …

来源

indigo