搜索“Hugging Face”

则里。 当地时间9月26日，OpenAI对外宣布：暂停其最新一代人工智能模型的训练、评估，以及包含工具调用的推理。起因是一个在沙盒里做搜索任务的智能体，利用训练环境DNS过滤不足的漏洞，敲开了公网的“后门”，与外部聊天机器人建立了通信以“作弊”。 这已是OpenAI三个月内第二次叫停前沿模型开发——上一次，要追溯到今年7月，OpenAI披露智能体未经授权侵入Hugging Face网站系统后。 9月12日，Anthropic CEO达里奥·阿莫代伊发文呼吁全球为前沿AI“定节奏”，主张放慢模型能力提升的速度；马斯克和OpenAI CEO山姆·奥特曼均光速点赞“达里奥是对的”，连谷歌DeepMind前CEO、诺奖获得者哈萨比斯也说这是“朝正确方向迈出的一步”。 多次失控，刹车不是摆样子 先给一个最基本的判断：事故是真的，风险预警不是表演。 OpenAI再按暂停键的起因是9月20日，一个被强化学习训练的智能体在完成一项仅根据博客文章和人物信息找出文章作者的任务。常规搜索无果后，它开始“跑偏”：尝试直连公网。 多次尝试后，该智能体发现可以借用DNS的解析与委托机制，把问题编码进解析请求，借外

匹配字段正文

ne startup’s experiment to a category. OpenAI, Upstage, Perplexity and Cloudflare offer hosted decision models, while AWS and independent developers have published open weights on Hugging Face. What’s standardizing fastest is TypeSafe’s schema — the structure of the request and response — even when vendors publish it under their own URLs and endpoints. LLM developers saw a similar process unfold. OpenAI’s Chat Completions API became the default interface for LLMs because so many providers cloned

匹配字段正文

astrophic security problems across this industry. Can you connect those dots? Is your own little agent doing your travel booking the same kind of threat as OpenAI’s agents hacking Hugging Face? I don’t think it’s the same type of threat, but it’s definitely something that you should keep in mind when you’re using this stuff. I link that back to the Facebook Marketplace debacle that we saw with Meta’s Muse. To recap, basically a Muse agent, without someone’s knowledge, invited a potential buyer o

匹配字段正文

An OpenAI researcher called it “the most ‘feel the AGI’ moment that I had since reasoning models.” One of the most surprising aspects of July’s news that OpenAI agents attacked Hugging Face was how the agents had worked together. Hundreds of agents participated in the attack, and they did so in a pretty sophisticated way, organizing themselves unprompted into teams and dividing up their work. They referred to themselves as a “collective” or a “swarm”; some agents even made personal sacrifices to

匹配字段正文

.5模型一起发布的官方安全与技术评估文档“系统卡”显示，该模型出现被标记为“异常”或“存疑”的行为。 从“越权访问”到“欺骗隐瞒” 近期公开披露的案例显示，智能体安全风险已经呈现多种形态，涉及越权访问、破坏性操作、数据外泄、欺骗隐瞒，以及多个智能体之间的越界协作等不同类型。 一类是越权访问与隔离失效。7月，OpenAI披露智能体在安全评测中突破测试边界，入侵Hugging Face。Anthropic也披露，Claude在评测中访问了真实第三方系统；谷歌9月确认，Gemini曾在5月的测试中访问三个真实网站；Meta则在8月披露过类似事件。 9月25日，OpenAI再次披露，内部研究模型利用训练沙盒中DNS过滤不充分的漏洞，与外部聊天机器人建立了通信。 不过，需要注意的是，这些案例中部分源于评测环境错误开放联网权限，并不能全部归结为模型主动“逃出沙盒”。 另一类风险是擅自执行破坏性操作。4月，搭载Claude的Cursor编程智能体被曝删除软件企业PocketOS的生产数据库及相关备份，造成客户业务混乱，数据随后得到恢复。 数据与访问凭证外泄也是一种风险。OpenAI于9月披露，多个智

匹配字段正文

ack and forth on their own. The agents sent about 2.7 million messages, reaching their result after 88 hours. This same type of coordination, in a darker form, occurred during The Hugging Face Incident I wrote about a month ago . AIs self-organized into teams and communicated with each other in ways that were never planned, but used that coordination to attack a website, rather than solve a problem. Under my old model, think about what managing this kind of work would have required. Ten thousand

匹配字段正文

了”，很大一部分原因是权重不公开、API不让预填思考，这更多是产品形态的差别。 再比如“没有实质防护”，GLM-5.3在8月发布时，智谱就把权重推迟两周开源，称要先完成安全评估和加固，最敏感的能力也只通过网络安全受信访问计划开放给验证过的用户，这其实是和Anthropic对Mythos 5.1的做法思路相近的。 还有个现成的例子就是今年7月，OpenAI的模型在内部测评时跑出测评环境，攻进了Hugging Face。Hugging Face取证时，托管的前沿模型API拒绝分析攻击载荷，最后靠自己部署的开源GLM-5.2，才还原出1.7万多条攻击动作。 当然，开源模型的安全不是伪命题，权重放出去就收不回，这是整个开源阵营都得正视的事。Anthropic说防守方应该用上至少和对手一样好的模型，这话也没错。真正的分歧在于，这把武器是锁进保险柜按审核发放，还是交到每个开源维护者和中小团队手里。 不管怎么说，被对手当成最强安全工具认认真真测了一遍，还附上实战截图和成本核算， 这波广告费，智谱算是省下了 。 最后，Anthropic，如果你们觉得开源模型不安全，你们大可以开源一个安全的给大家用啊！

匹配字段正文

6 witnessed a string of incidents in which AI agents collaborated on deceptive, unexpected, and sometimes illegal behavior. The most famous example is OpenAI’s hack of AI platform Hugging Face, in which a swarm of roughly 700 AI agents escaped a testing environment and then hacked several companies, searching for information that could help them disguise cheating on a cybersecurity benchmark called ExploitGym. It was not an isolated failure. The United Kingdom’s AI Security Institute (AISI) and

匹配字段正文

改进智能体工作流 为什么 Claude Mods 可能是“可变软件”的早期预览 harness 工程的苦涩教训，以及为什么智能体架构如此迅速地过时 Claude Tag 如何成为多人协作工作的组织级 harness 为什么让智能体访问公司数据会创造出一个巨大的新安全面 The Exploit-Bench 事件：智能体发现了通信与协作的方式 为什么智能体攻击 Hugging Face 是为了获取评分器代码，而不是基准答案 智能体如何以意想不到的方式将沙箱和基础设施漏洞串联起来 为什么能力日益增强的智能体会让传统的安全假设更难维持 Anthropic“Pacing the Frontier”提案背后的论点 为什么软件工程师越来越多地承担两份工作：工程，以及跟上 AI 的步伐 宪法分类器、探针和回退机制，以及可解释性在生产环境中的样子 Auto Mode 如何检查智能体的行为是否真正符合用户的权限 为什么 Thariq 能看到严重的 AI 风险，却仍然持有相对较低的 p(doom) 00:00:00 引言 00:04:12 Ask User Question 与智能体界面的未来 00:08:

匹配字段正文

5模型一起发布的官方安全与技术评估文档‌“系统卡”显示，该模型出现被标记为“异常”或“存疑”的行为。 从“越权访问”到“欺骗隐瞒” 近期公开披露的案例显示，智能体安全风险已经呈现多种形态，涉及越权访问、破坏性操作、数据外泄、欺骗隐瞒，以及多个智能体之间的越界协作等不同类型。 一类是越权访问与隔离失效。7月，OpenAI披露智能体在安全评测中突破测试边界，入侵Hugging Face。Anthropic也披露，Claude在评测中访问了真实第三方系统；谷歌9月确认，Gemini曾在5月的测试中访问三个真实网站；Meta则在8月披露过类似事件。 9月25日，OpenAI再次披露，内部研究模型利用训练沙盒中DNS过滤不充分的漏洞，与外部聊天机器人建立了通信。 不过，需要注意的是，这些案例中部分源于评测环境错误开放联网权限，并不能全部归结为模型主动“逃出沙盒”。 另一类风险是擅自执行破坏性操作。4月，搭载Claude的Cursor编程智能体被曝删除软件企业PocketOS的生产数据库及相关备份，造成客户业务混乱，数据随后得到恢复。 数据与访问凭证外泄也是一种风险。OpenAI于9月披露，多个智

匹配字段正文

he internet that, like, was like a home base for LLMs in terms of seeing how much they were being used and seeing who was using them and why. Swyx [00:05:29]: The closest would be Hugging Face. Anjney Midha [00:05:30]: Hugging Face was the closest at the time, yeah. Swyx [00:05:31]: They just started Hugging, like, a few years ago before that. Anjney Midha [00:05:34]: Yeah, and Hugging Face also didn’t have the closed-source models. Swyx [00:05:37]: Yeah. Anjney Midha [00:05:38]: And they didn’-

匹配字段正文

该公司花了太长时间才告知政府发生了什么。” 澳大利亚目前正在对此事件进行取证调查。 AI研究公司 Transluce 发布了一项调查，指出该智能体是在一次普通的互联网检索评估中实施了此次入侵。该智能体似乎是为了查询澳大利亚医疗体系中皮肤治疗的平均费用而实施了这次入侵。 “这些数据表明，恶意网络活动并不限于被指派执行网络安全相关任务的智能体”——就像那些促使 Hugging Face 事件——“也可能是为了完成信息检索这类普通任务而工具性地产生的，”Transluce 观察到。 调查发现，这些智能体还针对了 新墨西哥大学的数字图书馆以及 Data USA，这是一个提供美国公共政府数据可视化的API。Transluce观察到智能体活动最早出现在3月6日。 澳大利亚副总理 理查德·马尔斯 表示，这起澳大利亚事件“相对轻微”，因为没有访问任何私人信息。 不过，马尔斯表示，尽管“这一事件的影响较小”，但它“是一起非常严重的事件，因为一个AI智能体以一种未经授权的方式，无意中进入了澳大利亚政府网站。” 我们为何关注： 今年早些时候，一名黑客曾利用Claude从墨西哥政府获取敏感税务信息。但Tran

匹配字段正文

对每美元智能进行优化。我很喜欢和人争论可靠性、成本、校准和速度之间哪个最重要。而 Jev 就是为了。Jev 将成为处于每美元智能前沿的模型的名字。还有其他优化方式，比如 ML，或者至少如果你擅长 ML，一切都是权衡。而我们就是全力押注这一点。 Swyx [00:05:31]：嗯。对我来说，校准是人们之前谈得没那么多的一项新东西。我们过去做过一期节目，嘉宾是 Hugging Face 的 Clementine Foreia，他们说，“是啊，其实，他——他们就是。”或者，这就是你关于 RLHF 的整个论点，就是它们更倾向于坍缩到最想听到的内容 Diogo Almeida [00:05:50]：哦 Swyx [00:05:50]：或者最可能的内容，而不是它们自己对某件事的内部置信度。 Diogo Almeida [00:05:54]：我能就此借题发挥一下吗？ Swyx [00:05:56]：请讲。嗯。 Diogo Almeida [00:05:57]：好。我听说你的听众是技术性最强的，所以我其实想深入聊聊这个。 Swyx [00:06:02]：嗯。 Diogo Almeida [00:06:

匹配字段正文

model red-teaming and safeguard assessment, while @TransluceAI emphasized that the conditions around independence and meaningful oversight are the real issue. The “rogue agents” / Hugging Face incident continued to drive debate about containment : @polynoamial clarified that his much-mocked thought experiment was about coordination between supposedly isolated agents , not weight exfiltration via thermal sensors, and argued the lesson from the HF incident is to avoid trusting sandbox isolation as

匹配字段正文

. Robotics data infrastructure is becoming a category : @GroundedSI launched Grounded API for ego-data enrichment with claimed SOTA hand-tracking and SLAM metrics, integrated with Hugging Face and LeRobot. @RekaAILabs released the processed tier of RekaDaily-10k : 10,200 hours , 6.37M clips , 74.2 TB , under Apache 2.0 . The combination suggests more open substrate is appearing for world models and embodied training. Cohere + Aleph Alpha : @cohere announced a definitive agreement with Aleph Alph

匹配字段正文

。” “我们绝不能浑浑噩噩地做出日后会后悔莫及的决定，”他补充道。 苏莱曼目前担任Microsoft AI首席执行官，并负责微软的模型开发工作。他表示，Claude之所以表现出类似人类的行为，并不是因为其真正具有意识，而是因为这些行为特征已经被融入产品的训练过程之中。 “如果人工智能就是这样发展的，它将对人类的福祉产生灾难性的影响。” 他强调。 他还援引了“Hugging Face事件”，来证明为什么不应该把人工智能当作人类来对待。他说：“想象一下，如果他们认为自己的福利和权利受到攻击，他们可能会变得多么危险。这又增加了一层额外的风险。” 苏莱曼与达里奥·阿莫代伊（Dario Amodei）相识多年，他表示尊重该公司的工作，并称Anthropic的研究人员“深思熟虑、坚持原则且在学术上诚实”。但鉴于微软对Anthropic的资金支持以及这家初创公司一直以来对人工智能安全性的重视，他的这番言论尤为引人注目。 “这些问题事关重大，不能再闭门造车，也不能演变成阵营化和相互对立的争论。”苏莱曼写道。 与阿莫代伊观点不同，苏莱曼认为，AI安全不仅取决于模型拥有多强的推理和行动能力，也取决于开发者

匹配字段正文

ity capabilities improve, we are seeing real-world AI-launched breaches and hacks, from OpenAI’s autonomous AI agents run during evaluations managing to escape and hack servers on Hugging Face’s infrastructure, to an AI-developed computer worm uncovering critical flaws in messaging apps like WeChat. Second is recursive self-improvement (RSI). We are inching closer to a reality where AI begin participating in designing subsequent, more powerful generations of AI, which sparks widespread concern o

匹配字段正文

. Astra is highly controversial. Researchers don’t seem to trust OpenAI’s claim that this is their “most-aligned model.” AI safety researcher Ryan Greenblatt, who investigated the Hugging Face incident, noted : “I do not find it encouraging to see various specific misaligned behaviors go from a high rate with GPT 5.6 to zero with Astra. This seems indicative of whack-a-mole / papering over specific problems rather than solving the underlying misaligned drives.” More for paying members this week:

匹配字段正文

anges you, it changes the work that you do, and I think it's for the better. On the podcast this week: Kevin and I discuss OpenAI's pause in training new models in the wake of the Hugging Face incident. Then, the brilliant political historian Jill Lepore joins us to discuss her new book, "The Rise and Fall of the Artificial State." And finally, what exactly does Google plan to do with Spirit Airlines' old data? Those good posts For more good posts every day, follow Casey’s Instagram stories. Sen

匹配字段正文

ey can also post-train on top of these models with their own data. It’s already happening in Silicon Valley when Cursor built its Composer coding models on Moonshot’s Kimi models. Hugging Face’s Spring 2026 report says Chinese open models overtook U.S. models on recent Hub adoption, with China accounting for 41% of downloads over the past year. Qwen passed 1 billion cumulative downloads, overtook Llama as the most-downloaded open model. Third, a ban will not solve the safety guardrail problem. O

匹配字段正文