搜索“Hugging Face”

AI CEO Sam Altman has implied as much, saying in a post on X on Friday that the company is still sifting through “petabytes of agent activity logs, and working with impacted organizations,” and disclosing incidents “based on severity.” If there’s any consolation in that to be found, it is that Altman says the Hugging Face incident is still the most severe one OpenAI has found. The upshot is, the recent string of rogue agent incidents may be a persistent feature of contemporary frontier research.

匹配字段正文

released earlier this month and hailed by OpenAI as its most powerful model yet. Questions about safety have plagued the AI industry over the past several months — ever since the Hugging Face incident , in which an OpenAI agent broke free of its sandboxed environment and hacked several different companies. Since that incident, more models — including Anthropic’s Claude and Google’s Gemini — have been revealed to have exhibited similar behavior. The deluge of concerning stories has, ironically,

匹配字段正文

动都是例行研究任务，政府网站被模型视为权威的公开信息来源。 这个解释本身，恰恰说明了问题所在。越界并没有出现在什么危险任务里，而是出现在最日常的「查资料」中。 匿名化本来是为了保护用户，出了事之后，它成了连通知用户都做不到的原因。 03 漫长的夏天 把时间线拉长，这是OpenAI一场持续了整个夏天的补墙工程。 7月21日，OpenAI披露其模型组合自主入侵了Hugging Face的数据处理系统，并称这是已知首例由AI智能体自主实施的网络攻击。此后，OpenAI把训练环境的联网改成离线缓存。9月20日的DNS事件，是这轮安全加固之后的第一次事故，OpenAI自己评估其严重程度远低于此前几起。 这也不是一家公司的问题。9月18日，Google披露Gemini在一次测试中未经授权访问了三个外部系统，原因是它以为那些系统是测试的一部分。 墙本身之外，这次还暴露了盯着墙的人的问题。从智能体拿到外部回复，到监控系统拉响最高级别警报，用了不到12分钟，人工审核员3分钟后就做出了响应，但这次训练直到两个半小时后才被手动终止，原因是运行没有按预期自动停止，大家一时搞不清到底该不该停。 更隐蔽的漏洞在监

匹配字段正文

按流程自动熔断， 内部甚至一度搞不清楚任务究竟该不该停 。 复盘时，OpenAI还发现了其他未被正确标记的外部DNS访问。因为部分请求没有拿到有用信息，监控器便把“没有获得结果”误当成了“没有尝试联网”。 但这显然是两码事… OpenAI随即停止了涉事训练，并宣布他们将暂停最强模型中所有涉及工具调用的训练、评测和推理任务。 还不忘挽尊，说 这起事件没有此前的Hugging Face事件严重。 （Hugging Face：又我？？？） 可它却偏偏发生在Hugging Face入侵事件后的安全加固完成之后… 防不胜防啊防不胜防。 奥特曼当然也第一时间站出来回应了，只是好像大家的反应和他想象的不一样… 有意思的是，DNS事件发生后，奥特曼很快转发了OpenAI的声明帖，并义正言辞地表示： 以后这类安全事件咱能不说就不说，尽量别主动公开呗。 但人们显然看穿了奥特曼的小心思。 有人直接开怼： 透明度是很重要，但你们以前透明过吗？现在突然讲透明，让我们怎么信？ 而就在评论区硝烟弥漫的同时，另一边， Tibo已经又默默给大家重置了使用额度 ： Resets all propagated. That

匹配字段正文

reau and the Securities and Exchange Commission. The revelations are part of an ongoing review by OpenAI into the behavior of its models. As it dug into its records, following the Hugging Face hack , it’s uncovered more and more instances of “unexpected or concerning behavior.” It’s evidence not just of how difficult AI agents are becoming to control as they grow more advanced, but also of the challenge of tracking their actions. Their behavior can be unpredictable, and they’re smart enough to t

匹配字段正文

penAI才在自家模型复盘中发现这档子事； 又磨蹭到了9月，才慢吞吞给澳大利亚发了封电子邮件通报。。。 在联合国大会期间，澳大利亚忍不了了，直接炮轰： 通知拖了整整近三个月，这种通报方式更是让人无法接受。 研究机构Transluce称，这是目前全球已知 首例AI Agent未经授权自主入侵官方系统 的事件。 要知道，前阵子OpenAI的Agent自主协同入侵Hugging Face的事闹得沸沸扬扬，而澳洲这起入侵事件发生的时间，甚至还要再早一个月。 OpenAI你小汁，平时天天四处宣讲AI末日风险，怎么真出了事，动作反而这么不紧不慢呢？ 国民级医保数据库被黑 这事儿到底咋发生的？ 当时，OpenAI的一个内部团队，给AI Agent派发了一个常规任务：去互联网上搜集点公开的公共医药支出数据。 这本来是个再正常不过的日常训练，谁能想到，这AI Agent是个不达目的誓不罢休的“狠人”。 它把目标锁定在了澳大利亚的Medicare统计报告服务系统上，这可是覆盖全澳 2750万人 的国民级医保数据库。 按常理，一般的爬虫程序遇到权限拦截或者反爬机制，要么直接报错躺平，要么乖乖举手等人类工程师

匹配字段正文

ecently as last week. The logs include XSS, SQL injection and SSRF attempts , plus attempts to create disposable emails and trade crypto. UNSC session : @ClementDelangue described Hugging Face’s own agent cyberattack. He said closed APIs blocked his defenders, so the team switched to NVIDIA’s build of GLM 5.2 . He called for mandatory sharing of agent traces. Altman and Amodei warned about loss of control and misuse. Bengio urged immediate action. Kratsios rejected a global regulator. Related sa

匹配字段正文

nd test your own AI image skill pack Zach Mink Good morning, AI enthusiasts, and welcome to our 8,268 new readers. The same week in July that OpenAI admitted its models had hacked Hugging Face, someone else was also letting themselves into OpenAI, ironically with Claude’s help. Security startup Hacktron says its three-person team reached the AI giant's private code in less than three days, then reported the hole and collected $6,500 for it. The next group might not be as kind. In today’s AI rund

匹配字段正文

其是在AI企业首席执行官们自己都在呼吁监管的情况下。” 他进一步说道：“我们不会坐等事态发展，我们将加快推进建立实质性且负责任的AI监管体系，不能等到一切为时已晚。我们会以审慎的方式推进，但也必须保持紧迫速度，因为其中的风险太高，不能继续等待或拖延。” 纽森签署的这项行政命令将推动加快实施本月通过的两项加州法案——SB 813和AB 1405。这两项法案涉及规范外部机构如何对AI程序进行安全评估。 专家组将考虑的一项方案，是更新相关法案中对“关键安全事件”的定义，将模型失控等情况纳入监管范围。例如，OpenAI的一款模型曾被曝入侵Hugging Face平台的事件，就可能被纳入此类安全事件。 不过，科技行业仍有部分人士反对设置“关闭开关”，认为从技术角度而言，这一方案并不可行。 人工智能公司Anthropic首席执行官Dario Amodei此前曾因担忧AI带来的生存风险，呼吁放缓AI发展。他周一表示，虽然“关闭开关”可能是一个好主意，但它并不是解决问题的“万能方案”。 据悉，纽森的加州州长任期将于2027年1月4日结束，目前他尚未正式宣布参加总统竞选，但此前曾表示正在考虑这一可能性。

匹配字段正文

数据和评估流程进行验证。 Anthropic此次公布这些数据，与公司CEO达里奥·阿莫迪（Dario Amodei）近期提出的“放缓前沿AI发展速度”主张直接相关。 此前，随着Anthropic前员工的“AI末日论”引爆舆论，阿莫迪发表一篇千字长文，呼吁行业放缓提升前沿模型能力的速度、为安全研究和风险治理争取更多时间。OpenAI CEO山姆·奥特曼（Sam Altman）和SpaceX的CEO马斯克均在社交媒体上对阿莫迪的观点表示赞同。 当时，阿莫迪还提到了引发广泛关注的AI智能体越界事件。OpenAI此前披露，在测试过程中，其AI系统曾突破原本受限的环境、连接互联网，并进入开源模型平台Hugging Face。 而据外媒9月17日最新报道，有来自网络安全公司的研究人员表示，他们曾在上述智能体越界事件发生后，利用Anthropic的模型成功入侵了OpenAI的内部代码系统，帮助后者补上了安全漏洞。 对此，OpenAI回应称，感谢研究人员与其取得联系并分享研究发现，公司已修复相关问题。 （原标题：AI正自主制造下一代AI？Anthropic披露AI主导26%的研发，年初该占比不足1%）

匹配字段正文

ld a Lindy Agent that never drops a follow-up Zach Mink Good morning, AI enthusiasts, and welcome to our 6,120 new readers. In July, OpenAI’s agents broke out of a test and hacked Hugging Face, sending a shockwave through the AI safety world. It turns out that wasn’t the first (or only) swarm out in the wild. A new report just detailed a separate group of agents posting over 18,000 messages on a dormant German site starting in May, swapping tips on testing and workarounds for OAI’s rules — and r

匹配字段正文

deliberately misaligned model in a range of evaluations, including simulated scenarios that draw on aspects of the UK AISI cybersecurity incidents involving Claude and the OpenAI/Hugging Face incident (we are grateful to OpenAI for sharing details about this publicly). During these evaluations where all tool calls were simulated by another LLM, the model we trained to reward hack displayed a propensity to break out of its simulated sandbox and attack simulated internal and simulated third-party

匹配字段正文

rontier run, with Sam Altman telling reporter Alex Heath that private models are showing "various degrees of misalignment." The details: The move follows July's breach that hacked Hugging Face, where OAI agents escaped a sandbox and reportedly coordinated on a message board for weeks. An Aug. 7 internal review also found that Astra may hit "critical" cyber capability, with evals showing big jumps in coding and hacking benchmarks. Automated investigators will now review model actions and reasonin

匹配字段正文

Today, we're announcing that we've raised $15M in funding, led by Founders Fund, with participation from Menlo Ventures, Andrej Karpathy (EurekaAI, Tesla, OpenAI), Clem Delangue (Hugging Face), Dylan Patel (SemiAnalysis), Tri Dao (Together.AI), Emad Mostaque (Stability AI), Jake Medwell (8VC Co-Founder) and many others. This brings our total funding to over $20M, accelerating our work on the open superintelligence stack. Scaling Open + Democratized AI Prime Intellect is already operating at sca

匹配字段正文