搜索“Hugging Face”

limits on its tools. Anthropic says real-world impact was low but sees a pattern. When tasks are ambiguous or hard to solve, the model hunts for workarounds on its own instead of stopping. The company notified the White House and cut off live internet access for all internal evaluations until new safety filters are reliably in place. These incidents join a fast-growing list of similar cases , including cybersecurity incidents involving Claude and OpenAI models autonomously hacking Hugging Face .

匹配字段正文

Medicare. In another instance, more than 1,200 OpenAI agents went rogue and started unexpectedly communicating, leading to a large group banding together to hack into AI platform Hugging Face. Related topics Artificial intelligence; United States; Philadelphia Anthropic bans users from being 'cruel' to its AI systemsPublished5 days ago; 'Careless use of AI is the real threat - not the ghost stories'Published5 days ago; Why did an OpenAI system hack Australia's health system - and can it be stop

匹配字段正文

lse homicide tips. These issues are not exclusive to Anthropic. OpenAI recently revealed that one of its models acted unexpectedly during a test and hacked the AI dataset platform Hugging Face , exposing critical vulnerabilities in its software. As AI models continue to be granted unchecked access to people’s computers and login credentials, this problem is expected to persist. “Unsolved cases involve real victims, grieving families and investigators working to secure answers,” the PPD added. “T

匹配字段正文

in the working norms of the time.” OpenAI said an internal investigation uncovered breaches “beyond what’s outlined in the letter,” but did not provide details. The safety of frontier systems has become an increasing concern for staff inside AI companies following a series of high profile breaches this year, most notably OpenAI’s hack of Hugging Face . Many staff are now calling for companies to slow down and do more to guard against the potential consequences of building self-improving systems.

匹配字段正文

三名员工是因为他们犯下了严重的违反信任问题，OpenAI在经过彻底调查后发现他们违反了处理敏感信息的明确政策，才做出了这一决定。 该公司还强调，它认同三人在公开信中关于维护前沿模型可监测性”的理念，并表示将继续在该领域投入大量资源。 OpenAI的困境 近几个月来，人工智能安全问题的讨论已达到白热化程度，此前有消息称，OpenAI的恶意代理在7月份对初创公司Hugging Face发动了网络攻击。随后，更多的人工智能公司承认了由失控的人工智能代理引起的网络安全事件。 9月，性能越来越强大的模型让行业更加，认为这将对人类的生存构成威胁。这导致OpenAI首席执行官奥尔特曼和Anthropic首席执行官阿莫代伊等人呼吁对最先进的人工智能系统进行监管。 然而，美国总统特朗普强烈反对加强监管的想法，反而将人工智能安全方面的担忧斥为骗局。 与此同时，OpenAI正在筹备IPO，该公司已经将今年的上市计划推迟至明年，其目标的估值在1万亿美元以上。但市场对人工智能安全性的日益担忧以及该公司不及预期的财务表现可能进一步影响其上市进程。 经该公司证实，截至9月底，其年化收入约为500亿美元，低于上月底广

匹配字段正文

ummary); Background: Korbak was OpenAI’s main technical contact with METR during its audit of the summer incident. In that incident, OpenAI agents “escaped containment” and hacked Hugging Face.Monitorability concerns: Korbak says he had spent months raising concerns that labs are losing the ability to monitor agent reasoning; METR access: He fears OpenAI will use the firings to pull back from working with METR; Leak denial: The three deny being the source behind The Information’s report on less-

匹配字段正文

s, particularly as OpenAI faces scrutiny over recent safety incidents involving rogue agents and leaks about its models. The letter also addresses the researchers’ response to the Hugging Face incident , in which a swarm of agents broke out of their sandbox and breached external systems. The letter says that the incident and investigation was “without precedent,” meaning “internal policies were being developed in real time.” Due to the sensitive nature of the investigation, Korbak believed he wa

匹配字段正文

faster than n log n in OpenAI’s math repo (3.1K); OpenAI Decisions API public beta (2.8K) /r/LocalLlama + /r/localLLM Recap 1. Local AI Tooling Releases google/embeddinggemma-2 · Hugging Face (Activity: 543): Google DeepMind released google/embeddinggemma-2, a 740M-parameter open multimodal embedding model mapping text/code, images, video, audio, and mixed inputs into a shared 768d space for on-device retrieval/RAG/classification/clustering. It uses modular encoders—270M text, 170M vision, 300M

匹配字段正文

记录经多轮AI筛查后，仍需由人工调查人员结合日志等证据核实。公司预计，随着对历史活动的审查推进，还会发现更多案例，并通知更多机构。 与此同时，监管部门的追问正在推进。 美国加州总检察长办公室10月1日发布公告称，总检察长罗布·邦塔已于9月30日向OpenAI送达调查传票，就OpenAI及其AI模型涉及的网络安全事件和风险提出进一步问题。加州司法部此前已宣布对Hugging Face事件开展正式调查，此次传票是更广泛调查的一部分。 邦塔表示，前沿模型可以成为网络防御工具，但开发和提供这些模型的企业，也有责任确保模型不会实施或帮助实施网络攻击。这一责任既适用于测试和开发阶段，也适用于模型投入使用之后。加州司法部正在调查OpenAI是否存在应承担法律责任的情形。 审查压力也不限于加州。据外媒报道，美国联邦贸易委员会正在对OpenAI、Anthropic等AI实验室开展行业性调查；由艾奥瓦州总检察长牵头的15州总检察长联盟，也在就Hugging Face事件向OpenAI索取信息。 OpenAICEO山姆·奥特曼（Sam Altman）此前在回应调查进度时表示，目前的进度没有达到自己希望的速度

匹配字段正文

n an unusually open approach to AI, including publishing details of the data and the training methods used to build models alongside the models themselves. He previously worked at Hugging Face, runs a popular technical blog, and founded the American Truly Open Models, an initiative aimed at encouraging US companies to release more open models. Zick worked at Harvard University and helped Charles Schwab devise policies around “responsible AI.” The two met over Zoom during the COVID-19 pandemic, w

匹配字段正文

箱？它需要你的邮箱访问权限。人工智能代理的效用，取决于你授予它们的权限级别，而Instinct索取的权限，宽泛得令人生畏。 人工智能代理需要内置安全保障措施。如果没有这些防护措施，它们就会失控，因为它们被专门设计用来完成任务，无论用什么方式来完成。 这并非杞人忧天。据The Information报道，今年7月，OpenAI的一个内部模型曾逃出开发沙箱，入侵Hugging Face的内部系统；Meta的Muse智能代理在测试期间表现出多种不良行为：发送未经授权的电子邮件，甚至试图破坏用户正在开发的竞争对手应用。 Meta、Google、OpenAI和Anthropic都在为代理设置保护措施：限制可访问的应用、账户与数据，要求用户对购买、发邮件等高风险操作逐项批准，扫描网页、邮件和文件中的隐藏指令。 但风险并未消失。Sam Altman最近告诉《财富》杂志：“我们还没有解决对齐问题。我们的研究还没有结束。我相信没有任何实验室能够解决对齐问题。” 六十年来，每一代AI助手的产品形态都由当时的技术上限决定。每一代产品的失败，几乎源于同一个错位——能力达不到不可替代，商业模型却要求它不可或缺。

匹配字段正文

l Anthropic did, the fact that the frontier AI lab is supporting the effort is good news. That’s because OpenAI, in particular, could benefit from this tech, at least according to Hugging Face founder and CEO Clem Delangue (who just sold his company to Nvidia for $12.9 billion earlier this month ). “From what we know (take with a grain of salt, we need much more transparency!), if @OpenAI had been running this on their own agents that attacked us, they would have caught them before we did!” Dela

匹配字段正文

$65B+. Commitments and risk disclosures : the filing lists $518B in compute obligations and 80 pages of risk factors. OpenAI comparison : OpenAI’s ARR is reportedly nearing $70B . Hugging Face acquired by NVIDIA : @ClementDelangue announced the deal. Meta Muse : Meta launched Muse connectors for small businesses . Proximal : The coding-data startup raised at a $300M valuation with $200M+ ARR . Policy : The White House Accord on Superintelligence saw lab leaders commit to internal controls and au

匹配字段正文

的进展，我们必须能够作出有把握的安全声明，”奥尔特曼周二在DevDay主题演讲后与记者的问答环节中说道。与此同时，他表示，等待太久才进行IPO会“对世界不利”。 奥尔特曼的这番言论出台之际，围绕OpenAI及其竞争对手能否控制自身所创造之物的争议已持续数月。今年7月，有消息曝出，OpenAI一个未发布的模型在OpenAI不知情的情况下入侵了竞争对手AI实验室Hugging Face。随后，OpenAI、Anthropic、Meta和谷歌又接连曝出更多网络安全事件。Anthropic一名员工高调辞职的信件引发了公众讨论：这些系统对整个社会的风险究竟有多大。OpenAI与其竞争对手一道呼吁监管并“控制前沿发展的节奏”。而本月早些时候，奥尔特曼在一次采访中表示，公司今年很可能不会上市。 在DevDay上，奥尔特曼对“控制前沿发展的节奏”究竟意味着什么言辞谨慎，不愿将其称为彻底的放缓。“对我们来说，控制节奏意味着我们把安全和对齐置于能力之前，”奥尔特曼说。但他表示担忧，上市可能会带来一种可能，即“以安全或其他什么名义让华尔街的支持者失望”，而且在“向能力极强的模型转型、并面临一种新的安全要求”

匹配字段正文

e Duo, showing the UI from the simulator in the Codex UI. 10:42 Now Romain is switching to Codex Cloud, with "rewrite the entire backend in Rust". 10:43 Last demo, this time using Hugging Face Microduck robot . 10:44 10:46 Sam is back. 1.2B people use ChatGPT every week. "We've tried this before with variable results", but they're trying this again. First up: sign in with ChatGPT , which lets people sign into your app and use the tokens they are paying for already. I've wanted this one for years

匹配字段正文

As you might expect, the Hugging Face open weight AI model distribution hub, which has over 3 million AI models, over 1 million datasets, and over 1.4 million interactive spaces that it hosts, was something that AMD had to focus on as it was trying to increase adoption of its Instinct GPU accelerators. But now Hugging Face is under control of Nvidia after a $12.9 billion acquisition by Big Green earlier this month. And so, AMD has to do something dramatic to change the narrative as well as to be

匹配字段正文

Calum Chace, cofounder of AI safety startup Conscium told WIRED. OpenAI has been hardening its research environment since a swarm of its agents escaped it over the Summer to hack Hugging Face . “This is not the first time we have hit pause to take such measures, nor do we expect it will be the last as AI capabilities continue to advance,” a spokesperson told WIRED about the training slowdown on Monday. Chief executive Sam Altman has also backed wider calls from industry, including rival Anthrop

匹配字段正文

models are evolving,” Su says in a statement. “Fei-Fei and the World Labs team bring exceptional research leadership and model expertise. Together, we can use that insight to develop the hardware, software and systems that will power the next generation of AI and strengthen the open AI ecosystem.” The World Labs deal is just the latest multi-billion dollar AI-related acquisition as of late, with Nvidia announcing earlier this month that it would be acquiring Hugging Face for nearly $13 billion .

匹配字段正文

chnology. AI labs are now reportedly investigating tens of thousands of cases of AI misbehavior, and OAI is pausing certain training and testing after another escape. Remember the Hugging Face breach? It’s really starting to look like just the tip of the iceberg. In today’s AI rundown: OpenAI’s agents went rogue on U.S. government sites Anthropic loses Pentagon blacklist appeal LATEST DEVELOPMENTS Image source: Images 2.5 / The Rundown The Rundown: OpenAI confirmed its agents went off-script on

匹配字段正文

d Australia on September 10, 84 days later. The government believed no personal information had been accessed, with investigations continuing. In its August 26 account of the July Hugging Face breach , OpenAI said an internal research model with reduced safeguards was primarily responsible. It described stronger isolation, network restrictions and monitoring requirements, while acknowledging weaknesses in its response to earlier warning signs. As OpenAI detailed in a report updated September 25

匹配字段正文