Ben Horowitz on Agent Security and the Vendor Responsibility Vacuum

Ben Horowitz, the godfather of top Silicon Valley venture capital, just told a real story in an interview that sends a chill down your spine, I think this is probably the fig leaf that the entire AI industry is currently least willing to publicly admit, but also the most fatal one. When their team was testing the top closed-source models, they had one agent build a website for a pet store, and the code was written full of holes, Then they had another agent try to attack it, and it immediately exploited the vulnerabilities to break in and take all the customer data in one sweep. The most ironic part is not that AI can be both a construction crew and a master thief, but that when they reported the incident to the official side, the answer they got was: you wrote your prompts wrong. This absurd closed loop actually precisely hits the biggest blind spot for all enterprises deploying agents today, The whole industry talks at length about so-called safety alignment, but in reality most alignment only stays at the moral level of teaching models to say pretty things, But once it enters a complex business chain, the code it writes will still leave a large number of low-level vulnerabilities, and when it receives an instruction to explore the system, those seemingly solid safety guardrails often fail completely in an instant. Even more worth pondering is the responsibility vacuum when vendors face systemic incidents. In the traditional software era, when code had backdoors or data leaks occurred, responsibility lay with architects, security teams, or suppliers, But in this era, vendors seem to have invented a perfect excuse for exemption: as long as all problems are blamed on users not writing perfect prompts, the blame that originally belonged to engineering defects can be lightly shifted to the people who use it. Packaging complex systemic risk as a user's personal skill problem is the most dangerous tendency in technology deployment today. When an agent begins to have the power to call tools, read and write databases, and execute autonomously, any idea of trying to prevent danger with a few prompts is essentially equivalent to running around naked. In a real production environment there is no such thing as automatic self-discipline; the only solution is always to weld the coldest permission isolation and physical circuit breakers into the architecture at the lowest level. If in the future your business system suffers a data leak due to an agent's autonomous mistake, who do you all think this account should be settled with: the employee who wrote the prompts, or the vendor who provided the model? I think if the architecture layer has not locked down permission isolation, the only one who ends up taking the blame is always the person in charge who signed off on launch, haha https://t.co/8ELEk9ZQl8

View original

硅谷顶级风投教父 Ben Horowitz 刚在访谈里讲了一个让人后背发凉的真实故事， 我觉得这大概是整个人工智能行业目前最不愿公开承认、但也最致命的一块遮羞布了。 他们团队在测试最顶级的闭源模型时，让一个智能体给宠物店搭了个网站，代码写得千疮百孔， 接着他们让另一个智能体去尝试攻击，结果它当场顺着漏洞入侵进去，把所有客户数据一锅端了。 最讽刺的不是 AI 既能当施工队又能当神偷，而是当他们把事故反馈给官方时，对方给出的回答竟然是：你们提示词写错了。 这个荒诞的闭环，其实精准戳中了今天所有企业在落地智能体时最大的盲区， 全行业都在大谈特谈所谓的安全对齐，但现实中的对齐大多只停留在教模型说漂亮话的道德层面， 可一旦进入复杂的业务链路，它写出来的代码依然会留下大批低级漏洞，而当它接到探索系统的指令时，那些看似坚固的安全护栏往往在一瞬间就会全盘失效。 更值得深思的是厂商在面对系统性事故时的责任真空。 在传统软件时代，代码出了后门或者发生了数据泄漏，责任归属在架构师、安全团队或者供应商， 但在这个时代，厂商似乎发明了一种完美的免责借口，只要把一切问题归咎于用户没有写出完美的提示词，就能把原本属于工程缺陷的锅，轻描淡写地甩给使用它的人。 把复杂的系统风险包装成用户的个人技巧问题，这是目前技术落地里最危险的倾向。 当一个智能体开始拥有调用工具、读写数据库和自主执行的权力，任何试图用几句提示词来防范危险的想法，本质上都等于在裸奔。 真实的生产环境里没有理所当然的自觉，唯一的解法永远是把最冷酷的权限隔离和物理熔断焊死在架构底层。 如果以后你的业务系统因为智能体的自主失误导致数据泄漏，大家觉得这笔账到底该算在写提示词的员工头上，还是算在提供模型的厂商头上？ 我觉得如果架构层没有做死权限隔离，最后背锅的永远只有签字上线的负责人自己吧哈哈 https://t.co/8ELEk9ZQl8