Google DeepMind 与斯坦福论文发现提示词中的彩色图形会左右模型写代码
so interesting. 🤔
Google DeepMind 与斯坦福的新论文发现，提示词里加入与任务无关的纯色图形会改变视觉语言模型的编码选择。研究测试了 288 种图形，在二叉树问题上，一个红色爱心让 Claude Opus 4.7 和 GPT-5.4 的递归解法比例从 0% 升到 100%；当图形确实影响了 Gemini 3.1 Pro 时，其 71.5% 的思考轨迹仍称该图像无关。作者建议对会读取图像的智能体做有无图像的对照测试，不要指望推理轨迹会提示视觉影响。
推荐理由
AI 评分 64
原文给出具体实验数字，说明无关图像能改变模型的编码选择，且模型的推理轨迹未必会提示这种影响。
主题
正文
so interesting. 🤔
New Google DeepMind and Stanford paper finds that a red heart picture in the prompt can decide how AI write code, that simple shapes in images can steer AI on unrelated tasks, often without the models saying so.
plain colored shapes in a prompt change how AI models write code, though the shapes have nothing to do with it.
Vision-language models now read screenshots and images as part of normal work. If an unrelated picture changes the model's choice, builders need to see it, ideally in the model's own reasoning.
They added a plain colored shape, like a red heart, to a coding prompt and told the model to look at it. Across 288 shapes, they tracked how often Claude Opus 4.7, Gemini 3.1 Pro, and GPT-5.4 chose recursion.
A red heart moved Claude Opus 4.7 and GPT-5.4 from 0% recursive solutions to 100% on a binary tree problem. When a shape did steer Gemini 3.1 Pro, 71.5% of its thinking traces called the image irrelevant.
If your agent sees images, test its decisions with and without them, and do not count on reasoning traces to flag visual influence.