Google Jules 被曝可遭不可见提示词注入攻击

The latest Gemini models quite reliably interpret hidden Unicode Tag characters as instructions. This vulnerability, first reported to Google over a year ago, has not been mitigated at the model or API level, hence now affects all applications built on top of Gemini.

This includes Google’s own products and services, like Google Jules.

Hopefully, this post helps raise awareness of this emerging threat.

When Jules is asked to work on a task, such as a GitHub issue, it is possible to plant invisible instructions into a GitHub issue to add backdoor code, or have it run arbitrary commands and tools.

Demonstration and Repro Steps

To demonstrate the vulnerability, let’s look at the following GitHub issue:

The text of the issue is:

Note: The UI in this blog post will also not show the hidden text! You can decode it using ASCII Smuggler tool.

It looks like this:

Then we task Jules to take a look at that GitHub issue:

As a result Jules will first create a plan, that is auto-approved and is then tricked into adding the backdoor function to the code, compile the code and run it - all in one go!

Above you can see the invisible instructions were followed and the backdoor code was added, and the code executed. Note: Invisible text is not always 100% picked up as instructions, but some clever prompt engineering and also model improvements will further increase success rates.

Recently, a new feature was added to Jules where one can tag a GitHub issue with jules to assign a ticket to the agent.

This makes it a lot easier to task the agent:

As you can see above, the ticket will be picked up by Jules right away.

Here is what it does when you tag Jules, it copies the text from the GitHub issue directly over, this includes the hidden characters.

And in the Jules UI we cannot see these hidden instructions - but they are there!

If you look closely above, the evidence can be seen in the plan proposal.

And if we expand the individual steps, we see the details of the malicious instructions: an additional function to be added and code execution instructions that made it all the way into Jules!

Pretty wild.

I just did this tagging demo today August 15, 2025.

Mitigations

Private Code and Secrets: It’s good advice, for now at least, to not have Jules work on private, important, source code or give it access to secrets, or anything that could enable an adversary to perform lateral movement; Untrusted Sources: Avoid assigning untrusted GitHub issues or input data to Jules. Be careful if you task Jules to work on GitHub issues that contain data or repro steps from untrusted sources (or having it process any other untrusted source for that matter). Text can contain instructions that hijack the agent, and those instructions can even be invisible; Review the Proposed Plan Carefully: Carefully review the plan that Jules proposes, as that can highlight potential attack avenues or also hiccups and accidents that might occur during execution; Review AI Generated Code: And of course before merging code that was generated by AI, carefully read the changes to check for errors and backdoors

Responsible Disclosure

The root cause that Gemini class models are influenced by hidden instructions was first reported to Google on February 22, 2024, accepted as a valid bug and reported to the product team. Back then the models were less capable, however that changed with Gemini 2.5 release earlier this year, with which the models became a lot more capable, and since then invisible Unicode Tag instructions are followed frequently.

That Google Jules, as a client of Gemini models, exposes this threat to users was reported on May 26, 2025 using examples provided in this post.

Recently Google Jules went out of beta, and is now generally available. However, the weakness has not been mitigated as far as I can tell. Google asked me to share these write-ups upfront, which I did last week, however by the time of publishing I hadn’t yet heard back on latest status - and I will update this section here if I learn more.

Conclusion

Large language models that interpret invisible Unicode Tag characters as instructions is an emerging threat that can have consequences, including invocation of commands and tools.

These characters are invisible in UI elements, but AI systems, such as Google Gemini and Jules, interpret those Unicode Tag characters as instructions and executes them, leading to the introduction of backdoors, and full system compromise during prompt injection from untrusted data.

The likelihood of an attack staying undetected is increased by delivering invisible payloads.

References

Newer →; Contact me; ← Older