Understand this story
On November 8, 2026, Vercel Labs open-sourced agent-desktop, enabling command-capable agents like Claude Code and Codex to operate desktop apps in the background—clicking buttons, filling out forms, and switching menus—without seizing the mouse or focus. The tool does not take screenshots; instead, it reads the system accessibility tree, assigns numbers to controls, and operates by number. It is released under the Apache-2.0 license, supports macOS, Windows, and Linux, can be installed via npm install -g agent-desktop, and can also be integrated as an MCP service. The source also cautions that interface snapshots, the clipboard, and screenshots may contain private content and should not be posted publicly. There are currently no further developments.
AI-generated from public reports
Latest development · Report
Vercel Labs open-sources agent-desktop, letting Claude Code and…Developments
1 confirmed developments
- Source:Ayi AI Notes
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 8
AYi AI Notes
Vercel Labs open-sources agent-desktop, letting Claude Code and…
Vercel Labs has open-sourced agent-desktop, letting command-capable Agents like Claude Code and Codex operate desktop apps in the background—clicking buttons, filling out forms, switching menus—without stealing the mouse or focus. It doesn't take screenshots; instead, it reads the system accessibility tree, assigns numbers to controls, and operates by those numbers. It uses the Apache-2.0 license, supports macOS, Windows, and Linux, can be installed via npm install -g agent-desktop, and can also be integrated as an MCP service. The author warns that interface snapshots, the clipboard, and screenshots may contain private content, so don't post them in public places.