Doubao Work launches record & replay feature, supports narrating while operating for up to 30 minutes

Doubao Work is still grinding away,

Today I discovered that it has quietly rolled out a record & replay feature, the same kind of operation-recording skill as Codex.

To recap briefly, this skill is like a super-simple version of RPA (robotic process automation). You don't need to know how to program, and you don't need to wire up components to build a workflow. You just record the process of operating your computer and hand it to the AI, and the AI writes the step instructions itself and reproduces the same operation next time.

What's even slicker is that, compared with Codex, Doubao Work also adds a narration feature, and you can record for up to 30 minutes. I can narrate while I operate, telling it why I chose this option at this step, so I don't have to memorize mouse trajectories by rote, and I don't need to get a perfect recording in one take, which greatly lowers the pressure of using it.

I first tried controlling Grok Bot to automatically collect GitHub project updates as a test run,

During the execution of a recorded Skill, unlike Codex's Computer Use, if I want to use the corresponding window in the middle, it will actively disconnect.

Drawing on my experience with Codex, I often run into the situation where I open a conversation, that conversation needs to use my browser, but at the same time I want to open another conversation, and the two sides conflict. This is especially obvious when operating software; browsers are actually okay, and you can work around it through other Skills.

Let me be more specific.

For example, I want to use Codex to control Grok and have the same environment produce a daily report for me. But because I'm simultaneously recording a video, or talking to another Agent, the pages it pops up will interrupt me. I keep minimizing gpt and retrying, and many times it's precisely because of repeatedly switching interfaces that the operation fails.

So I think the two mechanisms of pause and retry each have pros and cons. If I got to choose, on my local machine, that is, the computer I use myself, I'd want an automatic pause mechanism, because it won't interrupt my main work. But if it's running on a remote Mac mini, retry is better, since it doesn't get in my way anyway.

In my last article, I didn't write that this record-and-replay skill would be used in combination with the existing Computer Use and Browser Use (browser automation). Take Doubao, for example: it actually first returns what the current state is, and then gives the subsequent actions.

So the next step, the best approach, is to have it do patrol inspection.

I deliberately emphasize the word patrol inspection because in the past I might have said to check every 5 or 10 minutes. But actually, many times, if you're not sure about the time interval, you can just let the model do the patrol inspection itself; they will judge the appropriate time interval on their own until they observe the task is complete.

The record-and-replay skill is, on the whole, complementary.

When the automation you want to do has no API, and doesn't just run in the browser, and the software being operated spans multiple steps, or the workflow spans multiple pieces of software, then I highly recommend recording it as a skill. Especially since it can also be paired with narration. Let me use a very classic case to explain to everyone just how important narration is. Take one of my screen-recording skills,

Previously, to save trouble, I directly maximized the page I was currently operating before starting the screen recording, and it would reproduce my operation every time. But this creates a problem: it can only record a video exactly the same size as my computer screen, while a lot of the time my material actually needs to be 4:3.

This involves an operation that is a bit hard to describe in words,

I need to shrink all the windows on the screen except the current window. This action changes from "enlarge one window to full screen" to "minimize all windows except the one currently in focus on the screen," from "a certain action" to "a certain type of action." Because the software I open each time is not the same.

This is also part of why, even though so many Agents today already have Computer Use capabilities, and also have browser automation capabilities, I still think record-and-replay capability is needed.

Because opening multiple screen-recording apps at the same time will conflict with each other, I recorded the latter part with an iPhone. You can see that when Doubao Work encounters a situation where the transparent bar of the screen-recording software appears in a different place on the screen each time it records, it can still take a full-screen screenshot, zoom in, and confirm before pressing the button.

There's an even more outrageous operation,

I think many Skills won't specifically do it, but it's very well suited to our usage habits.

I think with Computer Use or browser automation, the most common problem everyone has always wanted to solve is going to the platforms we use every day to get hot posts, low-follower viral content, and so on. One mountain you can't get around here is how to avoid risk control.

With the record-and-replay skill, you can take advantage of each person's unique ADHD habits.

Suppose today you want to find low-follower viral posts on Xiaohongshu. Browse with the web page; you don't need to restrict yourself to fixed pages and a fixed rhythm. You just need to finish within 30 minutes. In between, you can scroll Douyin, scroll X, or even go to Bilibili to watch a video; it doesn't matter. The closer it is to your daily habits, the better. After you finish recording, just package it and hand it to Doubao, and then an information-gathering skill that is almost impossible to detect is born!

Most Agents get detected because they operate in a very fixed way on fixed web pages, and the click intervals are also very fixed. In the past, I tried clicking at random intervals of 15 to 60 seconds each time, but they still got detected after a long time. What was measured later was still abnormal operation time—who spends over an hour scrolling a certain little red book site on the web? That's just too unlike a human.

At first I thought about letting it operate for 5 minutes, then stopping for 6 minutes, creating a fairly random rhythm. But since we have multi-platform collection needs, why not try collecting from multiple platforms? My approach is to chain 5 platforms together; in reality, only 2 to 3 platforms are what we actually need to collect, and the platforms in between are meaningless actions.

The explanation I gave while operating was like this.

I want to demonstrate a kind of random cross-platform collection: switch among Douyin, Xiaohongshu, and X in no particular order, stay a few minutes randomly on each platform, and when I see content worth referencing, note it down in notes. Later, give you a topic, collect for 30 minutes at this rhythm, and finally give me the table.

I don't want to repeatedly explain to the Agent why this is done; just internalize it through a skill into a rigid action. It's like doing experiments in a lab—sometimes you have to put something in the microwave, then run out to eat hot pot and come back, and then it succeeds.

Most of the reasons things got monitored before were because they were on fixed web pages, then operated in a very fixed way, and the click timing was also very fixed. Although sometimes I differentiated it and had it follow random time intervals of 15 to 60 seconds, it would still, over a long time—who scrolls this stuff for over an hour, and on the web version too—be just too unlike a human.

Besides collection, I have another very painful point, which is the formatting and publishing of WeChat official account articles.

My current process is to choose a formatting app from Feishu, copy the draft to the WeChat official account backend, adjust line spacing, upload videos, paste the ending, cover, main and subheadings, and send a preview. Doing this whole set takes half an hour. So I want to try whether Doubao Work can also replicate it.

During the first use, I was also worried about whether some redundant screen segments would affect the quality of the entire skill.

Because the Doubao Work feature prioritizes audio conversion during processing, we can explain clearly at the beginning of the recording what actions are going to be done next. That way, even if there are some extra frames in the middle or at the beginning or end, or if a message notification in the middle slightly interrupts the recording process, it doesn't matter.

Amazing.

Doubao Workbench has also updated many small scattered features before, so you can see its work progress, background tasks, and conversation outputs here. Although it will pause the task when it is using the same interface as you, opening multiple web browsers at the same time will not cause this problem. I can put it in the background and let it operate on its own, without affecting the normal use of other workflows.

After testing these cases, Doubao Work is actually much more serious than I imagined, and iterates much faster.

There are also some similar scenarios.

For example, Figma and Canva cover templates need to export multiple styles; the same goes for CapCut—material entry and sorting, multi-format file export after rendering, and separate audio export. These operations are all very suitable for making into skills through record and replay.

I'm very excited that Codex is not the only choice.

Tibo reset it 11 times within these 28 days.

I pedal when I get up, pedal when I sleep, and pedal on my phone when I go out. I've already lowered the tier from $200 to $100, and I still can't use it all up.

I can already imagine how insufficient the quota will be after 28 days. I guess by then I'll only be willing to use Terra a little for browser automation.

Now I plan to migrate a batch of record & replay skills made with Codex over to Doubao Work.