Qwenputer embeds a RISC-V CPU in the Qwen3.5-4B weights and uses GDN state as memory
On Qwen3.5-4B, Qwenputer uses a compiler to write a 32-bit RISC-V CPU into about 11 layers of additional weights, and treats the model's built-in Gated DeltaNet state as addressable RAM, running C and MicroPython in the forward pass without calling an external runtime. In testing, the DeepMind arithmetic set reached about 97.5–100%, NUPA long integers improved from 48.6% for the base model to 100%, and about 230,000 CPU output positions showed zero mismatches against the reference machine. The costs are a checkpoint of about 18.8 GB, several decode tokens required per CPU instruction, and the need to drive it through RvChat rather than bare generate().
Why read this
SelectedAI score 79
The original text provides arithmetic benchmark comparisons along with costs such as checkpoint size and decoding overhead, which can be used to judge the feasible boundaries of the in-weight computation approach.
Topics
View original post1/4 Meet Qwenputer: Qwen3.5-4B with a full RISC-V CPU inside its weights, using the model's own Gated DeltaNet layers as RAM. It runs C and MicroPython in its forward passes, no external runtime. On DeepMind math: base 55–90%, base + Python tool 95–100%, Qwenputer 97.5–100%. https://t.co/aA7sAX8sDe
View original
1/4 Meet Qwenputer: Qwen3.5-4B with a full RISC-V CPU inside its weights, using the model's own Gated DeltaNet layers as RAM. It runs C and MicroPython in its forward passes, no external runtime. On DeepMind math: base 55–90%, base + Python tool 95–100%, Qwenputer 97.5–100%. https://t.co/aA7sAX8sDe
Body · AI translation
This idea is crazy!
Qwenputer uses a compiler on Qwen3.5-4B to write a deterministic RISC-V CPU into extra layers and weights, and treats the model's built-in Gated DeltaNet (GDN) state as addressable RAM, running addition, subtraction, multiplication, and division, C, and MicroPython in the forward pass without calling an external runtime; large-number arithmetic comes close to "base model + Python tool," and it also casually squeezes in an experimental "hard drive." https://t.co/3GEnraNsvt
So what exactly is Qwenputer? The author explains it clearly: it is not a plugin, not a tool call, not a sandbox. The base weights stay as they are, with about 11 layers of "computer" weights added in front (relative to Qwen's 32 layers). The CPU is a 32-bit RISC-V (RV32IM, plus Zba and a small bignum unit), passes the official riscv-tests, and is aligned with qemu; the author claims it ran about 18 million instructions freely with no errors. Each instruction becomes several generated tokens (address, value, STEP), and these tokens are the execution trace; a small driver, RvChat, loads the firmware and keeps the trace outside Qwen's own layers, so the model can use the already-familiar tool-call format to issue instructions "internally" and read back results.
Why can GDN serve as RAM? In Qwen3.5, 3 out of every 4 layers are Gated DeltaNet (a linear RNN). By setting both the gate and the write strength (beta) to 1, and then using one-hot keys to select rows, it can precisely overwrite and read back individual slots; the state size does not grow with conversation length, making it cheaper than treating ordinary attention as memory. This is the practical version of Schlag et al.'s "linear attention ≈ fast-weight associative memory": addressable memory inside a production model, large enough to run an entire CPU.
What the author demonstrated A user asks -2241842 × 8077. The grid on the right visualizes the GDN state: from Qwen ingests characters, a/b hold the operands, and the register and memory regions jump around; on the left, the model finally writes out the answer -18107357834. The point is: large-number multiplication is not "guessing the next token," but is computed by the CPU inside the weights.
Test numbers DeepMind arithmetic set: base model about 55–90%, base model + Python tool about 95–100%, Qwenputer about 97.5–100%. NUPA long integers: base model 48.6%, Qwenputer 100%, on par with +Python. About 230,000 CPU output positions were compared against a reference machine, with zero mismatches; the ordinary text distribution is almost unchanged relative to the base model (KL about 6e-4). The cost: the checkpoint is about 18.8 GB (about twice the base model, mostly zero padding); each CPU instruction requires several decode tokens, so large computations take seconds; and it cannot be run with bare generate() yet—RvChat is required. The author batches about 3,200 CPU tokens/second on an L4.
"Hard drive" and next steps Directories and chained storage are implemented in the GDN state (about 1,024 slots, up to about 512 files / 53,000 tokens of text), and state can be saved and reloaded across sessions; the author reports about 90% on altered fact recall and about 30% on pasting old chats. There are also Filesystem weights. The next step is to have Qwen write Python and run it on MicroPython inside the weights; and the author points to two safety-oriented follow-ups: hybrid models hiding reasoning in RNN state to evade CoT monitoring, and turning a monitor compiled into the weights into a built-in tool the model cannot turn off.