Rock Matrix Open-Sources EdgeRock-4B Text-Image Reasoning Model
Rock Matrix has open-sourced EdgeRock-4B, a small text and image reasoning model with fixed-capacity memory. The model fully retains the most recent 1,024 tokens and stores earlier important information in a separate memory bank, thereby keeping the attention KV cache size fixed and preventing long conversations and long reasoning from continuously consuming more GPU memory. Rock Matrix reports an average pass@1 of 82.9% on AIME 2024–2026, compared with 18.8% for the strongest bounded-state model it tested.
Why read this
AI score 65
The original text provides the specific approach to a fixed-capacity KV cache and AIME comparison data, which can be used to assess the memory trade-offs of long-context reasoning in small models.
Topics
Body
Rock Matrix has open-sourced EdgeRock-4B, a small text-and-image reasoning model with fixed-capacity memory.
EdgeRock-4B keeps its attention KV cache fixed in size by holding the last 1,024 tokens in full and saving important older details to a separate bank.
Standard language models keep a growing record of every token they read, so long chats and long reasoning take up more GPU memory over time.
Rock Matrix reports an 82.9% mean pass@1 on AIME 2024–2026, compared to 18.8% for the strongest bounded-state model it tested.