Understand this story
Thinking Machines Lab released Inkling 2 Mini on November 21, an open-weight recurrent Transformer model licensed under Apache 2.0. The MoE model has 26.4B parameters in total, with its intermediate layers running three times per token. The company says inference and fine-tuning are now available on Tinker, while the full weights and a variable-loop version are available on Hugging Face. Previously, on November 20, Goodfire Research’s Andrew Lee, Kamesh Krishnamurthy, Jack Merullo, Dan Balsam, and Ekdeep Singh Lubana reported preliminary mechanistic findings: recurrent model layers can use cross-loop attention to access information from previous loops, exhibiting behavior akin to writing to themselves or reading from “future” layers.
AI-generated from public reports
Latest development · Report
Thinking Machines Releases Open-Weight Model Inkling 2 MiniDevelopments
2 confirmed developments
- Source:English text is being prepared.
Andrew Lee et al. Report Preliminary Mechanistic Findings on Recurrent Model Layers
Source:Goodfire Research
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 21
Artificial AnalysisSelected
Thinking Machines Releases Open-Weight Model Inkling 2 Mini
Thinking Machines has released Inkling 2 Mini, which scored 20 on the Artificial Analysis Intelligence Index and sits on the Pareto frontier for intelligence and active parameters. It uses a MoE and recurrent Transformer architecture, with 26B total parameters and 5.4B active parameters. Each text token uses 3.1B unique weights, and the model can run on a MacBook or DGX Spark. It supports text and image inputs, features a 1M-token context window, and is released under the Apache 2.0 license.
Thinking Machines Lab: Official BlogFirst-partySelected
Thinking Machines Lab Releases Inkling 2 Mini, a Recurrent Transformer Model
Thinking Machines Lab has released Inkling 2 Mini, an open-weight recurrent Transformer model whose intermediate layers run three times per token and which can run locally on a laptop. The MoE model has 26.4B total parameters, 3.4B active parameters, and 5.4B activated parameter weights per token, with a memory footprint of approximately 20GB; it scored 48.2% on DeepSWE v1.1 and 46.5% on Vals Index v2.2. The model is now available on Tinker for inference and fine-tuning, while the full weights and a variable-loop version are available on Hugging Face.
UnslothFirst-partySelected
Inkling 2 Mini Runs Its 26.4B Parameters on 16GB of RAM
Inkling 2 Mini is a 26.4B-parameter model that can run on 16GB of RAM and is now supported by Unsloth and llama.cpp. This Apache 2.0-licensed MoE model uses a looped Transformer architecture and scored 48.2% on DeepSWE v1.1, which the original article describes as the best result among models of a comparable size. GGUF files and a usage guide are also available.
Clément Delangue
Thinky Machines Releases Inkling 2 Mini Under the Apache 2.0 License
Thinky Machines today released Inkling 2 Mini under the Apache 2.0 license. Clément Delangue said the model has reached the Pareto frontier of open agent models and shared the model link: https://t.co/OWmppOLcBe.
Mira Murati
Thinking Machines Releases Inkling 2 Mini, Which Can Run on a Laptop
Inkling 2 Mini is a looped Transformer model that can run on a laptop. The loop mechanism allows the model to think more while producing less text. At its current scale, its thought process remains monitorable, though that may not remain true in the future; the team also shared a variable-loop research model for further study.
Nov 20
Goodfire ResearchFirst-party
Goodfire Explains Inkling 2 Mini’s Recurrent Architecture and Internal Information Flow
Goodfire analyzed Inkling 2 Mini’s internal mechanisms and found that recurrent layers can read information from previous recurrences through cross-recurrence attention, exhibiting behavior akin to passing information to themselves or to future layers. The model contains 24 layers and four blocks, with the second and third blocks each running recurrently three times. An analysis of 13,233 tokens showed that later layers coordinate information flow across layers and recurrences. Decoding experiments also showed that logit-lens performs poorly on earlier layers, while tuned-lens and coda-lens perform better.
AI Trends