Cognition Releases SWE-2 Flash Coding Model and Discloses Training Details
Cognition releases the fast coding model SWE-2 Flash, based on thinkymachines' Inkling-Small (276B total parameters, 12B activated parameters), at 5x lower cost and 3x faster than SWE-2. Training uses large-scale RL, with adjustments made to the SWE-2 recipe, but the main gains come from higher-quality, more diverse RL tasks; the research team audited task instructions, environment configurations, and verifiers and added several new task types. The model also adds a mid-training stage before RL for the first time, trained on trillions of tokens of code, scientific articles, and blogs, and is available free of charge to Pro, Max, and Teams users in Devin Desktop and Devin CLI until the end of this month.
Why read this
SelectedAI score 71
The official disclosure of the SWE-2 Flash training recipe, with emphasis on data quality and task diversity rather than algorithmic changes, serves as a reference for those interested in coding model training.
Topics
Body
Our training proceeded with large-scale RL. While we made some tweaks to our RL recipe from SWE-2, most of the gains came from higher-quality, more diverse RL tasks.
Our research team carefully audited the instructions, environment setup, and verifiers for our tasks, while also curating several new task types.