Mirendil Releases Its First Model Fern 0.1, Supporting a 10M Token Context
Mirendil has released its first model, Fern 0.1, a coding model trained from scratch with a context window of up to 10M tokens, equivalent to roughly 23 MB of code, or about 500,000 lines. Mirendil-Base was pretrained by a single researcher in 3.5 months, with a total compute cost of less than $10 million; AI agents wrote more than 99% of the code, and the training data came only from public sources. RULER single-needle retrieval scored 100% at every length from 32K to 10M tokens, but Terminal Bench 4.0 scored only 6.1%, and SWE-Bench Verified scored 73.4%. Currently only hosted early access is available, and the weights have not been open-sourced.
Why read this
AI score 60
The original text also presents the long-document retrieval performance with a 10M token context and the clear shortcomings on agentic coding benchmarks, making it easier to judge the real boundaries of ultra-long-context models.
Topics
Body
Add as a preferredsource on Google
Mirendil has released Fern, its first AI model. Mirendil Fern is a from-scratch coding model with a context window of up to 10M tokens. Mirendil explains how that equals about 23 MB of code, or roughly half a million lines. That is a codebase on the scale of Megatron-LM, held in context at once. Fern 0.1 is the first version, built from Mirendil-Base after a short round of post-training. For developers, its like, you load a whole repo, a full training log or days of agent traces at once.
TL;DR
Size: 0.1 to 0.5T parameters (exact count and active params not disclosed); context up to 10M tokens.
Runs on: Hosted early access only; no open weights or quantized builds announced.
Performance: Strong long-context retrieval, mid-pack agentic coding versus open-weight models, well behind frontier models.
Best: 100% on RULER single-needle retrieval at every length from 32K to 10M tokens (base model).
Worst: 6.1% on Terminal Bench 4.0, versus 66.4% for Claude Opus 5.5 in Mirendil’s own table.
What is Mirendil Fern?
Fern is a language model from Mirendil, an AI R&D lab.
TEXTTABLELAYOUTIs your PDF parser actually getting it right?omni_parse_bench · scores text, tables & layout · open sourceStar on GitHub→Sponsored
Fern is aimed at AI R&D work, not general chat. Mirendil lists 6 target uses:
Working across larger codebases
Agentic coding that runs commands until tests pass
Research code: training loops, data pipelines and evaluations
Debugging training runs from long logs and stack traces
Reading full agent or training traces end to end
Auditing pretraining shards for duplicates, contamination and junk
How was Fern built?
Fern sits on Mirendil-Base, which Mirendil says one researcher pretrained in 3.5 months. Total compute was under $10M, including experiments, agent token spend and data processing. AI agents wrote over 99% of the code. The researcher chose what to try and when to intervene.
Training data came only from public sources: GitHub, public code, web pages and public datasets. Mirendil says it used no distillation, no synthetic data and no licensed data. The code was written from scratch on PyTorch and JAX.
Fern is the smallest of 3 planned models. Mirendil-Base-Large (0.5 to 2T parameters) is over 80% through pretraining. Mirendil-Base-XL (2 to 10T) is over 20% through.
Does longer context actually help?
Mirendil tested next-token prediction on its own repositories, concatenated into a 42.9 MB stream. It measured cross-entropy in bits per byte, where lower is better. Window sizes here are measured in bytes, not tokens.
Fern’s loss fell from 0.246 at a 32K-byte window to 0.155 at 16M and 23M bytes. The other models were scored up to about 1M tokens, their advertised limit. At a 1M-byte window, Kimi-K3 scored 0.140 and Fern 0.165. So Fern is not the best predictor at equal context. Its advantage is that the curve keeps going past where others stop.
How does Fern score on RULER?
RULER is a standard long-context benchmark. Mirendil ran 5 RULER tasks on Mirendil-Base, the pre-post-training model. It used greedy decoding, thinking off and exact string matching.
RULER task32K1M4M10M
Single needle100100100100
Multi-key retrieval100989890
Multi-key, all needles100100100100
Variable tracking1009575100
Multi-hop QA (HotpotQA)65656045
Retrieval holds up well. Multi-hop reasoning drops to 45% at 10M. Each 10M cell uses only 20 examples, versus 100 elsewhere. Mirendil says broader long-context results are coming.
How does Fern 0.1 do on agentic benchmarks?
Mirendil compared Fern 0.1 with open-weight and frontier models. Scores for other models come from model cards, leaderboards or Mirendil’s own runs.
Benchmark (pass %)Fern 0.1Kimi-K3GLM-5.3Claude Opus 5.5
DeepSWE v1.144.267.366.974.2
Terminal Bench 4.06.112.641.966.4
SWE Atlas QnA53.266.159.466.4
SWE-Pro v2 Hard76.588.284.3100.0
SWE-Bench Verified73.493.495.498.2
Fern 0.1 beats Inkling and Nemotron 3 Ultra on 4 of 5 tests. It ties Beam on SWE-Pro v2 Hard at 76.5. It trails Beam on SWE-Bench Verified, 73.4 versus 81.6. Every frontier and large open model in the table leads it.
How does Fern compare with similar models?
FeatureFern 0.1Kimi-K3GLM-5.3
DeveloperMirendilMoonshot AIZ.ai
Total params0.1 to 0.5T2.8T753B
Active paramsNot disclosed104B~40B
ContextUp to 10M tokens1,048,576 tokens1,048,576 tokens
InputText, codeText, image, videoText
LicenseNot disclosedKimi K3 LicenseGLM-5.3 License
WeightsNot releasedOpenOpen
AccessEarly access requestOpen weights, APIsOpen weights, APIs
SWE-Bench Verified*73.493.495.4
Bits per byte at 1M bytes*0.1650.1400.170
*Figures from Mirendil’s announcement charts.
Key Takeaways
Fern 0.1 supports up to 10M tokens, about 23 MB of code.
Bits per byte keeps falling to 0.155 at a 23M-byte window.
RULER single needle stays at 100% from 32K to 10M tokens.
Agentic coding lags: 6.1% on Terminal Bench 4.0.
Base model built by 1 researcher plus agents for under $10M.
Michal Sutter+ postsBio ⮌
Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.
Michal SutterOpenAI Launches GPT-6.1 Terra, Halving GPT-6.1 Sol’s API Price for Agent Work
$uv pip install omni-parse-bench✓text✓tables✓layout→ scored per pageStar on GitHub→Sponsored
RELATED ARTICLESMORE FROM AUTHOR
OpenAI Launches GPT-6.1 Terra, Halving GPT-6.1 Sol’s API Price for Agent Work
Nace AI Releases Drex for Search for AI Agents: 92.05% on BrowseComp-Plus With 12.6 Search Calls
Nanbeige Releases Nanbeige4.3-33B, a 33B Open Agentic Coding Model