Infini-AI-Lab open-sources the Hermit harness adapter framework
Infini-AI-Lab has released the open-source framework Hermit, which adapts AI agents by training external harness adapters while keeping the model and harness frozen; v0.1.0 was released on October 25, 2026. The framework is implemented in Python under the Apache-2.0 license and can be installed via pip install hermit-harness, and it includes five evolution methods: GEPA, SkillOpt, MetaWorld Model, Harness², and Continual Harness. On GLM 5.3 Flash, the MWM adapter achieved 0.795 accuracy on the CL-Bench database exploration task, higher than the in-context baseline of 0.580, but most results are self-reported data from a single run.
Why read this
AI score 67
The original text presents the adapter training mechanism, 5 built-in methods, and self-reported benchmark comparisons, making it easy to assess the adaptation approach that leaves the model and harness unchanged.
Topics
Body
Add as a preferredsource on Google
Infini-AI-Lab has released Hermit, an open-source harness adapter framework for AI agents. Hermit is a framework from Infini-AI-Lab that trains an external ‘harness adapter’ while the agent’s model and harness stay frozen. The first release, v0.1.0, shipped on October 25, 2026. CMU postdoc Haizhong Zheng announced it on X on October 27. For developers, the pitch is simple: adapt Codex or Claude Code agents to new tasks without fine-tuning or forking the harness.
TL;DR
Footprint: A Python framework. No parameters of its own. Apache-2.0, pip install hermit-harness.
Runs on: The offline Quick Start needs no GPU, model account or dataset. Real runs use any API backend or a local vLLM, SGLang or Ollama server.
Performance: Reported gains over no-adapter baselines on OfficeQA, CL-Bench and ServeLearnBench, all with GLM 5.3 Flash.
Best: MWM adapter hits 0.795 accuracy on CL-Bench database exploration, vs 0.580 for the in-context baseline.
Worst: The prompt-only adapter scores 0.470 on the same venue, below that 0.580 baseline.
Bottom line: Best: one training loop for 5 harness evolution methods. Worst: most results are single runs, self-reported.
What is a harness adapter?
A harness adapter is a trainable external harness. It works like a plugin with training support built in. A model adapter, such as LoRA, adapts a model. A harness adapter adapts an existing agent instead.
TEXTTABLELAYOUTIs your PDF parser actually getting it right?omni_parse_bench · scores text, tables & layout · open sourceStar on GitHub→Sponsored
The adapter can hold instructions, skills, memory, tools, executable code and agent coordination logic. It can even be a whole repository. Its specification says which parts are learnable and how updates get checked. The base model and harness never change.
The team gives 3 reasons for this design. Many harnesses are closed-source. Editing open ones can break carefully tuned component interactions. And a learned adapter can move between compatible models and harnesses.
How does Hermit train an adapter?
Hermit uses an RL-style training loop. A Trainer runs a batch of tasks through the acting agent with the current adapter. A judge scores the outputs. An Evolver reads the trajectories and feedback, then proposes a new adapter. The Trainer checks it against the template and commits it as the next version.
Every adapter version lives in a git tree. You can inspect, roll back or ship any version. Learner programs can run sandboxed, which requires Linux with bubblewrap.
Five evolution methods ship today: GEPA, SkillOpt, MetaWorld Model (MWM), Harness² and Continual Harness. ACE and ReHarness are listed as coming soon. You can also write your own Evolver.
What are the benchmark results?
All runs below use GLM 5.3 Flash as the acting model. The numbers come from the repo’s example READMEs and are self-reported.
OfficeQA (recipe) asks 172 test questions about U.S. Treasury Bulletins, scored by exact match:
GEPA and SkillOpt train on 50 questions and are tested frozen. Continual Harness and Harness² instead learn while streaming the test set once. The team reports a +7.6 point gain for Harness² over its reference run, with a 95% interval of +1.7 to +13.4.
CL-Bench (database, poker) tests stateful environments from the Continual Learning Bench paper. On database exploration, MWM reaches 0.795 accuracy. The benchmark’s in-context baseline scores 0.580. On exploitable poker, MWM wins 3.08 big blinds per hand, vs 2.51 for in-context learning.
ServeLearnBench (recipe) streams 624 retail support requests under a hidden, shifting policy. MWM scores 45.3 on hidden-policy tests. The best learning method on the leaderboard, CH, scores 28.7. An oracle given the policy scores 89.2.
How does Hermit compare with other agent optimizers?
FeatureHermitGEPAACEAgent Lightning
What it trainsExternal harness adapterAny text parameter (prompts, code)Context playbookPrompts and model weights
Changes model weightsNoNoNot disclosedYes, via RL or SFT
Methods5 built in, custom EvolversReflective evolution, Pareto selectionGenerator, Reflector, CuratorRL, prompt optimization, SFT
Deploy targetsCodex, Claude Code, Pi; plugin exportDSPy (dspy.GEPA)Not disclosedLangChain, OpenAI Agents SDK, AutoGen, CrewAI and more
LicenseApache-2.0MITApache-2.0MIT
Key result0.795 on CL-Bench database (self-reported)AIME 2025: 46.6% to 56.6% (GPT-4.1 Mini)+10.6% on AppWorldNot disclosed
How do you install and deploy it?
Install from PyPI with pip install hermit-harness, or clone the repo for examples. The CLI covers check, train, eval, deploy and convert. Models are referenced as / . Supported backends include Fireworks, OpenAI, Anthropic, DeepSeek, vLLM, SGLang and Ollama.
hermit deploy writes a workspace for Codex, Claude Code or Pi. hermit convert exports a trained adapter as a native plugin for Codex, Claude Code, Pi or Hermes. A finished Lean collaboration adapter coordinates 5 Codex agents proving one theorem together.
Key Takeaways
Hermit trains an external adapter; model and harness stay frozen.
5 methods ship: GEPA, SkillOpt, MWM, Harness², Continual Harness.
MWM reaches 0.795 on CL-Bench database vs 0.580 in-context.
Adapters deploy to Codex, Claude Code and Pi, or export as plugins.
Results are self-reported, mostly single runs, with GLM 5.3 Flash.
Asif RazzaqWebsite | + postsBio ⮌
Asif RazzaqPrime Intellect Releases Prime Mega MoE: Trainable Fused MoE Kernel, 3.11x Faster Forward Pass
Asif RazzaqNVIDIA Releases Nemotron 3.5 Transcribe: An Open 30B MoE Model for 40-Language Transcription with ~4B Active Parameters
Asif RazzaqPolymathic AI and Princeton Researchers Release FlashMap, a Flow Map Drafter for 6.14x Faster LLM Decoding
$uv pip install omni-parse-bench✓text✓tables✓layout→ scored per pageStar on GitHub→Sponsored
RELATED ARTICLESMORE FROM AUTHOR
Prime Intellect Releases Prime Mega MoE: Trainable Fused MoE Kernel, 3.11x Faster Forward Pass
NVIDIA Releases Nemotron 3.5 Transcribe: An Open 30B MoE Model for 40-Language Transcription with ~4B Active Parameters