AI Trends

Hugging Face Releases Qwen3.8-27B Harness Adapters, Improving Agentic Coding Performance and Efficiency

Hugging FaceUpdated Nov 20 21:52

AI summary

Hugging Face has released LoRA harness adapters for Qwen3.8-27B tailored to pi, Hermes Agent, and OpenCode, and merged the three into a super-adapter capable of running across different frameworks. Each of the three specialized adapters outperformed the base model in Terminal-Bench 2.1 and SWE-bench Pro evaluations within its respective framework. With thinking mode disabled, the super-adapter increased average pass@1 from 39.5% to 49.0%, while reducing the number of tool calls needed to solve tasks jointly by 53–73%. Training used in-framework RL, TRL AsyncGRPOTrainer, and 871 agentic coding tasks. The related adapters, GGUF files, RL tasks, and safeagents plugin have all been made publicly available.

Why read this

SelectedAI score 71

The article presents a complete workflow for reinforcement learning within real-world agent frameworks and uses cross-framework evaluations to demonstrate the performance and cost trade-offs of specialized and merged adapters.

Topics

Tags

Body · AI translation

TL;DR

We trained one LoRA adapter for Qwen3.8-27B per agent harness (pi, Hermes Agent, OpenCode), with RL inside the harness itself.

Datasets, training, and agent rollouts all happen on the Hub in Jobs using TRL's AsyncGRPOTrainer, OpenEnv, and Harbor.

Each harness adapter beats the base model under its own harness on both benchmarks, in both performance and efficiency.

The three LoRAs are also merged into a single super-adapter that runs unchanged under pi, Hermes Agent, and OpenCode.

All artifacts (adapters, GGUFs, RL tasks, and the safeagents plugin) can be browsed through this Hugging Face collection.

Table of Contents

Training inside agent harnesses

Generating custom RL tasks on Hugging Face

Testing the filters with short training runs

AsyncGRPOTrainer for more efficient GRPO training

Training setup

Super-adapter: merging all adapters into one

Final results

Open question: What did the model really learn to do?

Closing words and future directions

Training inside agent harnesses

We leverage the Harbor integration in OpenEnv and TRL’s OpenEnv guide to train the model inside the harnesses we chose for the final training. We built ready-to-use Docker images for all our RL tasks, with the target harnesses already pre-installed. In our final Docker images, we use Pi 1.5.2, OpenCode 2.2.7, and Hermes Agent v0.22.6. The diagram below from the aforementioned blog post clearly explains the entire protocol:

Most RL frameworks train a reimplementation of the agent loop: the agent is written against the framework's own interface, such as verl's AgentLoop, SkyRL-Gym environments, rLLM's agent and environment classes, slime's custom rollout functions, or ART's client inside your own Python agent. Here, the harness runs exactly as shipped, and the trainer learns from the exact tokens it produced, recorded by a proxy between the harness and vLLM. Recent work is moving in the same direction: Agent Lightning v1.0 keeps the deployment-time harness in the loop through a proxy ("harnessed agentic RL"), and SkyRL's Harbor integration trains on Harbor trials, converting their chat histories into tokens for the trainer.

Generating custom RL tasks on Hugging Face

For this recipe, we created a custom dataset for agentic coding using the Hub, OpenEnv, and Repo2RLEnv. We combined task retrieval from existing RL environments, task generation with Repo2RLEnv, task validation with OpenEnv, and sampling into a single data pipeline. This gave us the highest-quality set of validated and relevant tasks for improving Qwen3.8-27B on agentic coding. In this section, we’ll walk through how we built it so that you can try it in your own domains.

RL training needs tasks the agent can attempt, a verifier that scores each attempt, and evidence that the verifier is correct. A verifier that accepts wrong answers teaches the wrong behavior. A task that is already feasible will not teach the model anything.

This section describes how we built the task set from three sources: two on the Hugging Face Hub and a third generated with Tasksmith, a pipeline in Repo2RLEnv. Every task passes an eligibility screen with openenv validate and the same difficulty check against the base model, Qwen3.8-27B.

Three sources

Each task uses the Harbor format, which Terminal-Bench 2.1 also uses. A broad search of the Hub for Harbor environments returned about 59,000 tasks, but they included duplicates, evaluation sets, and tasks marked as not for training. We filtered them down to just two Hub sources built for training and generated a third set ourselves:

SWE-smith (Python, Go) injects bugs into real Python and Go repositories, using either an LLM rewrite or a mutation such as a flipped operator. The tests that catch each bug become the task's tests. It adds volume.

SWE-Gym, packaged as Harbor tasks in TaskTrove, turns real GitHub issues and their merged fixes into tasks. It adds real-world bugs.

Tasksmith generates tasks from merged pull requests. An LLM agent writes the instructions and tests, and Tasksmith checks the result: the reference and a valid alternative must pass, while the unfixed code and a known-wrong solution must fail.

We used Tasksmith to target gaps. We mapped the skills tested by our target benchmarks (Terminal-Bench 2.1, the hard subset of SWE-bench Pro, and DeepSWE), then chose pull requests that exercise them, such as HTTP streaming, filesystem operations, and serialization. From 30 merged pull requests, Tasksmith produced 24 tasks across 10 Python repositories that passed its checks, 17 of them on a second run. Its checks do not replace ours, so these tasks go through the same pipeline as the rest.

Validation with openenv validate

A metadata screen runs first. It retains tasks from training splits with permissive licenses. It removes duplicates, tasks marked as not for training, the 253 tasks in our target benchmarks, and any task from a repository used by those benchmarks. We then ran the semantic checks from openenv validate on every task in fresh trials:

The reference solution earns the maximum reward ( semantic.oracle_max ).

Doing nothing scores clearly lower ( semantic.floor_gap ).

No canary scores more than doing nothing ( semantic.canary_floor ).

A replay of the reference gives the same reward ( runtime.episode_determinism ).

Each trial ran as its own Hugging Face Job, which let us validate dozens of tasks at once. openenv validate offers the same option for any Harbor task:

858 of 1,127 Hub candidates passed. Wrong-answer controls caught defects, including a Tasksmith task that accepted an incomplete implementation and a Terminal-Bench 2.1 verifier (regex-log) that accepted known-wrong answers. Most failures were infrastructure problems, such as images that did not start. Failed tasks stay in a rejected split with the failing check recorded.

Difficulty check with the base model

A valid task can still teach nothing. GRPO (group relative policy optimization) scores each attempt against the other attempts at the same task, so a task where every attempt passes, or every attempt fails, gives no learning signal. Before training, Qwen3.8-27B attempted each task 4 times, or 8 times for a stratified sample of six tasks per repository.

Of 442 scored tasks, Qwen3.8-27B solved 151 every time and 120 never. The 171 tasks it solved only sometimes form the qualified set. The figure below shows how the scores pile up at both ends and how they differ by source. The model solves most Go tasks every time, fails every attempt on more than a third of the SWE-smith Python tasks, and lands in the middle on 73% of the SWE-Gym tasks.

Qwen3.8-27B results on 442 scored tasks. Green marks the qualified tasks, which the model solves on some attempts but not all.

This example shows how we filtered and validated 1,127 tasks to 278 which we included directly in the final dataset. The pipeline then took the remaining 522 rejected tasks, repaired and complexified them with Tasksmith and repeated validation.

Testing the filters with short training runs

The pipeline applies two filters: validation removes broken tasks, and the difficulty check keeps tasks the base model solves only sometimes. This section tests whether those filters change what the model learns. We trained four short runs of up to 50 steps, each on tasks from a different stage of the pipeline, and compared them with the base model on Terminal-Bench 2.1.

Each run trains Qwen3.8-27B with LoRA and GRPO, using TRL's AsyncGRPOTrainer , on 32 tasks. All four use the same agent harness (OpenCode with up to 48 model calls), the same settings and one 8×H200 Hugging Face Job, and the reward is the verifier's pass or fail. The task sets differ in how they were filtered:

Unvalidated : 24 accepted tasks plus 8 that failed validation.

Validated : 32 accepted tasks, chosen without regard to difficulty.

Unqualified : 32 accepted tasks that the base model always or never solves.

Qualified : 32 accepted tasks that the base model solves on some attempts.

The figure below shows how the tasks impacted model learning. The table lists each run's steps and the checkpoint we evaluate. The chart tracks training reward across subsets of tasks with differing validation status.

The dataset

harness-hub/agentic-coding-envs holds 871 tasks from 35 repositories. Hub tasks are capped at 40 per repository, so no single project dominates. The table below shows what each source contributes. The dataset has two views: every accepted task, and the qualified subset for GRPO training. Rejected attempts stay in a separate config.

AsyncGRPOTrainer for more efficient GRPO training

We leveraged the stable version of AsyncGRPOTrainer from TRL, with the main advantage of the trainer being that it does not wait until all rollouts are completed before moving to the next training optimization step.

For more detailed reading on the trainer and its building blocks, please refer to the AsyncGRPO blogpost .

Training setup

To obtain our final harness adapters, we trained LoRA adapters on top of Qwen/Qwen3.8-27B using TRL's AsyncGRPOTrainer . The adapters were attached to the model using the all-linear flag. To maximize the training efficiency, thinking mode was turned off during rollout stages, and we evaluated our final adapter with and without thinking for broader comparison.

The final reward function consists of the verifier's correctness, combined with a small tool-efficiency bonus (0.1, derived from early experiments from the previous iteration of harness adapters which showed that 0.1 worked best).

We performed a small LR sweep on 50 steps using Termgrade environments (1e-5, 5e-5 and 1e-4) and noticed that 1e-5 barely moved training correctness (from 0.52 to 0.50), 1e-4 collapsed the rollouts instead of improving them (turns per rollout moved from 18 to 6, with entropy rising), and only 5e-5 clearly improved (from 0.49 to 0.60 by step 40). Longer 5e-5 runs then showed signs of instability (collapsing rollout lengths, rising entropy), so we settled on 2e-5 for the final runs.

Training infrastructure

Each harness adapter was trained using NVIDIA 2xB200 GPUs, with one GPU allocated for the vLLM server and the other allocated for the training process. We first performed some stress testing to maximize the number of RL tasks that could be trained within a single optimization step. For that, we “maxed out” the --token-budget argument from AsyncGRPOTrainer to the limit of our setup. The chart below shows how the number of groups per step evolved for each harness (blue: Pi, green: OpenCode, and orange: Hermes).

As the model improves during training, it can fit more and more groups into each training step, diversifying the signals the model receives at each optimization step.

Final mixture

The final data mixture consists of 750 tasks from harness-adapters RL environments and 300 tasks from termgrade. To maximize the diversity of tasks sampled during training, we slightly customized the data-sampling strategy, as explained below:

Before training, the tasks are placed in a fixed order that interleaves them by family (i.e., by repository for our generated RL environments. For Termgrade, we simply sorted them by name, since each task is considered its own family), so every training step mixes as many repositories as possible while preserving the mixture’s distribution.

safeagents in the loop

We also integrate the safeagents tool to block any model misbehavior during training. The final plugin, which consists of regex and pattern rules specific to the RL tasks and test paths, is hosted at harness-hub/qwen-safeagents-plugin and can be used out of the box with the latest version of safeagents by anyone who wants to reproduce the training setup.

The number of tools blocked by safeagents decreases consistently during training, indicating that the model becomes “safer and safer” after each optimization step. Below is a diagram of safeagents-blocked calls per 1,000 tool calls, by training step, for each harness adapter.

Super-adapter: merging all adapters into one

Merging models often combines their capabilities while preserving most of what each one can do. Once the three harness adapters had been trained, we asked two questions: Does merging them reduce accuracy? And can a single “super-adapter” be accurate and efficient under every harness?

We began by comparing the weight updates of the three adapters layer by layer. The cosine similarity between any two of them is about 0.01–0.02: each adapter pushed the model in a nearly orthogonal direction during training. That is the best-case scenario for merging, since the updates barely interfere with one another.

We concatenate the three LoRA adapters along the rank dimension (PEFT’s combination_type="cat"). The merged adapter has rank 96 (3 × 32) and applies exactly w · (ΔW_pi + ΔW_hermes + ΔW_opencode). Prior work on merging (task arithmetic, TIES, KnOTS) finds that scaling each update below 1 works best, so we compared three weights across all 89 Terminal-Bench 2.1 tasks, under each harness, using the step-120 adapters:

⅓ (the average): each harness retains only one-third of its own update.

½: each update at half strength.

1 (the plain sum): every update applied in full.

The plain sum over-applies the updates. The agents become very short (13 tool calls per task, compared with 49 for the base model), and while Hermes peaks at 62.8%, pi falls back to the base model’s level. The average dilutes each harness’s own update. ½ provides the best balance, so we built the final super-adapter from the released checkpoints (pi step 160, hermes step 160, opencode step 150) at weight ½.

Adapter merging evaluation results

Under each harness, on the tasks graded in every run (50–51 on SWE-bench Pro and 82–85 on Terminal-Bench 2.1), with one attempt per task (the efficiency gain relative to the base model is shown in parentheses):

“Thinking” is the chat template’s thinking mode at its default effort (xhigh). The specialists were trained and served with thinking turned off.

With thinking off, the super-adapter improves on the base model across all six harness–benchmark pairs (mean pass@1: 39.5% → 49.0%; 71 tasks newly solved versus 34 lost, p < 0.001).

Despite not being trained at all with thinking on, it is the strongest model we evaluated in thinking mode. Compared with the base model using thinking at the same effort, it wins on five of the six pairs and ties on the sixth (mean: 55.5% → 63.5%; 67 newly solved versus 37 lost, p = 0.004). The largest gains are on SWE-bench Pro: +16 points under hermes and +20 under opencode. Under pi on SWE-bench Pro, where thinking alone already reaches 74%, it ties.

Use the specialist harness adapters when thinking is off. Each harness’s own adapter leads on four of the six pairs, with the largest margins under pi (64.0% versus 46.0% on SWE-bench Pro and 56.5% versus 48.2% on Terminal-Bench). Splitting one model across three harnesses costs pi the greatest share of its gains. If you use only one harness and keep thinking off, choose its specialist.

The super adapter is also much more efficient. On the tasks both the base model and the super-adapter solve, with thinking off, it makes 53–73% fewer tool calls and generates 44–60% fewer tokens per task, with fewer tool calls on 111 of 130 tasks. With thinking on, it still cuts tool calls by 31–48% and tokens by 24–43% on SWE-bench Pro. On Terminal-Bench, tool calls drop by 19–25% and token counts stay about the same.

The ½ weight was chosen on Terminal-Bench 2.1 with the step-120 adapters, so the Terminal-Bench numbers are slightly optimistic; SWE-bench Pro played no part in the choice. Full paired results are in the super-adapter's model card.

Final results

This section reports what happened during training, then how the adapters score on held-out benchmarks inside and outside their own harnesses.

Training results

Training ran for a cumulative total of 598 steps, with a total of 78,114 samples trained, which consists exactly of 167M trained tokens.

During training, we see a clear signal that the model learns to solve tasks in fewer and fewer steps, with no clear sign of saturation. Despite having some clear increasing trend sometimes, training correctness seems to overall stay flat. This can be explained by the fact that the model meets new tasks during training, but the number of groups that fit in a step’s token budget more than doubles as trajectories get shorter. The accuracy gains show up on the held-out benchmarks in the section below.

On the other hand, safeagents was able to block multiple suspicious attempts. Across the three training runs, a total of 1.3M tool calls have been checked, and a total of 647 tool calls have been blocked.

Evaluation results

This section covers the evaluation setup, the in-harness results, transfer across harnesses, and an out-of-distribution check.

Evaluation setup

We evaluate each adapter inside the harness it was trained in, on two agentic coding benchmarks: all 89 tasks of Terminal-Bench 2.1 (1 hour and 100 agent steps per task) and the 51 hardest tasks of SWE-bench Pro (2 hours and 100 agent steps). We cap the maximum number of tokens per task to be 131072. We do not tune the agent settings and use the default setup for all harnesses. For example, Pi does not keep previous reasoning turns while OpenCode and Hermes do, which mainly affects the SWE-bench Pro results due to rapid context overflow. Therefore you cannot compare the results across different harnesses (using Pi does not necessarily lead to better results than Hermes or OpenCode), but it is fair to compare results within the same harness and draw conclusions.

Every rollout runs through the same Harbor and OpenEnv stack as training, in HF sandboxes, with one attempt per task at temperature 0.8 and thinking off. Each LoRA checkpoint is merged into the base weights and served alone with vLLM on a single GPU, at the same concurrency as the base model's eval.

For all the evaluation benchmarks, we report the pass@1 as well as the number of tool calls, total turns, for all tasks and for solved tasks.

For openness, all model trajectories for final evaluations are pushed in this public bucket: harness-hub/harness-adapters-trajectories.

Final evaluation results

All three specialist adapters beat the base model under their own harness, on both benchmarks. On SWE-bench Pro (hard 51), pi goes from 39.2% to 64.7% (+25.5 points: 15 tasks newly solved vs 2 lost, p = 0.002), hermes from 29.4% to 43.1%, and opencode from 35.3% to 51.0%. On the full Terminal-Bench 2.1, pi gains +16.3 points (16 vs 2, p = 0.001) and hermes +14.1 (19 vs 7, p = 0.03). Opencode gains +6.0 overall, and +13.6 on the 66 tasks that played no part in choosing its checkpoint (12 vs 3, p = 0.035).

The adapters also solve tasks much more efficiently: on the tasks both they and the base model solve, they make 56–72% fewer tool calls per task on SWE-bench Pro and 53–70% fewer on Terminal-Bench 2.1. The single super-adapter beats the base model under all three harnesses too (+6 to +18 points on SWE-bench Pro, +5 to +10 on Terminal-Bench 2.1), with just as large a cut in tool calls (53–73%). It keeps less of each specialist's gain with thinking off, most under pi, but with thinking on it beats the base model thinking at the same effort on five of six harness–benchmark pairs (see the super-adapter section). Each model card has the full paired results.

Does the efficiency and performance ability transfer across harnesses?

To answer this question, we also evaluated the pi harness adapter under Hermes Agent and OpenCode.

Under OpenCode it matches OpenCode's own specialist on Terminal-Bench 2.1 (44.8% → 54.0%, against 46.4% → 52.4%). Under Hermes it keeps about two-thirds of the specialist's gain (46.3% → 56.1%, against 44.7% → 58.8%), with the same ~45% cut in tool calls in both. Therefore it seems that the newly learned capabilities are transferred across harnesses.

Evaluation on out-of-distribution tasks and setup

What happens if we evaluate on out-of-distribution tasks, outside the harnesses with which the adapters were trained?

We ran all five models on 300 AutomationBench tasks, consisting of business automation workflow evaluations, using AutomationBench's own agent loop instead of Pi, Hermes, or OpenCode. This allowed us to cross-check two different questions: Can the model solve problems outside SWE-related tasks? What happens when the model is placed outside the harness with which it was trained?

The results show that the adapters remain close to the base model. Pi and OpenCode are slightly ahead (27.0% and 25.3% pass@1 versus 23.0%), the merged adapter is roughly level (23.7%), and Hermes is a few points behind (17.7%). The differences are within the noise margin, except perhaps for Hermes. Therefore, RL within the harnesses neither clearly transfers to this very different setting nor harms the model on it.

Open question: What did the model really learn to do?

We appreciate the feedback from users and the community. Initial community testing and results appeared extremely positive overall, and we hope to continue in this direction in the future. We also saw some surprising results, such as users reporting that the harness adapters work very well with Qwen3.8 derivatives such as Swift-1.5.

This is an open question that we considered after release. After carefully inspecting the trajectories generated during benchmark evaluation, it turns out that, in non-thinking mode, the model tends to generate some short text before proposing a tool call, unlike the base model, which generates tool calls with barely any preceding text.

This phenomenon can be viewed as a kind of “reasoning leakage,” in which the model learns to “escape” from its non-thinking version into its thinking version by generating text that can sometimes resemble reasoning traces (e.g., “...wait?” or “...let me see”). A recent paper also describes the behavior of LLMs during the RL stage very well.

Does this invalidate our models? We do not think so, because the final results remain valid: under exactly the same conditions—a fixed token budget—our adapters substantially outperform the base non-thinking model and match the performance of the thinking version while using tokens much more efficiently. This has been further confirmed by positive feedback from the community.

However, this highlights the need for proper follow-up work. We will try to reapply the same pipeline to pure thinking models and explore hybrid thinking/non-thinking training to better understand the behavior of RL training on hybrid thinking models. We will share our next findings—and harness adapters—with the community once we obtain conclusive results.

Closing words and future directions

Throughout this work, we demonstrated that training a model with RL inside the target harness leads to more efficient and therefore more accurate final models when evaluated under the same conditions. This observation aligns with and builds upon several previous works by Hugging Face teams, such as multi-harness-RL and the early iterations of harness adapters.

In the future, we will explore applying this recipe to more models while studying training strategies for hybrid thinking/non-thinking models in greater depth.

We are also openly sharing the complete recipe and pipeline with the community so that everything can be reproduced. We invite everyone to build their own RL tasks derived from their own codebases or any relevant materials, and to apply the same training recipe and ideas to their preferred model and harness to create more efficient agentic models. We hope this blog post will serve as a starting point for builders to create and share their own harness adapters in the future.

Models mentioned in this article 4

Qwen/Qwen3.8-27B

harness-hub/Qwen3.8-27B-opencode-adapted-GGUF

harness-hub/Qwen3.8-27B-super-adapter

harness-hub/qwen-safeagents-plugin

Datasets mentioned in this article 6

SWE-bench/SWE-smith-go

SWE-bench/SWE-smith-py

ai-and/termgrade-environments

harborframework/terminal-bench-2.1

harness-hub/harness-adapters

open-thoughts/TaskTrove

Spaces mentioned in this article 2

The ultimate guide to multi-harness RL

Train open models with RL inside real agent harnesses

AsyncGRPOTrainer is stable: async RL in TRL

Explore async reinforcement learning with AsyncGRPOTrainer

Papers mentioned in this article 5

Editing Models with Task Arithmetic

Resolving Interference When Merging Models

Model merging with SVD to tie the Knots

Agent Lightning v1.0: Towards Harnessed Agentic RL

Collections mentioned in this article 1

Qwen3.8-27B Harness adapters

More Articles from our Blog

Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL

Training a coding model to paint watercolours with TRL and OpenEnv

Models mentioned in this article 4

Source

Hugging Face