Hugging Face 发布 14.8B 开源 DNA 语言模型 Carbon-X

Add as a preferredsource on Google

Hugging Face has released Carbon-X, an open DNA language model with 14.8B parameters. Carbon-X is a decoder-only model from Hugging Face Bio. It reads raw DNA in a chat and answers in English. It predicts whether mutations are benign or pathogenic, flags regulatory regions, classifies organisms and predicts next bases. The weights ship under Apache 2.0, so labs can keep sensitive sequences on their own GPUs. Hugging Face built it with Zhongguancun Academy and TIGEM / University of Naples Federico II.

TL;DR

Size: 14.8B parameters (dense, no MoE), 40,960-token context.

Runs on: 1 x 80 GB GPU in bfloat16 via vLLM or Transformers. Official GGUF: Q8_0 is 15.8 GB, Q4_K_M is 9.6 GB.

Performance: 66.7% average on 5 DNA benchmarks, best of every open model tested (self-reported).

Best: 74.9% on BioReason Coding, ahead of every model tested, open or closed.

Worst: 29.8% on GenomeQA splice-site questions.

What is Carbon-X and how does it work?

Carbon-X starts from Qwen3-14B. DNA and text share 1 token stream in 1 decoder. There is no separate DNA encoder, unlike earlier work such as BioReason.

TEXTTABLELAYOUTIs your PDF parser actually getting it right?omni_parse_bench · scores text, tables & layout · open sourceStar on GitHub→Sponsored

The key structure piece is a hybrid tokenizer. It keeps Qwen3’s BPE vocabulary for text and adds 5,515 DNA tokens. These include 4,096 non-overlapping 6-mers, 1,364 tail tokens and 3 markers. One DNA token covers 6 base pairs. DNA runs of 6 or more A/C/G/T bases are tagged automatically.

The backbone is a standard Qwen3 stack. It has 40 layers, hidden size 5120 and GQA with 8 KV heads.

How was Carbon-X trained?

Training ran in four stages, per the model card:

Embedding warmup: about 10B tokens with the transformer frozen, so only the new DNA embeddings learn.

Mid-training: 302B tokens at 8,192 sequence length. The mix is 55% raw DNA, 20% English, 15% biology text and 10% instruction data.

SFT: 3 epochs over 20 dataset subsets, with responses checked by deterministic verifiers built with domain experts.

RL: asynchronous GRPO with verifiable rewards over 2 rounds, using TRL and vLLM.

As per the release post, learning rate was the hardest knob. High rates learned DNA but forgot English. A lower rate, a 14B model and a long run fixed the trade-off. RL applied directly to Qwen3-14B never beat the 50% chance baseline on FrontierVEP.

How does Carbon-X perform on DNA benchmarks?

All scores are self-reported by Hugging Face. Carbon-X ran with thinking disabled. Competitors ran at their best reasoning effort.

BenchmarkCarbon-XGPT-6.1 Sol (Max)Opus 5.5 (Max)Kimi K3 (Low)

BioReason Coding74.971.070.353.9

BioReason Non-SNV85.191.794.865.3

GenomeQA63.667.262.252.0

NextBase39.432.91.130.6

FrontierVEP70.474.976.250.1

Average66.767.560.950.4

Carbon-X sits 27 points above Qwen3-14B and 0.8 points below GPT-6.1 Sol. It leads on coding-variant effects and next-base prediction. Frontier models still win on complex variants and single-base variants.

Refusals shape these numbers. On NextBase, Opus 5.5 falls from 17.7% at low effort to 1.1% at max effort as refusals rise. Gemini 4 Argon drops from 58% to 38% on FrontierVEP as effort increases.

Carbon-X also keeps general skills: 79.3% prompt-level strict accuracy on IFEval.

What does Carbon-X cost to run?

Hugging Face prices Carbon-X at measured throughput on an H200 at $5 per hour. On that basis, it reports about 1/1,800 of GPT-6.1 Sol’s inference cost per problem. Scoring the 2.4 million ClinVar variants of uncertain significance would cost about $300. The same job costs $32,000 to $4 million on frontier models.

Carbon-X also works as a sub-agent. Kimi K3’s DNA index rose from 49.4% to 60.5% when it consulted Carbon-X.

How do you deploy Carbon-X?

vLLM: vllm serve HuggingFaceBio/Carbon-X --trust-remote-code on 1 x 80 GB GPU.

Transformers: v5 or later, with trust_remote_code=True for the DNA tokenizer.

llama.cpp: the official Carbon-X-GGUF repo needs llama.cpp b11742 or newer. Older builds tokenize DNA wrongly.

Agents: the repo ships an OpenAI-style tool, an Inspect agent and a carbon-dna skill for coding agents.

Keep enable_thinking=False. The model was post-trained and evaluated that way.

How does Carbon-X compare with other DNA and general models?

FeatureCarbon-XQwen3-14BGPT-6.1 SolEvo 2 40B

DeveloperHugging Face + partnersAlibaba QwenOpenAIArc Institute

Parameters14.8B14.8BNot disclosed40B

Context40,960 tokens32,768 native, 131,072 with YaRNNot disclosedUp to 1M sequence length

InputDNA + EnglishTextText (general-purpose)DNA only

Answers in EnglishYesYesYesNo

LicenseApache 2.0Apache 2.0Proprietary, API onlyApache 2.0

DNA average (HF eval)66.7%~40.0%67.5% (Max)Not evaluated

BioReason Coding74.9%Not disclosed71.0%Not evaluated

PriceOpen weightsOpen weights$2 / $10 per 1M input / output tokensOpen weights

Key Takeaways

Carbon-X is a 14.8B open model that reads DNA and answers in English.

It averages 66.7% on 5 DNA benchmarks, 0.8 points behind GPT-6.1 Sol.

It leads every tested model on BioReason Coding (74.9%) and NextBase (39.4%).

It runs on 1 x 80 GB GPU, or a 15.8 GB Q8_0 GGUF.

Splice-site accuracy is 29.8%, and it is not clinically validated.

Asif RazzaqWebsite | + postsBio ⮌

Asif RazzaqResearchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context Prefill Up to 11.3x Faster

Asif RazzaqNVIDIA Releases Nemotron 3.5 Bolt E4B 4GB: An Open Small Language Model for Local Voice Agents, Game Characters and Robots

$uv pip install omni-parse-bench✓text✓tables✓layout→ scored per pageStar on GitHub→Sponsored

Hugging Face 发布 14.8B 开源 DNA 语言模型 Carbon-X

推荐理由

主题

标签

正文