Cerebras 发布 ExomeBench：面向临床基因组变异解读的基准

Feb 23 2026

There has been tremendous progress in DNA and genomics modelling with transformer-based models, such as Nucleotide Transformer[1] and Evo[2,3]. These models are typically evaluated on structural and functional genomics tasks, such as predicting regulatory elements, chromatin accessibility, or other sequence-level properties, and they achieve impressive performance on these benchmarks. However, as most existing benchmarks focus on tasks related to general sequence modeling, it is unclear how well these state-of-the-art models perform on clinically relevant tasks, for example, predicting the pathogenicity of a variant or its association with disease phenotypes. There has been a gap in evaluating clinically meaningful variant interpretation, whether a model can use context from exome-regions to support health-relevant predictions.

ExomeBench aims to fills this gap. Built from ClinVar -derived single-nucleotide variants in exome regions, it includes five supervised classification tasks that emphasize clinically relevant signals. Unlike other existing benchmarks, it focuses on questions that arise in medical and translational genomics, enabling researchers to test models on clinically relevant tasks.

By complementing existing benchmarks, ExomeBench provides a public resource to help researchers and clinicians iterate faster and develop novel modeling techniques for clinical genomics.

ExomeBench is a research benchmark. It is not a diagnostic tool and should not be used to make clinical decisions. ClinVar labels reflect submitted interpretations and evidence levels; any model performance reported here does not constitute clinical validation.

ExomeBench is built from ClinVar (November 2024 release ) and focuses on variants in exome regions (the repository provides a detailed definition and region mask). We include only single-nucleotide variants (SNVs), the most common type of genetic variant in exomes.

For each variant, the input to a model is a DNA sequence window of 100 base pairs centered on the variant. These sequences are extracted from the GRCh38 reference genome, with the reference or alternate allele applied at the center position (see the repository for exact encoding details).

The dataset contains a total of over 158k variants across five clinically relevant tasks:

Each task has integer-encoded labels and an accompanying published label map for consistent interpretation.

To support proper evaluation, the data is split into train, validation, and test sets, ensuring that no exact variant appears in more than one split. For the PV task, we take an extra precaution with gene-disjoint splits- all variants from the same gene are kept in a single split. This prevents models from memorizing gene-specific patterns and makes performance a better indicator of generalization to new genes.

ExomeBench provides a carefully structured, reproducible dataset. We release the scripts used to create the dataset from ClinVar database.

ExomeBench includes five supervised classification tasks:

We report Matthews correlation coefficient (MCC) on the test set (higher is better). MCC is informative for imbalanced classification, which is common in clinical variant datasets.

Note: Some tasks (e.g., gene-attribution tasks like TFG) can be close to saturated and may be less discriminative across strong models; we recommend focusing comparisons on PV/CPS/CP for clinical interpretation signal.

To use ExomeBench, start with the dataset on Hugging Face and follow the end-to-end workflows in the project’s GitHub repository (data loading, training, evaluation, and hyperparameter sweeps). Those resources are the source of truth for reproducing the baselines and running your own models.

Corresponding Email: exome-bench@cerebras.net

This benchmark was developed as part of the work supporting the STRAND paper (in collaboration with Mayo Clinic).

If you use ExomeBench, please (1) cite the STRAND paper and (2) acknowledge ClinVar and GRCh38 as the underlying sources used to construct the benchmark.

Performance comparisons are based on third-party benchmarking or internal testing. Observed inference speed improvements versus GPU-based systems may vary depending on workload, configuration, date and models being tested.

1237 E. Arques Ave Sunnyvale, CA 94085