AI Trends

OpenAI Launches MedChemBench to Evaluate AI-Assisted Drug Development Capabilities

ITHome · Artificial IntelligenceUpdated Nov 21 22:16

AI summary

OpenAI has released MedChemBench to evaluate AI models’ reasoning capabilities in medicinal chemistry research, covering molecular structures, scientific explanations, compound selection, and property prediction. The benchmark contains 99 questions drawn from 117 papers and patents, involving 761 molecular structures; at the xhigh reasoning level, GPT-6.2 Astra achieved a usable score of 84.5%. The model performed strongly in reconstructing molecular structures and explaining experimental evidence, but still has considerable room for improvement in compound screening and numerical property prediction.

Why read this

AI score 50

The benchmark covers four types of tasks in medicinal chemistry research and highlights differences in model performance between structure reconstruction and property prediction.

Topics

Tags

Body · AI translation

OpenAI Launches MedChemBench Benchmark to Evaluate AI-Assisted Drug Development Capabilities

November 21, 2026, 08:06, ITHome — Haomiao

ITHome reported on November 21 that, on November 20 local time, OpenAI released the MedChemBench benchmark to evaluate AI models’ reasoning capabilities in medicinal chemistry research, including identifying molecular structures, interpreting experimental results, comparing compounds, and predicting their properties. OpenAI said that the models performed strongly in reconstructing molecular structures and explaining experimental evidence, but still have considerable room for improvement in compound selection and numerical property prediction.

ITHome learned from the official introduction that the benchmark contains 99 questions drawn from 117 papers and patents, involving 761 distinct molecular structures. Most questions provide models with context from a particular point in the research process, including the biological target, the researchers’ objectives, compounds previously studied, and some experimental results.

The official evaluation is divided into four sections, detailed by ITHome below:

Molecular structure: Assemble molecules from the structural diagrams and chemical groups provided in the question, assessing whether atomic connectivity and stereochemistry are correct;

Scientific explanation: Explanations must be supported by the given evidence, such as why one compound was selected from several candidates;

Ranking and selection: Whether compound rankings and final selections meet the question’s requirements;

Property prediction: Whether predictions match the numerical values and categories given in the original materials, covering both measured and computationally derived molecular properties.

According to officially released information, at the xhigh reasoning level, GPT-6.2 Astra achieved a usable score of 84.5%, compared with 83.3% for GPT-6 Astra, 82.0% for GPT-6.1 Sol, and 58.2% for GPT-5.6 Sol. Officials said that increasing the reasoning level can improve scores, but significant differences between models remain.

In the individual categories, Astra earned a usable score of 95.4% in the molecular structure section at the xhigh level, but scored only about half of the available points in both compound selection and numerical property prediction.

As an example, officials cited a question that required models to assemble a molecule from a given central structure and two side groups while preserving an 18-atom ring.

At the xhigh level shared by 11 models, all ten responses from GPT-6 Astra were correct. All ten responses from both GPT-5.6 Sol and GPT-5.6 Luna were incorrect: the former assembled a ring containing only 15 atoms, while the latter split the molecule into two disconnected fragments.

Another question was drawn from research on PRMT5 inhibitors. PRMT5 is a protein of interest in cancer drug development. The models needed to predict how strongly three related compounds would block PRMT5 inside cells, how easily they would cross the cellular barrier, and how quickly they would be degraded by liver enzymes in laboratory testing.

Subsequent experiments concealed from the models in the question revealed a trade-off: compound 23 could block PRMT5 inside cells at a lower concentration, while compound 24 crossed the cellular barrier more easily and was also degraded more slowly in liver enzyme testing.

Officials said these results cannot determine whether models are capable of discovering new drugs or improving laboratory decision-making. The possibility that the models had previously encountered the published materials cannot be ruled out, and every response was a single attempt made without tools, feedback, or new experiments. Although the benchmark is based on historical evidence rather than real-world drug discovery, its results help identify where models are making progress and where challenges remain. The team believes MedChemBench will support continued progress and looks forward to sharing updates as the models advance and their areas of application expand.

Source

ITHome · Artificial Intelligence