AI Trends

OpenAI Launches MedChemBench to Evaluate AI Capabilities in Assisting Medicinal Chemistry Research

Phoenix TechnologyUpdated Nov 21 10:21

AI summary

OpenAI has released MedChemBench to evaluate AI models' reasoning capabilities in medicinal chemistry research. The benchmark includes 99 questions drawn from 117 papers and patents, covering 761 distinct molecular structures. At the xhigh reasoning setting, GPT-6.2 Astra, GPT-6 Astra, GPT-6.1 Sol, and GPT-5.6 Sol achieved usable scores of 84.5%, 83.3%, 82.0%, and 58.2%, respectively. The models performed strongly in molecular structure reconstruction, but there remains substantial room for improvement in compound selection and numerical property prediction. The results also cannot establish whether the models can discover new drugs or improve laboratory decision-making.

Why read this

AI score 51

The benchmark divides medicinal chemistry tasks into four categories and highlights gaps in models' capabilities in structure reconstruction and property prediction, helping clarify the boundaries of current applications.

Topics

Tags

Body · AI translation

OpenAI Launches MedChemBench Benchmark to Evaluate AI-Assisted Drug Development Capabilities Download the IT Home app for an exclusive first look November 21, 2026, 08:07:33, from Shandong Updated November 21, 2026, 08:51:32

IT Home reported on November 21 that OpenAI released MedChemBench on November 20 local time, a benchmark designed to evaluate AI models’ reasoning capabilities in medicinal chemistry research, including identifying molecular structures, interpreting experimental results, comparing compounds, and predicting their properties. OpenAI said the models performed strongly in reconstructing molecular structures and explaining experimental evidence, but still have considerable room for improvement in compound selection and numerical property prediction.

IT Home learned from the official introduction that the benchmark contains 99 questions drawn from 117 papers and patents, involving 761 distinct molecular structures. Most questions provide models with context from a particular stage of a study, including the biological target, the researchers’ objectives, compounds previously studied, and some experimental results.

The official evaluation is divided into four sections, detailed by IT Home below:

Molecular structure: Assemble molecules based on the structural diagrams and chemical groups provided in the question, assessing whether atomic connectivity and stereochemistry are correct;

Scientific explanation: Explanations must be supported by the given evidence, such as why one compound was chosen from among several;

Ranking and selection: Whether the ranking and final selection of compounds meet the question’s requirements;

Property prediction: Whether predictions match the numerical values and categories given in the source material, covering both experimentally measured and computationally derived molecular properties.

According to officially released information, at the xhigh reasoning level, GPT-6.2 Astra achieved a usable score of 84.5%, GPT-6 Astra 83.3%, GPT-6.1 Sol 82.0%, and GPT-5.6 Sol 58.2%. Officials said that increasing the reasoning level can improve scores, but substantial differences between models remain.

In the individual categories, Astra at the xhigh level achieved a usable score of 95.4% for molecular structure, but scored only about half of the available points in both compound selection and numerical property prediction.

As an example, officials described a question that required the model to assemble a molecule using a given central structure and two side groups while preserving an 18-atom ring.

At the xhigh level shared by 11 models, all ten responses from GPT-6 Astra were correct. All ten responses from GPT-5.6 Sol and GPT-5.6 Luna were incorrect: the former assembled a ring containing only 15 atoms, while the latter split the molecule into two disconnected fragments.

Another question was drawn from research into PRMT5 inhibitors. PRMT5 is a protein of interest in cancer drug development. The model had to predict how strongly three related compounds would inhibit PRMT5 inside cells, how easily they would cross the cellular barrier, and how quickly they would be degraded by liver enzymes in laboratory assays.

Subsequent experimental results withheld from the question revealed trade-offs: compound 23 could inhibit PRMT5 inside cells at a lower concentration, while compound 24 crossed the cellular barrier more easily and was degraded more slowly in the liver enzyme assay.

Officials said these results cannot establish whether the models can discover new drugs or improve laboratory decision-making. The possibility that the models had previously encountered the published materials cannot be ruled out, and every response was a single attempt made without tools, feedback, or new experiments. Although the benchmark is based on historical evidence rather than real-world drug discovery, its results help identify where models are making progress and where challenges remain. The team believes MedChemBench will support continued progress and looks forward to sharing further developments as models and their applications advance.

“Special notice: The content above, including any videos, images, or audio, was uploaded and published by users of Dafeng Hao, a self-media platform operated by Phoenix New Media. This platform merely provides information storage services.

Notice: The content above (including the videos, pictures and audios if any) is uploaded and posted by the user of Dafeng Hao, which is a social media platform and merely provides information storage space services.”

Source

Phoenix Technology