AI Trends

OpenAI Releases MedChemBench, a Medicinal Chemistry Benchmark

Rohan PaulUpdated Nov 21 06:05

AI summary

OpenAI has released MedChemBench, a medicinal chemistry benchmark whose questions are drawn from 117 papers and patents and cover 761 scored molecular structures, with results later reported by researchers withheld. Each prompt stops at a real-world decision point and provides a biological target, existing compounds, and partial assay results. The best-performing model, GPT-6.2 Astra, scored about 84% overall and was nearly perfect at reading chemical information, but made the correct judgment only about half the time on critical decisions.

Why read this

AI score 53

The benchmark stops medicinal chemistry tasks at real-world decision points, making it possible to distinguish a model's ability to read chemical information from its ability to make critical judgments.

Topics

Tags

Body

OpenAI has now published MedChemBench, its medicinal chemistry benchmark.

Its problems come from 117 papers and patents, span 761 graded molecular structures, and withhold the results researchers reported later.

Each prompt stops at a real decision point, supplying the biological target, the compounds made so far and selected assay results.

the best model GPT-6.2 Astra gets about 84% overall, is near perfect at reading the chemistry, and is right about half the time on the decision that actually matters.

Source

Rohan Paul