OpenAI Releases MedChemBench, a Medicinal Chemistry Benchmark
OpenAI has released MedChemBench, a medicinal chemistry benchmark whose questions are drawn from 117 papers and patents and cover 761 scored molecular structures, with results later reported by researchers withheld. Each prompt stops at a real-world decision point and provides a biological target, existing compounds, and partial assay results. The best-performing model, GPT-6.2 Astra, scored about 84% overall and was nearly perfect at reading chemical information, but made the correct judgment only about half the time on critical decisions.
Why read this
AI score 53
The benchmark stops medicinal chemistry tasks at real-world decision points, making it possible to distinguish a model's ability to read chemical information from its ability to make critical judgments.
Topics
Body
OpenAI has now published MedChemBench, its medicinal chemistry benchmark.
Its problems come from 117 papers and patents, span 761 graded molecular structures, and withhold the results researchers reported later.
Each prompt stops at a real decision point, supplying the biological target, the compounds made so far and selected assay results.
the best model GPT-6.2 Astra gets about 84% overall, is near perfect at reading the chemistry, and is right about half the time on the decision that actually matters.
AI Trends