OpenAI's new internal model scores about 47% higher than GPT 6 Astra on BenchCAD 2.0

According to the tech media outlet TestingCatalog, starting October 31, the usage analytics pages of some Codex users showed a separate record for GPT 6.2 Astra, but the model selection interface still displayed GPT 6 Astra.

Screenshots posted by X platform blogger Token Gremlin show that about 19.8% of one account's model calls were counted under GPT 6.2 Astra, while about 80% were still attributed to GPT 6 Astra. The call ratios in screenshots from different users vary, and some developers speculate based on this that OpenAI may be conducting A/B testing through backend request routing.

In addition, on November 3, the overseas model tracking account LuminaBench claimed that two new identifiers, "rc-ada-or-vm" and "rc-ada-sh-vm," briefly appeared in the OpenAI API model list before disappearing. Some developers speculate that these codenames may be related to release candidate versions of a new model, but there is currently no evidence to confirm that they correspond to GPT 6.2 Astra.

It is worth noting that OpenAI has not officially announced GPT 6.2 Astra, nor has it confirmed the above-mentioned gray-scale testing or the release timing of the new model. Some model demo videos circulating on social platforms are also disputed as to their authenticity and cannot serve as evidence that the new model's performance has been verified.

Just as rumors of the gray-scale testing continued to ferment, OpenAI published a technical blog post titled "Accelerating Engineering Development for the Physical World" on November 3, disclosing the latest progress of AI models in mechanical design, electronic engineering, and physical manufacturing.

Among the most closely watched items was a set of test scores for an unreleased model.

In the BenchCAD 2.0 engineering benchmark, the current GPT 6 Astra scored an average of 0.594 under the highest reasoning setting, while a model OpenAI called a "newer internal model" reached an average score of 0.875, a relative improvement of about 47.3%.

BenchCAD 2.0 mainly evaluates a model's ability to reconstruct three-dimensional mechanical parts and assemblies, identify circuit board connection relationships, and similar tasks. OpenAI believes that this result preliminarily indicates that the new-generation model is expected to further improve the iteration efficiency of professional engineering design.

However, OpenAI did not disclose the official name of this internal model, nor did it explain whether it is the GPT 6.2 Astra mentioned in recent rumors. The above scores are results from a specific engineering benchmark and cannot yet directly represent the overall success rate of real manufacturing tasks.

In addition to publishing model benchmark scores, OpenAI also demonstrated a case in which GPT 6 Astra participated in physical hardware design.

According to the official introduction, two engineers used GPT 6 Astra to complete the design of a portable AI device in less than a week. The device can interact with Codex via voice, and the model participated in the design of the enclosure structure, internal mechanical components, PCB circuit boards, electronic components, user interface, and firmware software.

Throughout the design process, the engineers did not directly manually modify the designs generated by the model. Subsequently, a contract manufacturer produced a small batch of physical prototypes in less than a month.

But OpenAI also acknowledged that turning designs into manufacturable products still requires multiple rounds of manual review and iteration. GPT 6 Astra cannot yet always accurately account for production process constraints, manufacturing tolerances, and assembly details.

OpenAI also disclosed that since February of this year, the number of tokens consumed by its hardware team using AI models has increased 23-fold. Codex and ChatGPT are being used for computer-aided design, manufacturing drawing generation, engineering simulation, and design review, among other tasks.

In terms of external cooperation, OpenAI announced a new research collaboration with Mazda to explore the use of AI to assist in automotive aerodynamics and crash safety design. GPT 6 Astra can generate different front-end geometric shapes and combine them with airflow simulations to help engineers evaluate design options.

Mazda chief engineer Akihiro Nakata said that by using AI to explore and simulate dozens of design variants, engineers can more clearly understand which styling features affect aerodynamic performance and in which areas designers have greater styling freedom.

OpenAI also expanded its cooperation with IndyCar team Chip Ganassi Racing, attempting to convert 3D scan data of existing components into editable CAD models and 2D manufacturing drawings to reduce repetitive work for engineers in reverse modeling and prototype development.

From the new model identifiers appearing in the Codex backend to the officially published engineering benchmark scores of the internal model, a series of recent information shows that OpenAI is continuing to advance large models' capabilities in programming, computer operation, and physical engineering design.

However, whether GPT 6.2 Astra has officially entered gray-scale testing, whether the internal new model corresponds to this version, and when the new model will be released all still await official confirmation from OpenAI.