搜索“RAG”

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is safe completion rate, checking that the model’s response and actions are not disallowed according to our safety policies. Our evaluations are run on the model without system-level safeguards to ensure the model’s underlying behavior meets our safety bar. We continue monito

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is safe completion rate, checking that the model’s response and actions are not disallowed according to our safety policies. Our evaluations are run on the model without system-level safeguards, to ensure the model’s underlying behavior meets our safety bar. We continue monit

匹配字段正文

context, including differences in epidemiology, standard care practices, or access to care. These examples are each validated by 2+ physicians Each of these evaluations is the average of the relevant evaluations presented in Table 8 of the HealthBench work [8], for HealthBench Consensus and HealthBench Hard examples. Figure 7 We see large reductions in all three failure modes. For example, hallucinations on challenging conversations are reduced by 8x between OpenAI o3 and gpt-5-thinking. Errors

匹配字段正文

rovements to prompt injection and instruction hierarchy robustness compared to their predecessors. GPT-6.1 Sol is also highly robust. The plot below shows defender success rate averaged per defender query on indirect prompt injection attacks (higher is better). Figure 3 Table 6: Defender success rate averaged per defender query on instruction hierarchy attacks (higher is better). 5. Health For a full description of these evaluations, please see the Health section in the GPT-6 Astra card. 5.1 Hea

匹配字段正文

re built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the initial scores below. Error rates are not representative of average production traffic. Note these are new evaluations and may evolve over time. Table 1: Disallowed Content Evaluations - Production Benchmarks (higher is better) These are new evaluations that were not available when the August 15 model launched. We have run them retrospectively for these launches. 2.2 Jailbreaks We

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is not unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy. Our evaluations are run on the model without system-level safeguards, to ensure the model’s underlying behavior meets our safety bar. We continue monitoring thes

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is not unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy. Our evaluations are run on the model without system-level safeguards, to ensure the model’s underlying behavior meets our safety bar. We continue monitoring thes

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is not unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy. Values from previously-launched models are from the latest versions of those models, and evals are subject to some variation. Values may vary slightly from value

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is not unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy. Our evaluations are run on the model without system-level safeguards, to ensure the model’s underlying behavior meets our safety bar. We continue monitoring thes

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is not unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy. Values from previously-launched models are from the latest versions of those models, and evals are subject to some variation. Values may vary slightly from value

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is not unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy. Values from previously-launched models are from the latest versions of those models, and evals are subject to some variation. Values may vary slightly from value

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The metric is not unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy. Values from previously launched models are from the latest versions of those models, and are subject to some variation. Values may vary slightly from values published at

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is not unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy. Table 1: Production Benchmarks (higher is better) We are continually improving our disallowed content evaluations and categories, and compare performance on thes

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is not unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy. Table 1: Production Benchmarks (higher is better) Values from previously-launched models are from the latest versions of those models, and evals are subject to s

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is not unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy. Table 1: Production Benchmarks (higher is better) GPT-5.2-Codex generally performs on par with GPT-5.1-Codex-Max when used in a conversational setting, and withi

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is not unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy. Table 1: Production Benchmarks (higher is better) New evaluations, as introduced in the GPT-5 update on sensitive conversations. Overall, both gpt-5.1-thinking a

匹配字段正文

They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is not unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy. Table 1: Production Benchmarks (higher is better) New evaluations, as introduced in the GPT-5 update on sensitive conversations. These evaluations were introduce

匹配字段正文

ts multi-step research on the internet for complex tasks. The deep research model is powered by an early version of OpenAI o3 that is optimized for web browsing. Deep research leverages reasoning to search, interpret, and analyze massive amounts of text, images, and PDFs on the internet, pivoting as needed in reaction to information it encounters. It can also read files provided by the user and analyze data by writing and executing python code. We believe deep research will be useful to people a

匹配字段正文

exts See Watch Mode section 3.1.2.4. 3.2.3 Evaluations Table 13: Evaluations for Agent makes a mistake'' risk mitigations Editing permissions (for example on documents in cloud storage): 100.0%; Sending high-stakes communications: 99.9%; Completing financial transactions: 100.0% 3.3 User asks agent to do a harmful or disallowed task 3.3.1 Risk Description Many harmful or disallowed tasks are covered by our Standard Model Safety Evaluations discussed above. Beyond those, we developed new policies

匹配字段正文

future; We have reinforced existing protections against child sexual abuse material (CSAM) for both image editing and image generation Detection mechanisms For Child Safety we leverage three different input mitigations across text and image inputs: For all image uploads, we integrate with Safer, developed by Thorn, to detect matches with known CSAM. Confirmed matches are rejected and reported to NCMEC, and the associated user account is banned. Additionally, we utilize Thorn’s CSAM classifier to

匹配字段正文