Understand this story
Meta has published the production deployment results of Catching JiTTests: the system uses large language models to automatically generate executable tests for code changes awaiting merge, exposing defects before the code is merged. The system employs two workflows, intent-aware and dodgy diff, and uses an LLM-as-Judge ensemble together with the rule-based method RubFake to filter false positives, while reducing manual item-by-item review through a human-on-the-loop approach. Meta says 589 engineers have participated, with a true positive rate of 60%; in the first half of 2026, it caught 74 severe privacy SEVs at the diff stage, reducing the overall incidence of such issues by 20%.
AI-generated from public reports
Latest development · Report
Meta Puts JiTTests into Production: LLM-Generated Tests Catch Defects Before Code Is MergedDevelopments
1 confirmed developments
- Source:Meta Engineering Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 30
Meta Engineering BlogFirst-party
Meta Puts JiTTests into Production: LLM-Generated Tests Catch Defects Before Code Is Merged
Meta has published the production deployment results of Catching JiTTests, a system that uses LLMs to automatically generate executable tests for pending code changes, exposing defects before the code is merged. The system employs two workflows, intent-aware and dodgy diff, and uses an LLM-as-Judge ensemble together with a rule-based method, RubFake, to filter false positives, while reducing manual line-by-line review through a human-on-the-loop approach. Meta says 589 engineers have participated, the true positive rate is 60%, and in the first half of 2026 it caught 74 severe privacy SEVs at the diff stage, reducing the overall incidence of such issues by 20%.