How to Design AI Cost Control in Security Operations: Keeping Accuracy While Cutting Spending

Many companies that introduce generative AI into security operations face a problem not of accuracy but of cost. In security operations, it is becoming clear that using the most capable AI models for the routine, high-volume work that occurs every day is too expensive. Many teams first realize their AI usage costs have ballooned beyond expectations when the first invoice arrives.

The author leads a security operations team and has reduced the cost of AI-powered detection for Trust & Safety to about $1 per day. When he shares this figure, people are often surprised, because the assumption that enterprise AI is expensive is widespread. But if the work is designed properly, the cost does not become high. Teams that exceed their budgets are skipping the design step of determining which events should be sent to the model.

Detection triage that determines cost

Tiered detection filtering is not new. Even before large language models (LLMs) became widespread, security teams layered multiple filters to narrow the flood of events down to only those that human analysts should handle. The same idea applies to managing AI costs. The narrower and more precise the filters that narrow down which events are sent to the model, the lower the per-item cost of getting accurate results.

In Trust & Safety, most abuse can be identified before the model is even invoked. Pattern detection based on predefined rules can catch a substantial share of events first. Factors such as account creation date, email provider, and behavioral signals are sent to automated filters, which process obvious events and narrow down the rest. Only a portion of what passes through these filters reaches the LLM. The figure of about $1 per day comes from this design. Properly designed triage keeps expensive processing to a minimum.

A home security system offers an analogy: you do not need cameras monitoring everyone who passes in front of the house; you limit surveillance to those who actually break in. Advanced handling only needs to be brought in at that stage.

Tiering models and tiering costs

Events that pass the first screening are judged in stages by two models of different capability. In the first stage, a lightweight model makes an initial judgment, and only those it cannot decide are handed off to a more capable second-stage model.

The first-stage lightweight model returns output in a predefined format: a judgment of malicious or benign, along with a confidence level. High-confidence results are processed automatically, and only low-confidence events proceed to the second stage. The second-stage, more capable model has broader context and stronger reasoning ability.

Only a very small fraction of all events reach this second stage. The author's team systematically tested the effectiveness across multiple models. As a result, for the target use case, the accuracy gap between the lightweight model and the frontier model was only 1–2%. Meanwhile, the frontier model costs about 5 times more per token. This cost-effectiveness only holds when the events sent to the more capable model actually require its capabilities.

Prompt design is as important as model selection. One of the prompts the author created for agent-based detection exceeds 1,900 words. It covers every situation the agent might face, including criteria for deciding when to escalate and conditions for handling matters autonomously. Not every use case requires this depth. Some Trust & Safety prompts need only two or three sentences. The precision required of a prompt depends on how broad a scope of handling is delegated to the agent.

What separates accurate AI analysis from hallucination is the presence or absence of context. If you give the model only an abuse report and ask whether the user in question is malicious, the model may take the report at face value. If you provide the actual data about the subject of the report along with the report, the model can independently assess whether the claim is valid.

Areas that humans should handle

Clear-cut events are handled automatically. What requires judgment are ambiguous cases, and current agents do not yet possess that judgment.

For example, distinguishing a legitimate security researcher who stores malware samples for analysis from a malicious actor who stores the same content for the purpose of attacking others requires a discriminating ability that AI is still not good at. The same applies to exchanges on issue threads where interpretations of the terms of service may differ. Such events are routed to humans because the question itself requires context-based reasoning that current-generation agents cannot reliably provide.

Our goal is to enable reviewers to spend their time on events that truly require human hands. Engineers on the author's team are working not on reviewing individual events one by one, but on building the system, writing prompts, and defining the patterns that govern detection at scale.

What this means in practice

Attackers try new obfuscation techniques, and the author's team tunes prompts and models to catch them. This is not a one-time deployment but iterative work, and in nature it is close to detection engineering. Prompt design is one form of that: writing rules, validating outputs, and tuning when precision degrades—an iterative process.

Cost is a theme to consider from the design stage, not a problem to worry about after deployment. How much volume, in what format, and with what context you send to the model determines the bill. Many teams that find AI expensive have simply not made these decisions consciously.

This article has explained the design for keeping AI costs down in security operations while maintaining detection precision. What drives cost is not the performance of the model itself, but the design of which events to send to the model, in what format, and with what context. Pre-process clear-cut events with rule-based filtering, use lightweight and higher-tier models in stages, and route only cases requiring judgment to humans. This accumulation forms the foundation of operations that keep spending down while maintaining precision. AI cost is not an unavoidable given; it can be managed through design. When incorporating AI into security operations, the first thing required is to confront this design process.