Bain：AI 成本波动需实时主动管控

가

The body content of the article will be changed to this letter size.

스크롤 이동 상태바

Consulting firm Bain & Company says artificial intelligence (AI) is both the biggest driver of rising IT costs and the most powerful tool for controlling them.

According to Bain’s projections, a consumer goods company with annual revenue of $10 billion could see its yearly IT operating expenses surge from approximately $250 million today to $560 million by 2035 without effective cost management.

Many companies are therefore attempting to control IT spending, including AI costs. However, Bain noted that more than half fail to meet their targets, with roughly half falling short by 50% or more. Cost savings also tend to disappear within one to two years.

Bain identified five structural factors behind these failures. The first is insufficient cost visibility. Most companies lack end-to-end visibility into total IT spending, particularly spending on “shadow AI” outside the control of the chief information officer’s organization.

The second is budget cuts that fail to consider customer experience, execution risks and strategic positioning. Across-the-board reductions can undermine AI, data and product engineering capabilities that drive growth.

IT spending growth exceeds executive expectations. Bain & Company compared three scenarios for annual IT spending between 2025 and 2035 at a consumer packaged goods company with $10 billion in revenue. Executives expect spending to remain flat at approximately $250 million, but even under a well-managed scenario, AI adoption is projected to increase costs by about 75% to $441 million by 2035. Without cost controls, spending could exceed $560 million. The analysis suggests that IT spending could rise by at least 75% despite rigorous cost management. (Source: Bain & Company)

Other factors include governance that cannot keep pace with AI development, nonlinear AI economics and a lack of sustained financial discipline. AI technology evolves within weeks, while corporate budgets are typically set annually. Token costs fluctuate, spending is concentrated among a small number of heavy users, and surging consumption can offset declining unit prices.

Bain’s findings align with the four core areas of AI cost management identified by the FinOps Foundation.

As with cloud FinOps, the foundation emphasizes visibility, benefits measurement, rightsizing and the FinOps operating model. Visibility involves understanding contractual terms and actual usage to connect costs with business outcomes. Benefits measurement requires documenting and continuously tracking the business metrics that each AI use case is intended to improve.

Forbes offered similar recommendations earlier in October in an article titled “FinOps In The Age Of AI: Governing Intelligence At Scale.”

The article argued that companies seeking to evolve traditional IT cost management into an AI FinOps framework need an enterprise AI control plane with five core capabilities: real-time token observability, AI unit economics measurement, model governance, automated controls for AI-generated code and continuous optimization.

Changes in IT spending composition as companies adopt AI. The chart shows annual IT spending and its distribution across categories in 2025, 2030 and 2035 under a well-managed scenario. Spending on the AI and model layer, as well as agentic systems and orchestration, grows from a small share in 2025. Supporting layers, including infrastructure and governance, integrated data platforms, semantics and ontologies, also expand.

Token observability involves tracking consumption, latency and major cost drivers across models, applications and teams in real time. Model governance requires automated model selection and routing policies that account for cost, performance, risk and regulatory requirements, while automated code controls verify whether AI-generated code meets compliance requirements.

Accurate measurement is the foundation of visibility. Even detailed dashboards are of little value unless companies can reliably quantify cost drivers and unit prices, including those associated with individual AI models.

Without accurate measurement, evaluating AI performance is also impossible. Such assessments begin with forecasting future spending and setting budgets based on key cost drivers, including application programming interface (API) usage, model combinations and user adoption rates.

“Tokens must be recorded without omission for every request, broken down by model and provider,” said Lablup. “Treating costs as zero simply because the unit price is unknown or the model is an internally fine-tuned open-weight model will understate actual spending.”

Lablup entered the AI FinOps market in September with the launch of its AI cost management solution, Continuum Hub.

The company also stressed the importance of identifying who consumes AI resources, whether an individual user, team, agent or service, in a way that reflects the organization’s structure. Such attribution is necessary for effective budgeting and internal cost allocation, a principle also emphasized by the FinOps Foundation.

The foundation recommends calculating unit economics by dividing the total cost of an AI use case by its final business output. It also advises companies to examine whether consumption growth is proportional to declining unit prices and to select the least expensive models that meet required quality standards.

Forbes defines AI unit economics as measuring costs per customer inquiry, contract or workflow rather than simply calculating token expenses. This approach allows companies to assess actual return on investment (ROI). The critical question is not how much was spent, but what the spending achieved.

Data analytics platform provider Dataiku emphasizes business value beyond standardized ROI calculations. It argues that companies must evaluate how effectively each AI agent fulfills its intended purpose by combining metrics such as usage, cost and quality.

The FinOps Framework. An overview of the framework through which business, finance and engineering teams collaborate to manage costs and operations. (Source: FinOps Foundation)

Dataiku recently launched Agent Management, a product designed to identify AI agents within enterprises and measure their business outcomes and technical performance. It allows companies to determine whether agents generate value commensurate with their costs.

The platform continuously assesses whether agents perform their intended functions and deliver results that justify their expenses. Examples include agents that prepare campaign briefs for marketing departments, reconcile invoices for finance teams, support operational workflows, handle customer interactions, process sensitive data or manage real-time transactions.

Real-time monitoring is equally important for AI cost management. Cloud service expenses generally accumulate according to relatively predictable measures, such as server operating hours and storage capacity. AI costs, however, arise with each request and fluctuate much more widely, making retrospective analysis based on monthly invoices insufficient.

AI FinOps therefore requires real-time controls, including spending limits, anomaly detection and automatic model switching.

AI managed service provider Bespin Global identifies six requirements for effective AI cost management: unified visibility into graphics processing unit (GPU) and token costs, cost attribution by organization and service, budget and spending-limit controls, real-time anomaly detection and automatic blocking, model routing-based optimization, and value measurement based on unit economics.

“Discovering budget overruns through a month-end report is not cost control,” a Lablup official said. “Limits must be enforced when requests arrive, with policies determining whether excess requests are blocked or routed to lower-cost models.”

Why traditional IT cost management falls short for AI FinOps. Conventional transaction- and utilization-based models struggle with fixed reservation costs that remain payable regardless of utilization, nondeterministic token costs that can vary by several times or more than 100 times for a single request, architecture-time lock-in involving GPU types, VRAM and model sizes, and invisible cost chains involving as many as 37 underlying operations, including tokenization, vector database queries and safety filters. (Source: Rack2Cloud)

Lee Jung-woo, CEO of AI Gateway X, also emphasized proactive cost controls.

“Simply notifying administrators after budgets have already been exceeded cannot effectively manage costs,” Lee said. “Companies must assess expected costs and remaining budgets before requests are executed and apply policies, when necessary, before calling the actual provider.”

AI Gateway X plans to launch an AI FinOps service in October. Rather than relying on a dashboard that monitors spending after the fact, the service evaluates the estimated cost of each AI request and the current budget before forwarding it to the provider. It then allows or restricts the request according to corporate policies.

MegazoneCloud also identifies proactive control, real-time response and centralized management as the three pillars of AI cost management.

Proactive control involves setting token usage limits in advance by role, grade and project to keep AI consumption within budget. Changes to spending limits or their effective periods must pass through an approval process.

“Our AI FinOps service allows token limits and budgets to be allocated according to organizational structure, user roles and job grades,” MegazoneCloud said. “Administrators can control departmental AI usage and costs through review and approval processes, while API calls can be immediately restricted when established budgets are exceeded.”

The company also emphasized the need for more granular access controls, down to individual virtual keys and API keys where necessary. Usage and blocking histories should be recorded in audit logs so that spending and enforcement decisions can be traced.

These measures allow companies to forecast costs by organization and prevent unexpected spending increases even as AI usage grows rapidly.

Hyun-Seon, Park hspark@thelec.kr

My comment