TensorZero 工程博客：别只比每百万 token 价格，LLM API 还有隐藏成本
Stop comparing price per million tokens: the hidden LLM API costs · TensorZero
TensorZero 工程博客指出，仅比较每百万 token 价格会低估 LLM API 的真实成本，因为不同厂商的 tokenizer 会让同一输入产生不同 token 数。该团队用 OpenAI、Anthropic 和 Google 官方 token 计数 API 测试相同内容，发现工具定义类负载中 claude-opus-4-7 的成本是 gpt-5.4 的 5.3 倍，而两者标价仅相差 2 倍；Gemini 在文本和结构化数据上最便宜，但在工具定义上比 OpenAI 贵 46%。文章建议在真实场景中同时比较性能与成本，并提到其开源 LLM 网关可跨厂商追踪 token 用量与成本。
正文
当前显示原文，中文全文译文尚未就绪。
Summary
Token pricing is misleading: the same input produces 2.65x+ more tokens depending on the model. We got wildly different token counts from identical content using OpenAI, Anthropic, and Google’s official token counting APIs; Token efficiency varies by content type. Text, JSON, YAML, and tool definitions all tokenize differently. The cheapest provider changes depending on what you’re sending. The only way to know what you’re actually paying is to measure it; OpenAI has the most efficient tokenizer. On tool-heavy workloads, claude-opus-4-7 costs 5.3x more than gpt-5.4 despite list prices being only 2x apart
Most engineers know they need to evaluate models on their specific task because performance varies. But so does cost: the same input can cost several times more on one provider than another, even when their list prices look similar.
The typical metric for comparing LLM API costs is price per million tokens ($/MTok). Here’s what the major providers charge today:
But not all tokens are equal! Different providers use different tokenizers, so the same input produces wildly different token counts.
A tokenizer splits text into smaller units (tokens) that the LLM processes. Different tokenizers split the same text differently, producing different token counts.
For example, gpt-5.4 uses the tokenization below:
We sent identical inputs through each provider’s official token counting API and normalized against OpenAI’s:
We used the following input data for this experiment:
Multiplying list price by tokenizer efficiency gives you what you actually pay to process the same input.
The differences are dramatic. On tool-heavy workloads, claude-opus-4-7 costs 5.3x more than gpt-5.4 even though their list prices are only 2x apart. The rankings also flip depending on what you’re sending: Gemini is the cheapest option for text and structured data, but becomes 46% more expensive than OpenAI on tool definitions.
This analysis only considers base input token prices. In practice, cost comparisons get even more complex when you factor in prompt caching discounts, long-context pricing tiers, output tokens, and thinking tokens.
When choosing the right model for your task, you should compare both performance and cost in a real setting. A model that looks cheaper on paper might cost several times more once you account for how it counts tokens.
The only way to know what you’re actually paying is to measure it.
Our 11.5Kopen-source LLM gateway tracks real-world token usage and cost across providers. You can configure multiple variants and track usage and costs side by side.