AI COGS is the cost of serving your AI feature: inference (model API or GPU) calls, plus the retrieval, storage and compute around them. It lands on the same gross-margin line as hosting and support, and unlike a per-seat SaaS cost it scales with usage, so a heavy user can cost more than they pay. The headline fear (that AI is crushing SaaS margins) is half right: at the blended median, software margin has barely moved; on AI-native and AI-heavy products, it runs well below the SaaS norm and inference is the fastest-growing line. This guide covers what AI COGS is, where margins actually sit in 2026, and how to model and hold yours.
If you want the number for your own plan while you read, the AI cost and margin calculator below turns tokens and model prices into AI COGS, cost per customer and gross margin before versus after AI. For the wider picture, this pairs with SaaS unit economics and the pricing question it forces, covered in usage-based and outcome-based pricing.
What is AI COGS?
Cost of goods sold (COGS) is what it costs to deliver the product to a customer who is already paying. For classic SaaS that is hosting, third-party infrastructure, payment fees, and customer support, which is why software gross margins sit so high: once the code is written, serving one more customer costs very little. AI changes the shape of that cost. The moment your product calls a model, every request has a marginal cost, so AI COGS includes:
| AI COGS component | What it is |
|---|---|
| Inference | Model API charges (per input and output token) or the GPU compute to self-host |
| Retrieval / RAG | Vector search, embeddings, and the extra tokens context stuffing adds to each call |
| Orchestration | Agent loops and tool calls that turn one user action into many model calls |
| Data and storage | Storing embeddings, logs, and conversation history |
The key difference from ordinary SaaS COGS is that this cost is variable: it moves with how much each customer uses the model, not with how many seats they bought. That single fact is what pulls gross margin down and what makes a flat price risky.
What AI actually does to gross margin (2026)
Here the honest answer is more useful than the scary one. Two things are true at once:
At the median, SaaS margins have held. The 2026 Aleph and Benchmarkit SaaS and AI benchmarks (342 companies, full-year 2025 actuals) found software gross margin held at roughly 79-81% across four years, with no material AI-driven compression at the median. Usage-only businesses ran lower, around a 62% median, and margin rose with scale (about 72% below $5M ARR up to 86% at $50-100M ARR).
On AI-heavy products, margins run below the norm and are climbing from a low base. ICONIQ's July 2026 State of AI: The Builder's Economy (N=287) puts average gross margin on AI products at 53% projected for 2026 and 59% for 2027, up from 45% actual in 2025 (its January edition had 41% for 2024). In ICONIQ's January data, inference was about 20-23% of AI product costs at the scaling stage (a share of cost, not of revenue) and the fastest-growing line. The original marker for this pattern is older: Andreessen Horowitz noted back in 2020 that AI businesses often run 50-60% gross margins versus 60-80%+ for comparable software.
| Profile | Typical gross margin | Source |
|---|---|---|
| Traditional SaaS | 75-85% (blended ~79-81% at the median) | Aleph x Benchmarkit 2026 |
| Usage-heavy SaaS | ~62% median | Aleph x Benchmarkit 2026 |
| AI product line | ~53% and rising (45% in 2025, 59% projected for 2027) | ICONIQ Jul 2026 |
| AI-native business | 50-60% (origin of the pattern) | a16z 2020 |
So the compression is real, but it shows up on the AI line first, not in the blended median. If your product is a mostly-classic SaaS with an AI feature, your risk is the feature quietly dragging an 80% margin down. If you are AI-native, you are starting nearer 50-60% and the whole game is climbing toward the SaaS norm by making inference cheaper per unit of value.
The math: from tokens to margin
The chain is short. Cost per request = (input tokens / 1M x input price) + (output tokens / 1M x output price). Multiply by requests per customer per month, subtract whatever caching or routing to a cheaper model saves, and you have AI COGS per customer. Total AI COGS is that times your AI customers, and it lands on the P&L next to hosting and support:
Gross margin = (revenue - non-AI COGS - AI COGS) / revenue.
A worked example: 1,000 customers on a $49 plan is $49,000 of monthly revenue. At 500 requests each, 3,000 input and 900 output tokens per request, and mid-tier prices of $3 and $15 per million tokens (Claude Sonnet 4.6's list price; the newest mid-tier models list at $2 and $10), cost per request is about $0.0225. After a 20% caching saving that is $9 of AI COGS per customer, or $9,000 a month. With non-AI COGS at 12% of revenue, gross margin falls from 88% before AI to about 70% after. Not a catastrophe, but nearly 18 points, and it gets worse fast if usage climbs or the plan is cheap. Run your own numbers in the calculator at the end of this post.
The inference efficiency ratio
The quickest health check on an AI product is the inference efficiency ratio, a metric from Ben Murray (The SaaS CFO): AI revenue divided by AI inference cost. A ratio of 5:1 means you earn five dollars for every dollar of inference. As a rough guide, aim for 5:1 or better for an AI-native product, and higher for an AI feature attached to an otherwise healthy SaaS. Below about 4:1, the feature is eating the business, and no amount of growth fixes a unit that loses money at the margin.
The heavy-user trap
Averages lie here. If your typical customer runs 500 requests but a power user runs 2.5x that, the heavy user's AI cost can approach or exceed what they pay you, and a small cluster of them drags the whole cohort's margin down. Flat pricing plus variable cost is the exact recipe for it (seat plans cross-subsidize heavy users; hybrid pricing passes the cost through). This is why AI plans increasingly carry usage caps, fair-use limits, or a usage component: the tail is where a healthy-looking average goes underwater. Model the heavy-user band explicitly, not just the mean.
How to hold your margin
There are only two levers, and most teams pull both:
- Lower cost per request. Route easy calls to a smaller or cheaper model, cache repeated context, trim prompts, cap output length, and batch where you can. This is the first move because it protects margin without touching the customer.
- Change the price. Raise it, add a usage component, or move to a model where price tracks value. When an AI agent does the work rather than assisting a human, a flat seat price stops matching cost or value, which is why so many teams are shifting to usage-based or outcome-based pricing. If you sell prepaid credits, remember that unused credits (breakage) carry no cost and flatter the margin; the credit-based pricing accounting guide shows by how much.
The calculator sizes both for you: given your target margin, it shows the price rise or the cost cut needed to hold it.
AI model price history
Per-token prices are the input your AI COGS depends on most, and they do not only go down. The table lists standard API prices per million tokens as shown today on each vendor's own pricing page (Anthropic, OpenAI, Google), all read on October 23, 2026. The vendors keep older models on the page, so one read shows several generations side by side. A model's price today can differ from its price at launch.
| Vendor | Model | Tier | Input per 1M | Output per 1M | Note |
|---|---|---|---|---|---|
| Anthropic | Claude Opus 4 / 4.1 | frontier | $15 | $75 | |
| Anthropic | Claude Opus 4.5 to Opus 5 | frontier | $5 | $25 | |
| Anthropic | Claude Opus 5.5 | frontier | $4 | $20 | current |
| Anthropic | Claude Fable 5 / 5.1 | frontier | $10 | $50 | current top model |
| Anthropic | Claude Sonnet 4 to 4.6 | mid | $3 | $15 | |
| Anthropic | Claude Sonnet 5 / 5.5 | mid | $2 | $10 | current |
| Anthropic | Claude Haiku 3.5 | small | $0.80 | $4 | |
| Anthropic | Claude Haiku 4.5 | small | $1 | $5 | |
| Anthropic | Claude Haiku 5.5 | small | $0.10 | $0.50 | current; prompts up to 100,000 tokens ($0.50/$2.50 above) |
| OpenAI | gpt-4-0613 | frontier | $30 | $60 | June 2023 model version |
| OpenAI | gpt-4-turbo-2024-04-09 | frontier | $10 | $30 | April 2024 model version |
| OpenAI | gpt-4o-2024-05-13 | frontier | $5 | $15 | May 2024 model version |
| OpenAI | gpt-4o | mid | $2.50 | $10 | |
| OpenAI | gpt-5 | frontier | $1.25 | $10 | |
| OpenAI | gpt-5.5 | frontier | $5 | $30 | under 272K context |
| OpenAI | gpt-6-astra | frontier | $10 | $50 | current flagship, short context |
| OpenAI | gpt-6.1-sol | mid | $2 | $10 | current |
| OpenAI | gpt-3.5-turbo | small | $0.50 | $1.50 | |
| OpenAI | gpt-4o-mini | small | $0.15 | $0.60 | |
| OpenAI | gpt-6-luna | small | $0.10 | $0.50 | current |
| Gemini 2.5 Pro | frontier | $1.25 | $10 | prompts up to 200K tokens | |
| Gemini 3.1 Pro Preview | frontier | $2 | $12 | prompts up to 200K tokens | |
| Gemini 4 Argon | frontier | $2 | $10 | through November 30, 2026; $4.00 / $20.00 from December 1, 2026 | |
| Gemini 2.5 Flash | mid | $0.30 | $2.50 | ||
| Gemini 3.8 Flash | mid | $0.75 | $3.75 | through December 31, 2026; $1.50 / $7.50 from January 1, 2027 | |
| Gemini 2.5 Flash-Lite | small | $0.10 | $0.40 | ||
| Gemini 3.5 Flash-Lite | small | $0.30 | $2.50 |
Three things for a SaaS margin model:
- Small models got much cheaper; frontier models did not. OpenAI's small tier went from $0.50 / $1.50 (gpt-3.5-turbo) to $0.10 / $0.50 (gpt-6-luna), and Anthropic's from $0.80 / $4 (Haiku 3.5) to $0.10 / $0.50 (Haiku 5.5). At the top, OpenAI's newest flagships cost more than gpt-5 ($1.25 / $10): $5 / $30 for gpt-5.5 and $10 / $50 for gpt-6-astra. Anthropic's Opus line fell from $15 / $75 to $4 / $20, while its new top model, Fable, lists at $10 / $50. Routing simple requests to the small tier is where most of the savings are.
- Prices can rise on a date. Google lists Gemini 4 Argon at $2 / $10 through November 30, 2026 and $4 / $20 from December 1, and Gemini 3.8 Flash at $0.75 / $3.75 through December 31, 2026 and $1.50 / $7.50 from January 1, 2027. A forecast built on today's rate for those models understates next quarter's COGS by half. We worked through the margin effect in what the Gemini API price increase does to AI COGS.
- Tokens per request can change too. Anthropic says its tokenizer for Claude 4.7 and later models "produces approximately 30% more tokens for the same text". The same prompt can cost more on a model with the same per-token price, so measure tokens per request after every model change, not just the price.
Put your own model mix into the AI cost and margin calculator to see what a price change does to gross margin.
How to model AI COGS in your forecast
A one-off number is a start, but the decisions (what to price, which model to use, how much usage to allow) need the revenue side and the cost side moving together. Model AI COGS as a driver, not a fixed line: cost per request times projected requests per customer times customers, with adoption and usage ramping over time, sitting alongside your revenue waterfall so gross margin recalculates as either side changes. Then flex the assumptions: what a cheaper model does to margin, what a heavy-usage cohort does, what a price change does to both revenue and retention. That is exactly the kind of driver-based, scenario-aware view a spreadsheet fights you on and a SaaS financial model is built for. You can build it, and ask the AI CFO to pressure-test it, in Adlega.
Frequently asked questions
What is AI COGS?
AI COGS is the cost of serving an AI feature to paying customers: inference (model API calls or GPU compute), retrieval and embeddings, agent orchestration, and related storage. It sits in cost of goods sold because it scales with usage, so it directly reduces gross margin.
Does AI lower SaaS gross margin?
On AI-heavy products, yes: AI product margins average about 53% in 2026 (ICONIQ, July 2026) versus 75-85% for traditional SaaS. At the blended median, though, software margin has held near 79-81%, so the compression shows up on the AI line first rather than across the whole category.
What is a good gross margin for an AI product?
Traditional SaaS aims for 75-85%. AI-native products commonly start at 50-60% and work upward as inference gets cheaper per unit of value. An AI feature bolted onto a healthy SaaS should protect the blended margin and stay well above 60%.
What is the inference efficiency ratio?
AI revenue divided by AI inference cost. A 5:1 ratio (five dollars of revenue per dollar of inference) is a reasonable floor for an AI-native product; below roughly 4:1 the economics need fixing through cheaper inference or higher price.
How do you reduce AI COGS?
Route simpler requests to cheaper models, cache repeated context, shorten prompts and cap output length, batch calls, and self-host where volume justifies it. Pair cost cuts with a pricing model that tracks usage so heavy users cover their own cost.