AI COGS is the cost of serving your AI feature: inference (model API or GPU) calls, plus the retrieval, storage and compute around them. It lands on the same gross-margin line as hosting and support, and unlike a per-seat SaaS cost it scales with usage, so a heavy user can cost more than they pay. The headline fear (that AI is crushing SaaS margins) is half right: at the blended median, software margin has barely moved; on AI-native and AI-heavy products, it runs well below the SaaS norm and inference is the fastest-growing line. This guide covers what AI COGS is, where margins actually sit in 2026, and how to model and hold yours.
If you want the number for your own plan while you read, the AI cost and margin calculator below turns tokens and model prices into AI COGS, cost per customer and gross margin before versus after AI. For the wider picture, this pairs with SaaS unit economics and the pricing question it forces, covered in usage-based and outcome-based pricing.
What is AI COGS?
Cost of goods sold (COGS) is what it costs to deliver the product to a customer who is already paying. For classic SaaS that is hosting, third-party infrastructure, payment fees, and customer support, which is why software gross margins sit so high: once the code is written, serving one more customer costs very little. AI changes the shape of that cost. The moment your product calls a model, every request has a marginal cost, so AI COGS includes:
| AI COGS component | What it is |
|---|---|
| Inference | Model API charges (per input and output token) or the GPU compute to self-host |
| Retrieval / RAG | Vector search, embeddings, and the extra tokens context stuffing adds to each call |
| Orchestration | Agent loops and tool calls that turn one user action into many model calls |
| Data and storage | Storing embeddings, logs, and conversation history |
The key difference from ordinary SaaS COGS is that this cost is variable: it moves with how much each customer uses the model, not with how many seats they bought. That single fact is what pulls gross margin down and what makes a flat price risky.
What AI actually does to gross margin (2026)
Here the honest answer is more useful than the scary one. Two things are true at once:
At the median, SaaS margins have held. The 2026 Aleph and Benchmarkit SaaS and AI benchmarks (342 companies, full-year 2025 actuals) found software gross margin held at roughly 79-81% across four years, with no material AI-driven compression at the median. Usage-only businesses ran lower, around a 62% median, and margin rose with scale (about 72% below $5M ARR up to 86% at $50-100M ARR).
On AI-heavy products, margins run below the norm and are climbing from a low base. ICONIQ's January 2026 State of AI put gross margin on AI products at 52% (2026 projected), up from 45% in 2025 and 41% in 2024. In the same data, inference was about 20-23% of AI product costs at the scaling stage (a share of cost, not of revenue) and the fastest-growing line. The original marker for this pattern is older: Andreessen Horowitz noted back in 2020 that AI businesses often run 50-60% gross margins versus 60-80%+ for comparable software.
| Profile | Typical gross margin | Source |
|---|---|---|
| Traditional SaaS | 75-85% (blended ~79-81% at the median) | Aleph x Benchmarkit 2026 |
| Usage-heavy SaaS | ~62% median | Aleph x Benchmarkit 2026 |
| AI product line | ~52% and rising (41 to 45 to 52) | ICONIQ Jan 2026 |
| AI-native business | 50-60% (origin of the pattern) | a16z 2020 |
So the compression is real, but it shows up on the AI line first, not in the blended median. If your product is a mostly-classic SaaS with an AI feature, your risk is the feature quietly dragging an 80% margin down. If you are AI-native, you are starting nearer 50-60% and the whole game is climbing toward the SaaS norm by making inference cheaper per unit of value.
The math: from tokens to margin
The chain is short. Cost per request = (input tokens / 1M x input price) + (output tokens / 1M x output price). Multiply by requests per customer per month, subtract whatever caching or routing to a cheaper model saves, and you have AI COGS per customer. Total AI COGS is that times your AI customers, and it lands on the P&L next to hosting and support:
Gross margin = (revenue - non-AI COGS - AI COGS) / revenue.
A worked example: 1,000 customers on a $49 plan is $49,000 of monthly revenue. At 500 requests each, 3,000 input and 900 output tokens per request, and mid-tier prices of $3 and $15 per million tokens, cost per request is about $0.0225. After a 20% caching saving that is $9 of AI COGS per customer, or $9,000 a month. With non-AI COGS at 12% of revenue, gross margin falls from 88% before AI to about 70% after. Not a catastrophe, but nearly 18 points, and it gets worse fast if usage climbs or the plan is cheap. Run your own numbers in the calculator at the end of this post.
The inference efficiency ratio
The quickest health check on an AI product is the inference efficiency ratio, a metric from Ben Murray (The SaaS CFO): AI revenue divided by AI inference cost. A ratio of 5:1 means you earn five dollars for every dollar of inference. As a rough guide, aim for 5:1 or better for an AI-native product, and higher for an AI feature attached to an otherwise healthy SaaS. Below about 4:1, the feature is eating the business, and no amount of growth fixes a unit that loses money at the margin.
The heavy-user trap
Averages lie here. If your typical customer runs 500 requests but a power user runs 2.5x that, the heavy user's AI cost can approach or exceed what they pay you, and a small cluster of them drags the whole cohort's margin down. Flat pricing plus variable cost is the exact recipe for it. This is why AI plans increasingly carry usage caps, fair-use limits, or a usage component: the tail is where a healthy-looking average goes underwater. Model the heavy-user band explicitly, not just the mean.
How to hold your margin
There are only two levers, and most teams pull both:
- Lower cost per request. Route easy calls to a smaller or cheaper model, cache repeated context, trim prompts, cap output length, and batch where you can. This is the first move because it protects margin without touching the customer.
- Change the price. Raise it, add a usage component, or move to a model where price tracks value. When an AI agent does the work rather than assisting a human, a flat seat price stops matching cost or value, which is why so many teams are shifting to usage-based or outcome-based pricing.
The calculator sizes both for you: given your target margin, it shows the price rise or the cost cut needed to hold it.
How to model AI COGS in your forecast
A one-off number is a start, but the decisions (what to price, which model to use, how much usage to allow) need the revenue side and the cost side moving together. Model AI COGS as a driver, not a fixed line: cost per request times projected requests per customer times customers, with adoption and usage ramping over time, sitting alongside your revenue waterfall so gross margin recalculates as either side changes. Then flex the assumptions: what a cheaper model does to margin, what a heavy-usage cohort does, what a price change does to both revenue and retention. That is exactly the kind of driver-based, scenario-aware view a spreadsheet fights you on and a SaaS financial model is built for. You can build it, and ask the AI CFO to pressure-test it, in Adlega.
Frequently asked questions
What is AI COGS?
AI COGS is the cost of serving an AI feature to paying customers: inference (model API calls or GPU compute), retrieval and embeddings, agent orchestration, and related storage. It sits in cost of goods sold because it scales with usage, so it directly reduces gross margin.
Does AI lower SaaS gross margin?
On AI-heavy products, yes: AI product margins run around 52% in 2026 versus 75-85% for traditional SaaS. At the blended median, though, software margin has held near 79-81%, so the compression shows up on the AI line first rather than across the whole category.
What is a good gross margin for an AI product?
Traditional SaaS aims for 75-85%. AI-native products commonly start at 50-60% and work upward as inference gets cheaper per unit of value. An AI feature bolted onto a healthy SaaS should protect the blended margin and stay well above 60%.
What is the inference efficiency ratio?
AI revenue divided by AI inference cost. A 5:1 ratio (five dollars of revenue per dollar of inference) is a reasonable floor for an AI-native product; below roughly 4:1 the economics need fixing through cheaper inference or higher price.
How do you reduce AI COGS?
Route simpler requests to cheaper models, cache repeated context, shorten prompts and cap output length, batch calls, and self-host where volume justifies it. Pair cost cuts with a pricing model that tracks usage so heavy users cover their own cost.