Running LLMs locally vs paying for an API: the actual math

“Just run it locally, it’s free.”

I hear this a lot. It’s not free. You paid for the hardware, and the hardware draws power. The real question is: at your usage, which one costs less?

This is a math problem, so let’s do the math.

The local side

Running locally has two costs: the hardware, and the electricity to run it.

For hardware, the fair way to count it is amortization: spread the purchase price over the useful life of the machine. Three years is a reasonable assumption for a GPU.

monthly hardware cost = price ÷ (years × 12)

An RTX 4090 at $1,800 over 3 years:

1800 ÷ 36 = $50/month

That’s $50 a month before you generate a single token.

Electricity depends on how long the machine actually computes. That comes from your token volume and your throughput in tokens per second:

compute hours per day = tokens per day ÷ tokens/sec ÷ 3600

Say you generate 500,000 tokens a day on an 8B model at Q4, at a typical 40 tokens/sec:

500000 ÷ 40 ÷ 3600 = 3.47 hours/day

The card draws about 400W under load. Add idle draw for the hours the machine sits on but does nothing, say 8 hours at 50W:

(3.47 × 400 + 8 × 50) ÷ 1000 = 1.79 kWh/day

At the US average of $0.16/kWh, over 30 days:

1.79 × 30 × 0.16 = $8.60/month

Total local cost: $50 + $8.60 = about $59/month.

Notice something: electricity is small next to amortization. The purchase price dominates, which is why “already owned” hardware changes the whole equation.

The API side

The API side is one line. You pay per token, and output tokens are what dominate the bill:

monthly API cost = (tokens per month ÷ 1M) × price per 1M output tokens

Our 500k tokens/day is 15M tokens/month. At three price tiers:

budget (~$0.40/M):   15 × 0.40 = $6/month
mid-tier (~$2/M):    15 × 2    = $30/month
flagship (~$10/M):   15 × 10   = $150/month

And here’s the uncomfortable part: the local 8B model you’re running is a budget-class model. Its fair comparison is the $6/month line, not the $150 one. At this volume, the API wins easily.

Break-even

Local starts winning when the monthly savings, if any, pay back the hardware:

break-even months = hardware cost ÷ (monthly API cost − monthly local cost)

With the flagship comparison ($150/month API vs $58.60/month local):

1800 ÷ (150 − 58.60) ≈ 20 months

Twenty months to break even, and only if you’d genuinely have paid flagship prices for that traffic. Against the budget tier there is no break-even: the API is cheaper forever at this volume.

Higher volume helps, but less than you’d think. Push it to 2M tokens/day (60M/month) against the mid-tier at $2/M and the API costs $120/month, while local runs about $79 (the card now computes almost 14 hours a day, so electricity climbs to ~$29). Break-even: 44 months. Longer than the 3-year amortization window.

That’s the honest picture: at mid-tier and budget API prices, an $1,800 card struggles to pay for itself on cost alone. The break-even math only turns clearly in local’s favor against flagship pricing, or when the hardware costs you nothing.

I built a calculator where you can plug in your own hardware, electricity price, token volume, and API tier, and it gives you the break-even point: the local vs API cost calculator.

When local makes sense

The pure cost math favors the API more often than people expect. But cost isn’t the only variable.

Local wins when:

• You already own the hardware. With amortization at zero, you’re comparing electricity ($8.60/month in our example) to the API bill. Local wins at almost any volume.
• Privacy is a requirement. Medical data, legal documents, anything you can’t send to a third party. The cost comparison is irrelevant, local is the only option.
• Volume is high and quality needs are modest. Bulk classification, summarization, data extraction at millions of tokens per day. A small local model handles it and the API bill would be real money.
• You want zero marginal cost. Once the box is running, experimentation is free. No meter running while you learn. This matters more than it sounds: I do a lot of my throwaway experiments locally exactly because there’s no bill anxiety.

The API wins when:

• You need frontier quality. No consumer GPU runs the flagship models. This isn’t a cost question (I covered which models fit on which hardware in how much VRAM you need to run an LLM locally).
• Your volume is low or spiky. Amortization runs 24/7 whether you use the card or not. If you generate tokens a few days a month, per-token pricing is exactly what you want.
• You don’t want ops. Local means drivers, model files, updates, and a machine that must be on.

The short version

Amortized hardware is the big local cost, electricity is the small one. The API is a straight per-token line. At low volume the line stays under the fixed cost and the API wins; at high volume the line crosses it and local wins, usually within a year.

And whichever side you land on, keep the two swappable in your code. I wrote about that in an LLM adapter pattern for Cloudflare Workers: one interface, and the provider behind it is a config change.

Related posts about ai: