$1.8M a Month: What Cloud FinOps Taught Me About LLM Token Costs

At Paytm, my team took more than $1.8M a month out of a 55PB data platform. None of it came from a single clever trick. It came from a handful of unglamorous patterns applied relentlessly. Years later, running enterprise GenAI, I kept seeing the same patterns — just measured in tokens instead of instance-hours.

Pattern 1: Right-size the compute

Moving EMR fleets to AWS Graviton cut that compute to roughly a third of the price for the same work. The LLM equivalent is model routing: most requests don't need your most capable model. Route simple classification, extraction and summarisation to smaller models and save the frontier model for real reasoning.

Pattern 2: Don't run what you're not using

We replaced always-on clusters with transient, event-driven ones. In LLM terms: tune effort and thinking budgets per use case, cap agent loop lengths and kill idle sessions. Default-to-max is the always-on cluster of the AI era.

Pattern 3: Stop paying to move the same data

Replicating clusters per region removed inter-region transfer costs entirely. The token analogue is prompt caching. Large, stable system prompts and project context should be cached, not resent and repaid on every turn.

Pattern 4: Make cost visible to the people who create it

• Per-user and per-team token budgets with sensible limits.
• Showback dashboards next to productivity metrics, not in a finance spreadsheet.
• Proactive alerts on anomalies, before the invoice arrives.

Cost governance works when engineers see the bill as part of the feedback loop, not as a quarterly surprise.

The bottom line

Successful AI adoption means more usage, which means more spend. Teams that build cost controls into the platform from day one can keep saying yes to new use cases. Teams that don't end up rationing access — which is the fastest way to kill adoption.