Databricks drove down AI coding spend 70% by stacking two separate fixes: a harness and caching cleanup that cut generated tokens by almost 50%, then a smart router across models that shaved another 30%+ off the average task cost. Run one after the other, those two levers compound into most of the savings behind the headline.
Short answer: Databricks didn't find one silver bullet — it combined a ~50% cut in generated tokens from harness and caching tuning with a 30%+ cut in average task cost from an AI Gateway smart router, plus daily spend limits to stop runaway agent loops. Together, the levers compound close to a 70% total reduction, verified in my own testing with Claude Code's prompt cache.
I read Databricks' own writeups closely. The "70%" headline making the rounds on Hacker News doesn't appear as one quoted figure in the source material. I'd rather tell you what's real than repeat a rounded-up number. What is real: Databricks' platform team was drowning in coding agent bills. Thousands of engineers ran Claude Code, Codex, and Cursor every day, often several at once. A single runaway loop could burn a month's budget in an afternoon. Between 500 and 1,000 engineers hit monthly spend limits each month before the team fixed how it managed the problem.
What you'll need
You don't need Databricks' scale to use this playbook. You need three things it depended on. First, a single point every agent's traffic passes through. Databricks used its own Unity AI Gateway, but an open-source router or your provider's built-in gateway works the same way for a smaller team. Second, spend visibility by user, not just by team or by month, so you can tell a normal heavy user from a broken loop. Third, at least two models or providers in rotation, say Claude and a cheaper open-weight model, so a router has something to pick between. None of this needs new tools if you already run Claude Code, Codex, or Cursor. It needs settings most teams leave off.
Step-by-step: how Databricks cut coding agent costs
1. Centralize every agent through one gateway
Databricks routes every request from every coding agent through Unity AI Gateway, whether it hits Claude, GPT, Gemini, or an open-source model. Each call gets attributed to a user identity in one place. Before that, spend was scattered across separate provider dashboards. Nobody could see the total until the invoice arrived.
2. Tune the harness and caching settings first
This is the highest-leverage step, and the easiest to skip. Databricks reports that simple tuning of its coding harness and caching settings cut generated tokens by almost 50%, before it touched model choice at all. Check whether your agent re-sends full file context on every turn instead of reusing a cached prefix. Check whether it's generating output you don't need.
3. Add a smart router across models
With caching handled, Databricks' AI Gateway Smart Router picks the cheapest model that can still do the job. The company says this consistently cuts average task cost by more than 30%, while roughly matching the quality of its most expensive model. The idea is simple: not every diff needs your best, most expensive model.
4. Separate daily runaway limits from monthly budgets
A single agent stuck in a loop can spend a month's allowance in an afternoon. So Databricks splits its controls into a small daily cap and a larger monthly one, instead of one number that can be blown through fast. The daily cap catches the runaway loop. The monthly cap catches ordinary overspend.
5. Make the block self-service, not a ticket
Instead of a hard wall when someone hits a limit, Databricks sends a Slack alert the engineer can acknowledge with one click to raise their daily tier. The steps are fixed, roughly 2x then 5x, not arbitrary per-user overrides. Anyone genuinely blocked gets moving again in minutes. Anyone burning budget for no reason gets a natural pause point.
6. Reserve manual review for real outliers
Databricks sets a heavy-user threshold. Its own examples cite a $500 illustrative default monthly limit, with $2,500 or more triggering manual review. That way human attention goes to the handful of accounts worth investigating, not every engineer who ran one big batch job.
Example prompts you can copy
These are prompts I use to get an agent to police its own token spend instead of relying only on gateway settings:
- Force a context audit: "Before continuing, tell me how much of the current context is repeated file content from earlier turns, and suggest what's safe to drop from the next request."
- Ask for model-appropriate scope: "This is a one-line typo fix. Don't re-read the whole file or re-explain the codebase — just make the change and show me the diff."
- Check for a runaway pattern: "If you find yourself repeating the same tool call more than twice with no new result, stop and tell me what's going wrong instead of retrying again."
- Push caching-friendly structure: "Keep the system prompt and file context identical across turns in this session so caching applies; only change the specific instruction at the end."
Common mistakes to avoid
In my testing, the first mistake I made was assuming a smart router alone would fix the bill. It doesn't, if the harness still re-sends redundant context on every call. Caching and router savings compound, but only if you do the caching cleanup first. Bolt a router onto a wasteful harness and you just route the waste to a cheaper model, instead of removing it. The second mistake is one spend limit per person instead of separate daily and monthly caps. A single monthly number either blocks normal heavy use too early, or lets a broken loop run for weeks before anyone notices. Third, teams treat every over-limit engineer as a problem to chase down, which burns the reviewer's time. Databricks' fixed-tier, self-service unblocking exists so only real outliers reach a human. Last, don't skip attribution. If spend isn't tied to a user identity in one gateway, you're optimizing blind. You can't tell which 10% of usage drives 90% of the bill.
Databricks' cost levers, compared
| Lever | What it does | Reported savings | Effort to set up |
|---|---|---|---|
| Harness + caching tuning | Cuts redundant tokens per request | ~50% fewer generated tokens | Low — mostly config |
| AI Gateway Smart Router | Routes each task to the cheapest capable model | 30%+ lower average task cost | Medium — needs a router in front of agents |
| Daily + monthly spend tiers | Separates runaway-loop risk from normal overspend | Stops month-long budget burns in a day | Medium — policy + Slack integration |
| Prompt caching (Claude Code) | Reuses unchanged context instead of re-sending it | Cached reads cost 90% less than uncached input tokens | Low — on by default in most agents |
I tested the caching piece myself rather than taking Databricks' number on faith. I ran the same multi-turn coding session through Claude Code twice: once letting the prompt cache do its job, once forcing a fresh context on every turn. The gap matched Anthropic’s own published pricing: cached reads bill at roughly a tenth of the price of an uncached input token. On a long session with a lot of repeated file context, that alone accounts for a big chunk of the savings Databricks reports at fleet scale. It's what made the 50% harness-and-caching figure believable to me, not just a marketing line.
Tools that make this easier
You don't need to build Databricks' Unity AI Gateway from scratch to get most of this. If you're deciding which coding agent to standardize on before you even worry about routing costs, my AI coding assistant guide and my Cursor vs. Copilot comparison cover where each tool is strongest. If Claude Code is already your daily driver, my breakdown of Claude Code usage tracking with LangWatch covers third-party options for the attribution piece Databricks built in-house. For teams weighing Codex as a second or cheaper model in the rotation, see my guide on how to use ChatGPT Codex, and for a from-scratch Claude Code setup that already leans on caching correctly, my Claude Code setup guide walks through the config. If cost is pushing you to mix providers the way Databricks does, my ChatGPT alternatives for coding roundup is a reasonable starting shortlist for the router's second option.
My take
The "70%" headline oversells a single moment. Databricks never states that exact combined figure in its own posts. The honest math is two separately verified reductions, about 50% from harness and caching, 30%+ from routing, that compound close to it when stacked. That's the more useful finding: neither lever alone gets you there. A team that only adds a router, without fixing a wasteful harness first, lands well short of what the headline implies. My verdict after testing the caching piece myself: start with harness and caching cleanup before you buy or build a router. It's the cheaper fix, and it makes the router's job easier. Budget tiers matter more once you have more than a handful of engineers running agents unsupervised.
Frequently Asked Questions
Did Databricks really cut AI coding spend by exactly 70%?
Not as a single stated figure. Databricks' own blog posts report an almost 50% cut in generated tokens from harness and caching tuning, and a 30%+ cut in average task cost from its AI Gateway Smart Router. Stacked together, those compound close to 70%, which is likely where the widely shared headline comes from.
What is Unity AI Gateway?
It's Databricks' internal routing and governance layer that every coding agent's traffic — Claude Code, Codex, Cursor, and others — passes through, so spend gets attributed to a user identity and metered in one place instead of scattered across separate provider accounts.
How long does it take to set up something similar?
The harness and caching cleanup can happen in an afternoon if you're already using an agent with built-in caching, like Claude Code. Centralizing routing and spend attribution across multiple agents is the bigger lift and took Databricks a dedicated engineering effort, not a weekend project.
What's the easiest first step for a small team?
Turn on prompt caching if your coding agent supports it and check whether your harness is re-sending full file context on every turn. That's the ~50% lever, it's mostly a config change, and it doesn't require building a router first.
Does this only work at Databricks' scale?
No. The router and budget-tier pieces matter more as headcount grows, but harness and caching tuning — the biggest single lever in Databricks' own numbers — applies just as well to a single developer's Claude Code session as it does to a fleet of a thousand engineers.