Cost optimization
Where data platform costs hide: 12 places to look on Databricks and Snowflake
Twelve common sources of avoidable spend on Databricks and Snowflake, from idle compute to full refreshes, and how to find each one before your next renewal.
Hemant Walchale
· 2 min read
Data platform bills rarely grow because of one big mistake. They grow through dozens of small defaults nobody revisited. Here are the twelve places we look first.
Start with attribution
Before any of the list below, make spend visible by team, pipeline and query. On Databricks, use cluster and job tags with the system billing tables; on Snowflake, use warehouse naming, query tags and the account usage views. Agree a baseline with finance. Without it, savings can’t be proven, and they won’t be believed.
Compute
- Idle all-purpose clusters and warehouses. Interactive compute left running between work sessions. Set aggressive auto-termination and auto-suspend.
- Oversized compute. Clusters and warehouses sized for the worst day of the quarter, every day. Right-size on actual utilization, and scale out for peaks instead.
- Interactive compute running scheduled jobs. Jobs on all-purpose clusters cost more than on job compute. Move scheduled work to job clusters or serverless where it fits.
- One warehouse for everyone. A single large warehouse shared by BI, ad-hoc analysis and loads. Separate workloads so each can be sized and suspended on its own.
Pipelines
- Full refreshes that could be incremental. Rebuilding large tables nightly when only a small fraction changed. Incremental processing is often the single largest saving.
- Pipelines nobody uses. Jobs feeding reports that were retired years ago. Check consumers before you optimize, then delete.
- Over-frequent schedules. Hourly refreshes for data that’s read once a day.
- Small-file problems. Many tiny files slow every read. Compaction and sensible file sizes cut both runtime and cost.
Storage and data
- Duplicated datasets. The same data copied into several schemas or environments. Share or reference instead of copying.
- Unbounded retention. Time travel, fail-safe and historical snapshots kept longer than anyone needs. Set retention by data class.
Queries and usage
- A few expensive queries. A small number of queries usually dominate spend, often dashboards refreshing far more often than they’re viewed. Find the top 20 by cost and fix those first.
- No guardrails. Without budgets and alerts, costs creep back within months. Set budgets per team, alert on anomalies, and report spend against baseline monthly.
What results look like
In our analysis of 100 published Databricks and Snowflake customer stories, median reported cost reductions were around 50–60% depending on industry (see the benchmarks). Treat those as an optimistic ceiling (they’re the success stories vendors publish) and set your own target from your baseline.
Make it stick
Savings disappear when nobody owns them. Give each team its own budget and monthly report, review the top queries quarterly, and put cost checks into code review for new pipelines.
If you’d like this done for you, our cloud data cost optimization service starts with a one- to two-week baseline and a ranked savings backlog.
For how each platform bills, see Databricks pricing explained, Snowflake pricing explained and Microsoft Fabric pricing explained, plus 15 hidden costs on Databricks, Snowflake and Fabric.
Not sure where your own platform stands? Take the free 3-minute data platform scorecard, or see how we work on Databricks. Unfamiliar terms are explained in the data engineering glossary.