Snowflake Query Spilling to Remote Storage: Find & Fix It
Remote spill is the slowest, costliest way a Snowflake query can run. Here is the SQL to find every spilling query and dbt model, and how to fix them.
Fintel Analytics blog
Expert perspectives on data analytics, AI agents, internal tools, and business intelligence from the Fintel Analytics team.
42 of 206 posts
Remote spill is the slowest, costliest way a Snowflake query can run. Here is the SQL to find every spilling query and dbt model, and how to fix them.
When dbt models read from BigQuery views instead of materialised tables, BigQuery silently recomputes the full view logic every time — costing you money you never see coming. Here's how to find it and fix it.
BigQuery external tables look like a cheap shortcut in your dbt pipeline — until they silently inflate your query bill and break your freshness checks. Here is what actually happens in production.
dbt models can silently scan BigQuery tables that never appear in your DAG. Here's how to find them in under five minutes using INFORMATION_SCHEMA.JOBS.
Partitioned BigQuery tables without require_partition_filter set are silently full-scanned every run. Here's how to find every offending table and fix it.
JavaScript UDFs in dbt BigQuery silently consume disproportionate slot resources and block query cache hits. Here is how to find them, audit their cost, and fix them before they wreck your bill.
Your dbt project may be silently overpaying for BigQuery storage. Here is how to find the datasets where switching billing models saves real money.
When dbt models join across BigQuery datasets, partition pruning silently breaks — and your bill triples. Here's how to find and fix it.
Your BigQuery bill spiked and you have no idea which dbt model caused it. Here is how to fix that with dbt query labels — and the diagnostic SQL to prove it.
Non-deterministic SQL functions silently bust BigQuery's query cache across your entire dbt DAG. Here is how to find every offending model and fix it fast.
BigQuery table expiration silently deletes tables your dbt models depend on — often with no alerting. Here is how to find and fix it before production breaks.
Parsing nested JSON incorrectly in dbt BigQuery staging models is one of the most common — and most expensive — anti-patterns in modern analytics stacks. Here is how to find and fix it.
dbt models querying BigQuery wildcard tables without a _TABLE_SUFFIX filter scan every shard, every run — silently inflating your bill. Here is how to find them and fix them.
Your dbt incremental model has a WHERE clause, but BigQuery is still scanning the entire table on every run. Here's how to detect it and fix it for good.
dbt source fan-out silently duplicates BigQuery scans, breaks lineage, and strips your pipeline of test coverage. Here is how to find and fix it.
dbt script models using CREATE TEMP TABLE in BigQuery silently break lineage and inflate warehouse costs. Here is how to find and fix them.
The wrong materialisation on dbt intermediate models can silently triple your BigQuery bill. Here's how to detect it and fix it before it compounds.
When a dbt model references the same BigQuery source table more than once, BigQuery may scan it multiple times per run — silently inflating your bill with every pipeline execution.
Shuffle joins in BigQuery dbt models are one of the most costly and least visible performance problems a data team can have. Here is how to find and fix them.
dbt incremental models using the merge strategy in BigQuery can silently scan your entire target table on every run. Here's how to detect it and fix it fast.
Your BigQuery bill is growing but you can't pinpoint why. INFORMATION_SCHEMA has the answer — here's how to use it to attribute costs by user, model, and team before it's too late.
Your BigQuery tables are partitioned — but are your dbt models actually using those partitions? Here is how to find out and fix it before your bill does.
Choosing the wrong materialisation for dbt staging models is one of the most common — and most fixable — sources of runaway BigQuery costs. Here's the framework to get it right.
Repeated subqueries in dbt BigQuery models silently scan the same large table multiple times per run — find them with INFORMATION_SCHEMA and fix them today.
BigQuery offers free query result caching — but most dbt projects silently invalidate it on every run. Here's exactly why it breaks and how to fix it.
BigQuery silently re-evaluates CTEs every time they are referenced — a behaviour that can multiply query costs inside complex dbt models without a single warning.
dbt ephemeral models look efficient on paper but silently multiply BigQuery scan costs when referenced by multiple downstream models. Here is how to find and fix it.
Misusing STRUCT and ARRAY types in dbt BigQuery models silently inflates your bill and slows your pipelines. Here is how to find and fix it.
SELECT * in dbt BigQuery models is one of the quietest cost drivers in a growing data stack. Here's how to find every offending model and fix it.
A missing PARTITION BY in a BigQuery window function can silently scan millions of rows it never needed — and you won't see it until the invoice arrives. Here's how to find and fix it.
dbt full-refresh models in BigQuery silently balloon your cloud bill as data grows. Here's how to find the offenders, understand the real cost, and migrate safely to incremental.
BigQuery join skew silently inflates your bill and stalls dbt runs. Here is how to find it, fix it, and stop it coming back.
BigQuery materialized views can slash query costs and dashboard load times — but only when paired with dbt correctly. Here is the decision framework practitioners actually use.
Choosing the wrong BigQuery pricing model can double your bill. Here's the decision framework — with real numbers — to get it right at your stage of growth.
Over-materialised dbt models are one of the most common causes of runaway BigQuery bills. Here is how to find them and fix them before they compound.
Partitioned tables don't automatically mean cheaper queries. Discover the six silent mistakes that bypass partition pruning in BigQuery — and exactly how to fix them before your next bill arrives.
When you delete or rename a dbt model, BigQuery keeps the table. Here's how to detect orphaned ghost tables and safely drop them before they mislead your team.
A single misconfigured scheduled query can scan hundreds of TB before anyone notices. Here is how to find it — and fix it — in under five minutes.
dbt models that UNNEST repeated fields without a prior filter silently multiply row counts, inflate bytes billed, and corrupt downstream aggregations. Here is how to find and fix them.
Cloud data warehouse bills spiralling out of control? This practical guide covers the strategies, tools, and decisions that help businesses cut costs without sacrificing performance.
Full table rebuilds are silently draining your BigQuery budget. Here is the decision framework for getting dbt incremental models right — the first time.
Slow dbt models are costing you more than just time — they inflate cloud bills and delay decisions. Here's how to diagnose and fix the real causes.