Fintel Analytics blog
Expert perspectives on data analytics, AI agents, internal tools, and business intelligence from the Fintel Analytics team.
53 of 206 posts
JavaScript UDFs in dbt BigQuery silently consume disproportionate slot resources and block query cache hits. Here is how to find them, audit their cost, and fix them before they wreck your bill.
dbt script models using CREATE TEMP TABLE in BigQuery silently break lineage and inflate warehouse costs. Here is how to find and fix them.
Shuffle joins in BigQuery dbt models are one of the most costly and least visible performance problems a data team can have. Here is how to find and fix them.
Your BigQuery bill is growing but you can't pinpoint why. INFORMATION_SCHEMA has the answer — here's how to use it to attribute costs by user, model, and team before it's too late.
Choosing the wrong materialisation for dbt staging models is one of the most common — and most fixable — sources of runaway BigQuery costs. Here's the framework to get it right.
BigQuery silently re-evaluates CTEs every time they are referenced — a behaviour that can multiply query costs inside complex dbt models without a single warning.
dbt ephemeral models look efficient on paper but silently multiply BigQuery scan costs when referenced by multiple downstream models. Here is how to find and fix it.
SELECT * in dbt BigQuery models is one of the quietest cost drivers in a growing data stack. Here's how to find every offending model and fix it.
dbt full-refresh models in BigQuery silently balloon your cloud bill as data grows. Here's how to find the offenders, understand the real cost, and migrate safely to incremental.
Your BigQuery bill spiked and you have no idea which dbt model caused it. Here is how to fix that with dbt query labels — and the diagnostic SQL to prove it.
dbt models querying BigQuery wildcard tables without a _TABLE_SUFFIX filter scan every shard, every run — silently inflating your bill. Here is how to find them and fix them.
dbt source fan-out silently duplicates BigQuery scans, breaks lineage, and strips your pipeline of test coverage. Here is how to find and fix it.
The wrong materialisation on dbt intermediate models can silently triple your BigQuery bill. Here's how to detect it and fix it before it compounds.
Misusing STRUCT and ARRAY types in dbt BigQuery models silently inflates your bill and slows your pipelines. Here is how to find and fix it.
A missing PARTITION BY in a BigQuery window function can silently scan millions of rows it never needed — and you won't see it until the invoice arrives. Here's how to find and fix it.
Over-materialised dbt models are one of the most common causes of runaway BigQuery bills. Here is how to find them and fix them before they compound.
A single misconfigured scheduled query can scan hundreds of TB before anyone notices. Here is how to find it — and fix it — in under five minutes.
Remote spill is the slowest, costliest way a Snowflake query can run. Here is the SQL to find every spilling query and dbt model, and how to fix them.
When dbt models read from BigQuery views instead of materialised tables, BigQuery silently recomputes the full view logic every time — costing you money you never see coming. Here's how to find it and fix it.
BigQuery external tables look like a cheap shortcut in your dbt pipeline — until they silently inflate your query bill and break your freshness checks. Here is what actually happens in production.
dbt models can silently scan BigQuery tables that never appear in your DAG. Here's how to find them in under five minutes using INFORMATION_SCHEMA.JOBS.
Partitioned BigQuery tables without require_partition_filter set are silently full-scanned every run. Here's how to find every offending table and fix it.
When dbt models join across BigQuery datasets, partition pruning silently breaks — and your bill triples. Here's how to find and fix it.
Non-deterministic SQL functions silently bust BigQuery's query cache across your entire dbt DAG. Here is how to find every offending model and fix it fast.
BigQuery table expiration silently deletes tables your dbt models depend on — often with no alerting. Here is how to find and fix it before production breaks.
Parsing nested JSON incorrectly in dbt BigQuery staging models is one of the most common — and most expensive — anti-patterns in modern analytics stacks. Here is how to find and fix it.
Your dbt incremental model has a WHERE clause, but BigQuery is still scanning the entire table on every run. Here's how to detect it and fix it for good.
When a dbt model references the same BigQuery source table more than once, BigQuery may scan it multiple times per run — silently inflating your bill with every pipeline execution.
dbt incremental models using the merge strategy in BigQuery can silently scan your entire target table on every run. Here's how to detect it and fix it fast.
Your BigQuery tables are partitioned — but are your dbt models actually using those partitions? Here is how to find out and fix it before your bill does.
Repeated subqueries in dbt BigQuery models silently scan the same large table multiple times per run — find them with INFORMATION_SCHEMA and fix them today.
BigQuery offers free query result caching — but most dbt projects silently invalidate it on every run. Here's exactly why it breaks and how to fix it.
Accidental cross joins in dbt BigQuery models can turn a 1M-row table into a trillion-row disaster — and your bill won't wait. Here's how to find and fix them fast.
Choosing the wrong BigQuery pricing model can double your bill. Here's the decision framework — with real numbers — to get it right at your stage of growth.
Partitioned tables don't automatically mean cheaper queries. Discover the six silent mistakes that bypass partition pruning in BigQuery — and exactly how to fix them before your next bill arrives.
Cloud data warehouse bills spiralling out of control? This practical guide covers the strategies, tools, and decisions that help businesses cut costs without sacrificing performance.
Batch or CDC for Postgres to BigQuery? A practitioner's framework covering hard deletes, hidden merge costs and the pipeline failures that surface in production.
Choosing between dbt macros and persistent SQL UDFs in BigQuery is a decision most teams get wrong — and the cost shows up in your bill and your lineage graph.
Running a single dbt project works — until it doesn't. Learn when to split into multiple projects in BigQuery, how to do it without breaking lineage, and the hidden costs teams miss.
dbt models that UNNEST repeated fields without a prior filter silently multiply row counts, inflate bytes billed, and corrupt downstream aggregations. Here is how to find and fix them.
BigQuery materialized views can slash query costs and dashboard load times — but only when paired with dbt correctly. Here is the decision framework practitioners actually use.
Full table rebuilds are silently draining your BigQuery budget. Here is the decision framework for getting dbt incremental models right — the first time.
Most fintech and payments businesses are losing basis points on every transaction without knowing it. Here is how to build interchange fee analytics that actually show you where the money is going.
Slow dbt models are costing you more than just time — they inflate cloud bills and delay decisions. Here's how to diagnose and fix the real causes.
Undocumented dbt dependencies are silently breaking production dashboards at growth-stage companies. Here is how to audit your DAG and fix it before it costs you.
Model sprawl is the silent killer of growing dbt projects. Learn how to govern your dbt project before undocumented models, broken DAGs, and degraded test coverage cost you.
Most fintech ops teams fly blind on the metrics that matter most — settlement timing, PSP cost efficiency, and reconciliation breaks. Here is how to fix that with a proper payment operations analytics layer.
Multi-currency analytics is one of the most underengineered problems in fintech data stacks. This guide shows how to fix it before it costs you real money.
Manual regulatory reporting is costing fintech teams time, money, and regulatory goodwill. Here is how to build analytics infrastructure that makes compliance a data engineering problem — and solves it permanently.
Most fintech startups fly blind on treasury until it costs them. Learn how to build real liquidity visibility with data engineering and BI — before the gap becomes a crisis.
Most B2B SaaS companies reach Series A with their core growth metrics — ARR, NRR, churn — still living in spreadsheets. Here is how to fix that before it costs you.
Most startups don't realise their data pipeline is broken until a board meeting goes sideways. Here's how to build reliable data infrastructure from day one — before it costs you.
Manual payments reconciliation breaks at scale — and the cost is higher than most fintech teams realise. Here is how to automate it properly with modern data engineering.
Fintel Analytics