Fintel AnalyticsBigQuery Streaming Inserts Cost: Find What's Driving the Bill
Streaming ingestion is the BigQuery line item nobody budgets for. Run two INFORMATION_SCHEMA queries to find the tables behind it, then cut the cost.
Fintel Analytics blog
Data & AI Insights
Expert perspectives on data analytics, AI agents, internal tools, and business intelligence from the Fintel Analytics team.
111 of 207 posts
Snowflake Warehouses With No Resource Monitor: Find Them
Find every Snowflake warehouse running with no credit cap using runnable SQL, then set sensible quotas and stop the gap coming back.
Snowflake ACCOUNTADMIN Audit: Who Has It and Who Uses It?
Too many people, and too many dbt jobs, run as ACCOUNTADMIN. Use these Snowflake queries to find them and fix it safely.
Postgres to BigQuery Replication: CDC vs Batch Guide 2026
Batch or CDC for Postgres to BigQuery? A practitioner's framework covering hard deletes, hidden merge costs and the pipeline failures that surface in production.
Snowflake Automatic Clustering Cost: Find Tables Burning Credits
Serverless reclustering credits hide outside your warehouse bill. Run these SQL queries to find which dbt tables burn them and how to stop it.
Timezone Errors in Financial Reporting: Fix Your Day Boundaries
Why finance and ops report different daily totals for the same transactions, and how to fix timezone errors in BigQuery and dbt with one governed reporting date.
Snowflake Query Spilling to Remote Storage: Find & Fix It
Remote spill is the slowest, costliest way a Snowflake query can run. Here is the SQL to find every spilling query and dbt model, and how to fix them.
Redshift to BigQuery Migration: When It Pays Off in 2026
Is a Redshift to BigQuery migration worth it? A practitioner's guide to the break-even maths, the failure modes, and how to validate every number.
Undocumented dbt Models: Find Every Mart With No Description
Run two commands and one BigQuery query to list every undocumented dbt model in your project, rank them by real usage, and stop new ones shipping.
Snowflake Time Travel Storage Cost: Find Bloated Tables
Your active data is flat but Snowflake storage keeps climbing. Here is the SQL to find the tables responsible, and the dbt config that fixes it.
Snowflake Idle Warehouse Cost: Find the Credits You're Burning
Your Snowflake bill may be paying for warehouses that run nothing. Here is the SQL to measure idle credits per warehouse, fix auto-suspend and stop it coming back.
Spreadsheet to dbt Migration: Move Finance Logic Into SQL
A practitioner's guide to moving business-critical spreadsheet logic into tested dbt models. Learn what to migrate first and how to prove the numbers match.
BigQuery Service Account Key in dbt Repo: Find & Fix It
A BigQuery service account key committed to a dbt repo is a live credential, not a tidy-up task. Find it, rotate it and remove it for good.
BigQuery Dynamic Data Masking in dbt: The Access Pattern That Breaks at Scale
Building data masking logic inside dbt models is one of the most quietly dangerous patterns in a growing warehouse. Here's why it breaks — and how to fix it.
dbt Models Reading BigQuery Views: The Hidden Recompute Tax
When dbt models read from BigQuery views instead of materialised tables, BigQuery silently recomputes the full view logic every time — costing you money you never see coming. Here's how to find it and fix it.
BigQuery External Tables as dbt Sources: The Hidden Traps
BigQuery external tables look like a cheap shortcut in your dbt pipeline — until they silently inflate your query bill and break your freshness checks. Here is what actually happens in production.
BigQuery referenced_tables: Find dbt Models Scanning Ghost Tables
dbt models can silently scan BigQuery tables that never appear in your DAG. Here's how to find them in under five minutes using INFORMATION_SCHEMA.JOBS.
dbt Macros vs Persistent SQL UDFs in BigQuery: Choose Right
Choosing between dbt macros and persistent SQL UDFs in BigQuery is a decision most teams get wrong — and the cost shows up in your bill and your lineage graph.
BigQuery require_partition_filter: Find Tables Missing the Guard
Partitioned BigQuery tables without require_partition_filter set are silently full-scanned every run. Here's how to find every offending table and fix it.
JavaScript UDFs in dbt BigQuery: The Slot Cost You're Missing
JavaScript UDFs in dbt BigQuery silently consume disproportionate slot resources and block query cache hits. Here is how to find them, audit their cost, and fix them before they wreck your bill.
BigQuery Logical vs Physical Storage in dbt: The Silent Bill
Your dbt project may be silently overpaying for BigQuery storage. Here is how to find the datasets where switching billing models saves real money.
dbt Cross-Dataset Joins in BigQuery: Fix the Silent Scan
When dbt models join across BigQuery datasets, partition pruning silently breaks — and your bill triples. Here's how to find and fix it.
dbt Query Labels in BigQuery: Fix Your Cost Attribution (2026)
Your BigQuery bill spiked and you have no idea which dbt model caused it. Here is how to fix that with dbt query labels — and the diagnostic SQL to prove it.
BigQuery Column-Level Encryption in dbt: Protect Sensitive Data
BigQuery column-level encryption lets you protect PII and financial data at the field level — but wiring it correctly into dbt without breaking downstream models is harder than the docs suggest.
Non-Deterministic Functions in dbt BigQuery: Why Your Cache Never Hits
Non-deterministic SQL functions silently bust BigQuery's query cache across your entire dbt DAG. Here is how to find every offending model and fix it fast.
dbt Authorized Views in BigQuery: Fix Broken Access Control
Most dbt BigQuery teams grant dataset-level IAM instead of using authorized views — here's why that breaks security and exactly how to fix it.
BigQuery Table Expiration Breaking dbt Models: Find & Fix It
BigQuery table expiration silently deletes tables your dbt models depend on — often with no alerting. Here is how to find and fix it before production breaks.
Nested JSON in dbt BigQuery Staging: Fix the Anti-Pattern
Parsing nested JSON incorrectly in dbt BigQuery staging models is one of the most common — and most expensive — anti-patterns in modern analytics stacks. Here is how to find and fix it.
dbt Wildcard Tables in BigQuery Without _TABLE_SUFFIX Filter: The Silent Cost
dbt models querying BigQuery wildcard tables without a _TABLE_SUFFIX filter scan every shard, every run — silently inflating your bill. Here is how to find them and fix them.
dbt Multi-Project Architecture in BigQuery: Split It Right
Running a single dbt project works — until it doesn't. Learn when to split into multiple projects in BigQuery, how to do it without breaking lineage, and the hidden costs teams miss.
dbt Incremental Subquery Filter in BigQuery: Full Scan Every Run
Your dbt incremental model has a WHERE clause, but BigQuery is still scanning the entire table on every run. Here's how to detect it and fix it for good.
dbt Source Fan-Out in BigQuery: The DAG Anti-Pattern Costing You
dbt source fan-out silently duplicates BigQuery scans, breaks lineage, and strips your pipeline of test coverage. Here is how to find and fix it.
dbt Script Models & Temp Tables in BigQuery: The Hidden Cost
dbt script models using CREATE TEMP TABLE in BigQuery silently break lineage and inflate warehouse costs. Here is how to find and fix them.
dbt Intermediate Models in BigQuery: The Materialisation Mistake Costing You
The wrong materialisation on dbt intermediate models can silently triple your BigQuery bill. Here's how to detect it and fix it before it compounds.
dbt Models Scanning the Same BigQuery Table Twice: Find & Fix It
When a dbt model references the same BigQuery source table more than once, BigQuery may scan it multiple times per run — silently inflating your bill with every pipeline execution.
BigQuery Shuffle Joins in dbt: Fix the Hidden Cost in 2026
Shuffle joins in BigQuery dbt models are one of the most costly and least visible performance problems a data team can have. Here is how to find and fix them.
dbt MERGE Strategy in BigQuery: Why It's Scanning Your Whole Table
dbt incremental models using the merge strategy in BigQuery can silently scan your entire target table on every run. Here's how to detect it and fix it fast.
BigQuery INFORMATION_SCHEMA Cost Attribution: Stop the Guessing
Your BigQuery bill is growing but you can't pinpoint why. INFORMATION_SCHEMA has the answer — here's how to use it to attribute costs by user, model, and team before it's too late.
dbt run-operation in BigQuery: The Lineage Blind Spot (2026)
Using dbt run-operation or BigQuery stored procedures for transformations hides logic from your DAG, breaks lineage, and silently corrupts downstream models. Here is how to find and fix it.
Late-Arriving Data in dbt BigQuery: Fix Silent Metric Corruption
Late-arriving data silently corrupts dbt incremental models in BigQuery — producing wrong metrics that nobody catches until a decision has already been made. Here is how to fix it.
dbt Models Querying Partitioned BigQuery Tables Without Partition Filters
Your BigQuery tables are partitioned — but are your dbt models actually using those partitions? Here is how to find out and fix it before your bill does.
dbt Staging Layer Materialisation in BigQuery: Fix the Cost Leak
Choosing the wrong materialisation for dbt staging models is one of the most common — and most fixable — sources of runaway BigQuery costs. Here's the framework to get it right.
Repeated Subqueries in dbt BigQuery: Stop Paying Twice (2026)
Repeated subqueries in dbt BigQuery models silently scan the same large table multiple times per run — find them with INFORMATION_SCHEMA and fix them today.
BigQuery Query Cache Invalidation in dbt: Why Your Cache Never Hits
BigQuery offers free query result caching — but most dbt projects silently invalidate it on every run. Here's exactly why it breaks and how to fix it.
Hardcoded Table Names in dbt BigQuery: Find the Lineage Gaps
Hardcoded table names in dbt BigQuery models silently break your DAG, making downstream failures invisible until something stops working in production.
BigQuery CTEs Re-evaluated Every Time: The dbt Cost You're Missing
BigQuery silently re-evaluates CTEs every time they are referenced — a behaviour that can multiply query costs inside complex dbt models without a single warning.
dbt Ephemeral Models in BigQuery: The Hidden Cost Trap
dbt ephemeral models look efficient on paper but silently multiply BigQuery scan costs when referenced by multiple downstream models. Here is how to find and fix it.
BigQuery STRUCT & ARRAY Misuse in dbt: The Hidden Cost
Misusing STRUCT and ARRAY types in dbt BigQuery models silently inflates your bill and slows your pipelines. Here is how to find and fix it.
SELECT * in dbt BigQuery Models: The Silent Cost Multiplier
SELECT * in dbt BigQuery models is one of the quietest cost drivers in a growing data stack. Here's how to find every offending model and fix it.
BigQuery Window Functions Without PARTITION BY: The Silent Cost Killer
A missing PARTITION BY in a BigQuery window function can silently scan millions of rows it never needed — and you won't see it until the invoice arrives. Here's how to find and fix it.
dbt UNNEST Without Filters in BigQuery: Fix the Row Explosion
dbt models that UNNEST repeated fields without a prior filter silently multiply row counts, inflate bytes billed, and corrupt downstream aggregations. Here is how to find and fix them.
dbt Full-Refresh Models in BigQuery: Stop the Silent Cost Bleed
dbt full-refresh models in BigQuery silently balloon your cloud bill as data grows. Here's how to find the offenders, understand the real cost, and migrate safely to incremental.
Accidental Cross Joins in dbt BigQuery: Detect & Fix Them
Accidental cross joins in dbt BigQuery models can turn a 1M-row table into a trillion-row disaster — and your bill won't wait. Here's how to find and fix them fast.
BigQuery Join Skew in dbt Models: Find & Fix It in 2026
BigQuery join skew silently inflates your bill and stalls dbt runs. Here is how to find it, fix it, and stop it coming back.
dbt Models With No Tests: Find Them in BigQuery Now
Untested dbt models are silent liabilities. Learn how to find every model with no test coverage in BigQuery using runnable SQL and dbt commands — in under five minutes.
BigQuery Materialized Views With dbt: When to Use Them
BigQuery materialized views can slash query costs and dashboard load times — but only when paired with dbt correctly. Here is the decision framework practitioners actually use.
dbt Sources With No Freshness Config: Find Them in BigQuery
Every dbt source without a freshness config is a silent risk. Here's how to find every unchecked source in under five minutes using BigQuery INFORMATION_SCHEMA — and how to fix it.
BigQuery On-Demand vs Slot Reservations: Which to Choose
Choosing the wrong BigQuery pricing model can double your bill. Here's the decision framework — with real numbers — to get it right at your stage of growth.
dbt Models Materialised as Table in BigQuery: Fix the Cost Leak
Over-materialised dbt models are one of the most common causes of runaway BigQuery bills. Here is how to find them and fix them before they compound.
BigQuery Partitioning & Clustering Mistakes Killing Your Bill
Partitioned tables don't automatically mean cheaper queries. Discover the six silent mistakes that bypass partition pruning in BigQuery — and exactly how to fix them before your next bill arrives.
Orphaned dbt Models in BigQuery: Find & Drop Ghost Tables
When you delete or rename a dbt model, BigQuery keeps the table. Here's how to detect orphaned ghost tables and safely drop them before they mislead your team.
dbt Lineage Blind Spots: Why Downstream Models Break Silently
Undocumented dbt dependencies are silently breaking production dashboards at growth-stage companies. Here is how to audit your DAG and fix it before it costs you.
BigQuery Scheduled Query Costs: Find the Query Wrecking Your Bill
A single misconfigured scheduled query can scan hundreds of TB before anyone notices. Here is how to find it — and fix it — in under five minutes.
dbt Project Governance for Startups: Stop Model Sprawl
Model sprawl is the silent killer of growing dbt projects. Learn how to govern your dbt project before undocumented models, broken DAGs, and degraded test coverage cost you.
Over-Broad IAM Grants in BigQuery: Find & Fix Them Now
Over-broad BigQuery IAM grants are the most common—and most quietly dangerous—misconfiguration in fast-growing data teams. Here's how to find them in under five minutes.
BigQuery Access Control: Row & Column Security Done Right
Most growth-stage companies running BigQuery give everyone too much access. Here is how to implement row-level and column-level security that scales with your team.
PII Sitting Unmasked in BigQuery: How to Find It Fast
Unmasked PII in BigQuery is a compliance and security risk most teams discover too late. Here's how to find it in under five minutes using SQL against INFORMATION_SCHEMA.
dbt Snapshots in BigQuery: Stop Losing Historical Dimension Data
Most early-stage data teams don't realise they are silently overwriting dimension history. Here is how to fix it before it corrupts your reporting.
Silent NULL Propagation in dbt BigQuery: Find & Fix It
Silent NULLs in dbt and BigQuery pass every test, build successfully, and corrupt your metrics for weeks before anyone notices. Here is how to find them.
Multi-Tenant SaaS Analytics: Build It Right in 2026
Multi-tenant SaaS analytics is one of the most under-engineered parts of a B2B product. Get the architecture wrong and you risk data leaks, broken dashboards, and a rebuild at scale.
dbt Fan-Out Joins in BigQuery: Detect & Fix Inflated Metrics
Fan-out joins silently inflate dbt metrics in BigQuery — revenue looks higher, churn looks lower, and nobody notices until a stakeholder asks why the numbers don't add up.
dbt Incremental Models Strategy: When and How to Use Them
Full table rebuilds are silently draining your BigQuery budget. Here is the decision framework for getting dbt incremental models right — the first time.
dbt Source Schema Drift: Find It Before It Breaks Production
dbt's default on_schema_change='ignore' silently drops new source columns from incremental models. Here's how to detect and fix it before your dashboards lie to you.
Reverse ETL & Data Activation: Operationalise Your Warehouse in 2026
Your warehouse holds your best data — but it rarely reaches the tools where decisions get made. Here is how to fix that with reverse ETL and data activation.
Pipeline Incident Analytics: Stop Silent Data Failures in 2026
Silent data failures are the most expensive kind — your pipeline runs green while bad data corrupts every dashboard downstream. Here's how to build the analytics layer that catches them first.
Webhook Ingestion Analytics: Stop Losing Events in Your Pipeline
Webhook events are failing silently — and your dashboards are lying to you because of it. Learn how to build reliable webhook ingestion analytics that protect revenue and power trusted metrics.
API Product Analytics: Fix Metering Blind Spots & Revenue Leakage
Most SaaS and platform businesses have no structured analytics layer between their API logs and their billing systems — and that gap is silently costing them revenue. Here is how to fix it.
Internal Data Marketplace for Startups: Build It Right in 2026
An internal data marketplace stops teams rebuilding the same pipelines and fighting over different numbers. Here is how to build one that actually works at startup scale.
Usage-Based Billing Analytics: Fix Your Revenue Data in 2026
Usage-based billing creates complex revenue data that breaks most standard analytics stacks. Here is how to build the data layer that actually works.
Why Your Data Stack Breaks Under VC Due Diligence (And How to Fix It Before Your Series A)
Most early-stage data stacks look fine until a VC opens a data room request. Here's what breaks, why it costs founders term sheets, and how to fix it.
RegTech Data Pipelines: Automate Compliance Workflows in 2026
Manual compliance is costing fintechs more than they realise — in headcount, fines, and failed audits. Here is how to build the data infrastructure that fixes it.
Event-Driven Data Architecture for Fintech: Build It Right in 2026
Most fintechs collect event data but fail to operationalise it. Here's how to build an event-driven data architecture that gives you real-time intelligence — without the chaos.
Fintech Ledger Analytics: Fix Broken Balance Intelligence in 2026
Most fintechs have a ledger. Very few have analytics built on top of it. This guide shows exactly what breaks, and how to fix it.
Virtual IBAN Analytics: Fix Broken vIBAN Data in 2026
Virtual IBANs promise automated reconciliation — but the data stack behind them is usually broken. Here is how to fix it before it costs you.
Settlement Timing Analytics: Fix Your Post-Trade Data Stack
Settlement timing analytics tell you exactly where your post-trade process breaks down — before regulators or counterparties do. Here is how to build the data stack that survives T+1.
BaaS Analytics: Build the Data Stack Behind Embedded Banking
Most BaaS-powered products ship fast but build no analytics layer. Here's how to fix that — and what it costs you if you don't.
Payment Gateway Routing Analytics: Fix Your Routing Blind Spots
Most fintechs run multi-gateway stacks but have no analytics layer to tell them whether their routing logic is working. Here's how to fix that.
Alternative Data Analytics for Fintech: Build the Edge in 2026
Alternative data analytics gives fintech startups a real intelligence edge — but only if your data pipeline can actually ingest, clean, and model it at speed. Here's how to build it right.
Acquirer Analytics: Build the Data Stack That Wins in 2026
Most acquirers are flying blind on the metrics that matter most. Here is how to build the internal analytics stack that turns transaction data into competitive edge.
KYC/AML Compliance Analytics for Fintech: Build It Right in 2026
Most fintechs have a KYC vendor. Very few have a compliance analytics layer behind it. Here is how to build one that actually works in 2026.
Open Banking Data Analytics: Turn API Feeds Into Intelligence
Open banking API feeds generate enormous volumes of raw transaction data — but most fintech teams lack the analytics layer to turn it into decisions. Here's how to fix that.
Multi-Currency Analytics: Fix Broken Financials in 2026
Multi-currency analytics is one of the most underengineered problems in fintech data stacks. This guide shows how to fix it before it costs you real money.
PSP Data Analytics: Build Intelligence Into Your Payment Stack
Most fintechs track whether payments succeed or fail — but few build analytics that reveal why. This guide shows how to unlock real intelligence from your PSP data layer.
dbt Testing Strategy for Startups: Stop Shipping Bad Data
Most startups don't find out their data is broken until a stakeholder catches it in a board deck. A proper dbt testing strategy changes that — here's how to build one.
Regulatory Reporting Analytics for Fintech: Build It Right in 2026
Manual regulatory reporting is costing fintech teams time, money, and regulatory goodwill. Here is how to build analytics infrastructure that makes compliance a data engineering problem — and solves it permanently.
When Does a Startup Need a Data Warehouse? A 2026 Guide
Knowing when to graduate from spreadsheets to a data warehouse is one of the most consequential infrastructure calls a founder makes. Here is the framework we use.
Financial Close Analytics: Cut Your Month-End Close in 2026
Half of finance teams still take six or more days to close the books. Financial close analytics fixes the broken data pipelines causing that delay — here is how to build it.
B2B SaaS Metrics Stack: Build the Analytics That Scale
Most B2B SaaS companies reach Series A with their core growth metrics — ARR, NRR, churn — still living in spreadsheets. Here is how to fix that before it costs you.
DataOps for Startups: Build Reliable Data Pipelines in 2026
Most startups don't realise their data pipeline is broken until a board meeting goes sideways. Here's how to build reliable data infrastructure from day one — before it costs you.
SQL Semantic Layer: Why Your Metrics Are Broken in 2026
If finance and ops see different revenue figures, you don't have a dashboard problem — you have a semantic layer problem. Here's how to fix it.
Why Your dbt Models Are Running Slow — And How to Fix It
Slow dbt models are costing you more than just time — they inflate cloud bills and delay decisions. Here's how to diagnose and fix the real causes.
API Analytics for Fintech: Turn Your API Layer Into a BI Asset
Your APIs process every payment, every user action, every integration event — but most fintech teams never analyse that data. Here is how to fix that.
Payments Reconciliation Analytics: Automate & Scale in 2026
Manual payments reconciliation breaks at scale — and the cost is higher than most fintech teams realise. Here is how to automate it properly with modern data engineering.
Real-Time Personalisation Engines: Data Architecture in 2026
Real-time personalisation engines are reshaping how businesses engage customers. Discover the data architecture, use cases, and ROI behind building one in 2026.
Edge Analytics: Processing Data Closer to the Source in 2026
Edge analytics moves data processing closer to where data is generated — cutting latency, reducing costs, and enabling faster decisions at scale in 2026.
Data Lakehouse Architecture: The 2026 Business Guide
Data lakehouse architecture is reshaping how organisations store, govern, and analyse data. Learn why leading enterprises are making the switch in 2026.
Data Observability: How to Build Trust in Your Data Pipeline
Data observability is the silent backbone of trustworthy analytics. Discover how leading organisations are using it to eliminate costly data failures in 2026.
Real-Time Data Pipelines: A Business Leader's Guide to Streaming Analytics
Real-time data pipelines are transforming how businesses react to change. Learn how streaming analytics works and why it matters in 2026.
Cloud Data Warehouse Cost Optimisation: A 2026 Guide
Cloud data warehouse bills spiralling out of control? This practical guide covers the strategies, tools, and decisions that help businesses cut costs without sacrificing performance.
Data Mesh Architecture for Large Organisations: A 2026 Guide
Data mesh is reshaping how large organisations manage and scale their data. Learn the core principles, real-world benefits, and how to get started in 2026.
Real-Time Data Pipelines: A Guide to Streaming Analytics
Real-time data pipelines are reshaping how businesses make decisions. Learn how streaming analytics works, where it delivers value, and how to get started.