BigQuery Dynamic Data Masking in dbt: The Access Pattern That Breaks at Scale
Building data masking logic inside dbt models is one of the most quietly dangerous patterns in a growing warehouse. Here's why it breaks — and how to fix it.
Fintel Analytics blog
Expert perspectives on data analytics, AI agents, internal tools, and business intelligence from the Fintel Analytics team.
148 of 206 posts
Building data masking logic inside dbt models is one of the most quietly dangerous patterns in a growing warehouse. Here's why it breaks — and how to fix it.
When dbt models join across BigQuery datasets, partition pruning silently breaks — and your bill triples. Here's how to find and fix it.
BigQuery column-level encryption lets you protect PII and financial data at the field level — but wiring it correctly into dbt without breaking downstream models is harder than the docs suggest.
Late-arriving data silently corrupts dbt incremental models in BigQuery — producing wrong metrics that nobody catches until a decision has already been made. Here is how to fix it.
Most early-stage data teams don't realise they are silently overwriting dimension history. Here is how to fix it before it corrupts your reporting.
Your warehouse holds your best data — but it rarely reaches the tools where decisions get made. Here is how to fix that with reverse ETL and data activation.
Silent data failures are the most expensive kind — your pipeline runs green while bad data corrupts every dashboard downstream. Here's how to build the analytics layer that catches them first.
An internal data marketplace stops teams rebuilding the same pipelines and fighting over different numbers. Here is how to build one that actually works at startup scale.
Usage-based billing creates complex revenue data that breaks most standard analytics stacks. Here is how to build the data layer that actually works.
Most early-stage data stacks look fine until a VC opens a data room request. Here's what breaks, why it costs founders term sheets, and how to fix it.
Manual compliance is costing fintechs more than they realise — in headcount, fines, and failed audits. Here is how to build the data infrastructure that fixes it.
Most fintechs collect event data but fail to operationalise it. Here's how to build an event-driven data architecture that gives you real-time intelligence — without the chaos.
Virtual IBANs promise automated reconciliation — but the data stack behind them is usually broken. Here is how to fix it before it costs you.
Settlement timing analytics tell you exactly where your post-trade process breaks down — before regulators or counterparties do. Here is how to build the data stack that survives T+1.
Alternative data analytics gives fintech startups a real intelligence edge — but only if your data pipeline can actually ingest, clean, and model it at speed. Here's how to build it right.
Most neobanks scale their product long before they scale their data. Here's how to build the analytics stack that actually supports growth, profitability, and investor scrutiny.
Most acquirers are flying blind on the metrics that matter most. Here is how to build the internal analytics stack that turns transaction data into competitive edge.
Most crypto exchanges generate enormous volumes of data but operate with surprisingly immature internal analytics. Here's how to fix that — from fee revenue modelling to liquidity dashboards.
Stablecoin payment volumes are exploding — but the data infrastructure most fintechs are running is not built for on-chain flows. Here is how to fix that.
Open banking API feeds generate enormous volumes of raw transaction data — but most fintech teams lack the analytics layer to turn it into decisions. Here's how to fix that.
Digital wallet analytics breaks when transaction data isn't modelled correctly. Learn how to build a reliable, reconcilable analytics layer on top of your wallet data.
Most fintechs track whether payments succeed or fail — but few build analytics that reveal why. This guide shows how to unlock real intelligence from your PSP data layer.
Most startups don't find out their data is broken until a stakeholder catches it in a board deck. A proper dbt testing strategy changes that — here's how to build one.
Knowing when to graduate from spreadsheets to a data warehouse is one of the most consequential infrastructure calls a founder makes. Here is the framework we use.
Most startups don't realise their data pipeline is broken until a board meeting goes sideways. Here's how to build reliable data infrastructure from day one — before it costs you.
Real-time personalisation engines are reshaping how businesses engage customers. Discover the data architecture, use cases, and ROI behind building one in 2026.
Synthetic data is transforming how organisations train AI models — removing privacy barriers, reducing costs, and accelerating development timelines. Here's what business leaders need to know in 2026.
Edge analytics moves data processing closer to where data is generated — cutting latency, reducing costs, and enabling faster decisions at scale in 2026.
Data lakehouse architecture is reshaping how organisations store, govern, and analyse data. Learn why leading enterprises are making the switch in 2026.
Knowledge graph analytics is reshaping how enterprises understand connected data. Discover how leading organisations are using graph intelligence to unlock insights traditional databases simply cannot surface.
Data observability is the silent backbone of trustworthy analytics. Discover how leading organisations are using it to eliminate costly data failures in 2026.
Real-time data pipelines are transforming how businesses react to change. Learn how streaming analytics works and why it matters in 2026.
Cloud data warehouse bills spiralling out of control? This practical guide covers the strategies, tools, and decisions that help businesses cut costs without sacrificing performance.
Data mesh is reshaping how large organisations manage and scale their data. Learn the core principles, real-world benefits, and how to get started in 2026.
Real-time data pipelines are reshaping how businesses make decisions. Learn how streaming analytics works, where it delivers value, and how to get started.
Most finance teams have spend management tools — but still can't answer basic questions about where money is going. Here's why, and how to fix it.
Most BaaS-powered products ship fast but build no analytics layer. Here's how to fix that — and what it costs you if you don't.
Network tokens promise higher auth rates and lower fraud — but most fintechs can't measure whether they're delivering. Here's how to build the analytics layer that proves it.
Revenue-based financing is growing fast — but most RBF platforms are flying blind on portfolio risk, repayment performance, and unit economics. Here is how to fix that.
Manual regulatory reporting is costing fintech teams time, money, and regulatory goodwill. Here is how to build analytics infrastructure that makes compliance a data engineering problem — and solves it permanently.
Most fintech startups fly blind on treasury until it costs them. Learn how to build real liquidity visibility with data engineering and BI — before the gap becomes a crisis.
Manual payments reconciliation breaks at scale — and the cost is higher than most fintech teams realise. Here is how to automate it properly with modern data engineering.
Your active data is flat but Snowflake storage keeps climbing. Here is the SQL to find the tables responsible, and the dbt config that fixes it.
Your dbt project may be silently overpaying for BigQuery storage. Here is how to find the datasets where switching billing models saves real money.
Most dbt BigQuery teams grant dataset-level IAM instead of using authorized views — here's why that breaks security and exactly how to fix it.
Shuffle joins in BigQuery dbt models are one of the most costly and least visible performance problems a data team can have. Here is how to find and fix them.
SELECT * in dbt BigQuery models is one of the quietest cost drivers in a growing data stack. Here's how to find every offending model and fix it.
dbt full-refresh models in BigQuery silently balloon your cloud bill as data grows. Here's how to find the offenders, understand the real cost, and migrate safely to incremental.
Over-broad BigQuery IAM grants are the most common—and most quietly dangerous—misconfiguration in fast-growing data teams. Here's how to find them in under five minutes.
Multi-tenant SaaS analytics is one of the most under-engineered parts of a B2B product. Get the architecture wrong and you risk data leaks, broken dashboards, and a rebuild at scale.
Fintech platforms with partner ecosystems rarely have clean visibility into how revenue is split, earned, and reconciled. Here is how to fix that with a proper data stack.
Most fintech treasury teams are managing intraday liquidity from delayed reports and spreadsheets. Here is how to build the data stack that gives you real-time visibility — and why it matters more than ever in 2026.
Most fintech lenders drown in loan data but lack a single trusted view of portfolio performance. Here's how to build the analytics stack that actually works.
Most payment facilitators are sitting on a goldmine of merchant data and doing almost nothing with it. Here is how to fix that — and what it is worth when you do.
Most loyalty programmes generate plenty of data and almost no usable insight. Here is how to build the analytics stack that actually proves ROI and drives decisions.
BNPL platforms generate complex, high-velocity data that most early-stage teams are not equipped to model. Here is how to build the analytics infrastructure that actually scales.
Most fintech lenders don't have a modelling problem — they have a data infrastructure problem. Here's how to build the analytics stack that credit risk decisions actually depend on.
Fraud analytics for fintech payments is about more than catching bad transactions — it is about building the data infrastructure that stops your fraud model from killing legitimate revenue.
Multi-currency analytics is one of the most underengineered problems in fintech data stacks. This guide shows how to fix it before it costs you real money.
Most SaaS startups can name their LTV:CAC ratio. Far fewer can prove it. This guide covers how to build the data infrastructure behind unit economics that investors actually trust.
Half of finance teams still take six or more days to close the books. Financial close analytics fixes the broken data pipelines causing that delay — here is how to build it.
Your APIs process every payment, every user action, every integration event — but most fintech teams never analyse that data. Here is how to fix that.
Most cash flow forecasts are wrong — not because of bad assumptions, but because of broken data infrastructure. Here is how to fix it.
Most ML models never make it to production. Discover the MLOps best practices that close the gap between data science and real business value in 2026.
Find every Snowflake warehouse running with no credit cap using runnable SQL, then set sensible quotas and stop the gap coming back.
Too many people, and too many dbt jobs, run as ACCOUNTADMIN. Use these Snowflake queries to find them and fix it safely.
Batch or CDC for Postgres to BigQuery? A practitioner's framework covering hard deletes, hidden merge costs and the pipeline failures that surface in production.
Serverless reclustering credits hide outside your warehouse bill. Run these SQL queries to find which dbt tables burn them and how to stop it.
Why finance and ops report different daily totals for the same transactions, and how to fix timezone errors in BigQuery and dbt with one governed reporting date.
Remote spill is the slowest, costliest way a Snowflake query can run. Here is the SQL to find every spilling query and dbt model, and how to fix them.
Is a Redshift to BigQuery migration worth it? A practitioner's guide to the break-even maths, the failure modes, and how to validate every number.
Run two commands and one BigQuery query to list every undocumented dbt model in your project, rank them by real usage, and stop new ones shipping.
Your Snowflake bill may be paying for warehouses that run nothing. Here is the SQL to measure idle credits per warehouse, fix auto-suspend and stop it coming back.
A practitioner's guide to moving business-critical spreadsheet logic into tested dbt models. Learn what to migrate first and how to prove the numbers match.
A BigQuery service account key committed to a dbt repo is a live credential, not a tidy-up task. Find it, rotate it and remove it for good.
When dbt models read from BigQuery views instead of materialised tables, BigQuery silently recomputes the full view logic every time — costing you money you never see coming. Here's how to find it and fix it.
BigQuery external tables look like a cheap shortcut in your dbt pipeline — until they silently inflate your query bill and break your freshness checks. Here is what actually happens in production.
dbt models can silently scan BigQuery tables that never appear in your DAG. Here's how to find them in under five minutes using INFORMATION_SCHEMA.JOBS.
Choosing between dbt macros and persistent SQL UDFs in BigQuery is a decision most teams get wrong — and the cost shows up in your bill and your lineage graph.
Partitioned BigQuery tables without require_partition_filter set are silently full-scanned every run. Here's how to find every offending table and fix it.
JavaScript UDFs in dbt BigQuery silently consume disproportionate slot resources and block query cache hits. Here is how to find them, audit their cost, and fix them before they wreck your bill.
Your BigQuery bill spiked and you have no idea which dbt model caused it. Here is how to fix that with dbt query labels — and the diagnostic SQL to prove it.
Non-deterministic SQL functions silently bust BigQuery's query cache across your entire dbt DAG. Here is how to find every offending model and fix it fast.
BigQuery table expiration silently deletes tables your dbt models depend on — often with no alerting. Here is how to find and fix it before production breaks.
Parsing nested JSON incorrectly in dbt BigQuery staging models is one of the most common — and most expensive — anti-patterns in modern analytics stacks. Here is how to find and fix it.
dbt models querying BigQuery wildcard tables without a _TABLE_SUFFIX filter scan every shard, every run — silently inflating your bill. Here is how to find them and fix them.
Running a single dbt project works — until it doesn't. Learn when to split into multiple projects in BigQuery, how to do it without breaking lineage, and the hidden costs teams miss.
Your dbt incremental model has a WHERE clause, but BigQuery is still scanning the entire table on every run. Here's how to detect it and fix it for good.
dbt source fan-out silently duplicates BigQuery scans, breaks lineage, and strips your pipeline of test coverage. Here is how to find and fix it.
dbt script models using CREATE TEMP TABLE in BigQuery silently break lineage and inflate warehouse costs. Here is how to find and fix them.
The wrong materialisation on dbt intermediate models can silently triple your BigQuery bill. Here's how to detect it and fix it before it compounds.
When a dbt model references the same BigQuery source table more than once, BigQuery may scan it multiple times per run — silently inflating your bill with every pipeline execution.
dbt incremental models using the merge strategy in BigQuery can silently scan your entire target table on every run. Here's how to detect it and fix it fast.
Your BigQuery bill is growing but you can't pinpoint why. INFORMATION_SCHEMA has the answer — here's how to use it to attribute costs by user, model, and team before it's too late.
Using dbt run-operation or BigQuery stored procedures for transformations hides logic from your DAG, breaks lineage, and silently corrupts downstream models. Here is how to find and fix it.
Your BigQuery tables are partitioned — but are your dbt models actually using those partitions? Here is how to find out and fix it before your bill does.
Choosing the wrong materialisation for dbt staging models is one of the most common — and most fixable — sources of runaway BigQuery costs. Here's the framework to get it right.
Repeated subqueries in dbt BigQuery models silently scan the same large table multiple times per run — find them with INFORMATION_SCHEMA and fix them today.
BigQuery offers free query result caching — but most dbt projects silently invalidate it on every run. Here's exactly why it breaks and how to fix it.
Hardcoded table names in dbt BigQuery models silently break your DAG, making downstream failures invisible until something stops working in production.
BigQuery silently re-evaluates CTEs every time they are referenced — a behaviour that can multiply query costs inside complex dbt models without a single warning.
dbt ephemeral models look efficient on paper but silently multiply BigQuery scan costs when referenced by multiple downstream models. Here is how to find and fix it.
Misusing STRUCT and ARRAY types in dbt BigQuery models silently inflates your bill and slows your pipelines. Here is how to find and fix it.
A missing PARTITION BY in a BigQuery window function can silently scan millions of rows it never needed — and you won't see it until the invoice arrives. Here's how to find and fix it.
dbt models that UNNEST repeated fields without a prior filter silently multiply row counts, inflate bytes billed, and corrupt downstream aggregations. Here is how to find and fix them.
Accidental cross joins in dbt BigQuery models can turn a 1M-row table into a trillion-row disaster — and your bill won't wait. Here's how to find and fix them fast.
BigQuery join skew silently inflates your bill and stalls dbt runs. Here is how to find it, fix it, and stop it coming back.
Untested dbt models are silent liabilities. Learn how to find every model with no test coverage in BigQuery using runnable SQL and dbt commands — in under five minutes.
BigQuery materialized views can slash query costs and dashboard load times — but only when paired with dbt correctly. Here is the decision framework practitioners actually use.
Every dbt source without a freshness config is a silent risk. Here's how to find every unchecked source in under five minutes using BigQuery INFORMATION_SCHEMA — and how to fix it.
Choosing the wrong BigQuery pricing model can double your bill. Here's the decision framework — with real numbers — to get it right at your stage of growth.
Over-materialised dbt models are one of the most common causes of runaway BigQuery bills. Here is how to find them and fix them before they compound.
Partitioned tables don't automatically mean cheaper queries. Discover the six silent mistakes that bypass partition pruning in BigQuery — and exactly how to fix them before your next bill arrives.
When you delete or rename a dbt model, BigQuery keeps the table. Here's how to detect orphaned ghost tables and safely drop them before they mislead your team.
Undocumented dbt dependencies are silently breaking production dashboards at growth-stage companies. Here is how to audit your DAG and fix it before it costs you.
A single misconfigured scheduled query can scan hundreds of TB before anyone notices. Here is how to find it — and fix it — in under five minutes.
Model sprawl is the silent killer of growing dbt projects. Learn how to govern your dbt project before undocumented models, broken DAGs, and degraded test coverage cost you.
Most growth-stage companies running BigQuery give everyone too much access. Here is how to implement row-level and column-level security that scales with your team.
Unmasked PII in BigQuery is a compliance and security risk most teams discover too late. Here's how to find it in under five minutes using SQL against INFORMATION_SCHEMA.
Silent NULLs in dbt and BigQuery pass every test, build successfully, and corrupt your metrics for weeks before anyone notices. Here is how to find them.
Fan-out joins silently inflate dbt metrics in BigQuery — revenue looks higher, churn looks lower, and nobody notices until a stakeholder asks why the numbers don't add up.
Full table rebuilds are silently draining your BigQuery budget. Here is the decision framework for getting dbt incremental models right — the first time.
dbt's default on_schema_change='ignore' silently drops new source columns from incremental models. Here's how to detect and fix it before your dashboards lie to you.
Webhook events are failing silently — and your dashboards are lying to you because of it. Learn how to build reliable webhook ingestion analytics that protect revenue and power trusted metrics.
Most SaaS and platform businesses have no structured analytics layer between their API logs and their billing systems — and that gap is silently costing them revenue. Here is how to fix it.
Most fintechs have a ledger. Very few have analytics built on top of it. This guide shows exactly what breaks, and how to fix it.
Most fintech startups have no real-time visibility into their FX exposure — only spreadsheets and gut feel. Here's how to fix that before it costs you.
Most fintech dispute teams are drowning in volume with no analytics stack to manage it. Here is how to build one that actually works.
Most fintechs run multi-gateway stacks but have no analytics layer to tell them whether their routing logic is working. Here's how to fix that.
Most fintech ops teams fly blind on the metrics that matter most — settlement timing, PSP cost efficiency, and reconciliation breaks. Here is how to fix that with a proper payment operations analytics layer.
Poor authorization rates silently drain revenue from payment businesses. This guide shows exactly how to build the analytics stack to diagnose, monitor, and fix them.
Most fintechs track burn rate in a spreadsheet — until it breaks. Here's how to build a real analytics stack that gives finance and leadership actual visibility.
Most fintech and payments businesses are losing basis points on every transaction without knowing it. Here is how to build interchange fee analytics that actually show you where the money is going.
Most fintechs have a KYC vendor. Very few have a compliance analytics layer behind it. Here is how to build one that actually works in 2026.
SaaS and fintech companies silently lose 3–5% of ARR to revenue leakage every year. Here's how to build the analytics infrastructure that finds and fixes it.
Embedding financial products into your platform is one thing. Measuring whether they're working — and why — is another problem entirely. Here's how to build the analytics stack your embedded finance operation actually needs.
Most fintech card issuers are flying blind. They know their card is live — but they don't know why authorisation rates dropped, which spend categories drive interchange, or which cardholders are drifting to a competitor.
Most marketplace founders are measuring the wrong things. Here is how to build analytics that actually reveal whether your platform is healthy — or quietly failing.
Most cross-border fintechs are flying blind on their most important metrics. Here's how to build the analytics layer that gives you real corridor visibility.
Training-serving skew is the silent killer of production ML. Learn how to build feature pipelines that actually hold up — and stop your models degrading the moment they ship.
Most scaling e-commerce brands track ROAS religiously but can't explain why profit shrinks as revenue grows. Here's how to build the analytics stack that fixes that.
Aggregate retention metrics can mask serious churn problems at fast-growing companies. Here's how to build cohort analytics that surface the real story — and what to do when you don't like what you find.
Most B2B SaaS companies reach Series A with their core growth metrics — ARR, NRR, churn — still living in spreadsheets. Here is how to fix that before it costs you.
If finance and ops see different revenue figures, you don't have a dashboard problem — you have a semantic layer problem. Here's how to fix it.
Slow dbt models are costing you more than just time — they inflate cloud bills and delay decisions. Here's how to diagnose and fix the real causes.
Build or buy embedded analytics? This practitioner guide covers the real trade-offs, architecture decisions, and tooling choices for SaaS product teams in 2026.
Graph analytics moves beyond rows and columns to reveal the hidden relationships driving your business. Here's how leading organisations are using it in 2026.
Time series analysis is one of the most powerful — and underused — tools in business forecasting. Here's how forward-thinking organisations are using it in 2026.