A curated collection of real-world Architecture Decision Records (ADRs) highlighting pragmatic analytics engineering trade-offs, warehouse cost-benefit evaluations, and query performance optimizations.
In analytics engineering, the hardest problems are rarely about writing SQL syntax—they are about evaluating structural trade-offs:
- Should we adopt a new managed cloud warehouse or optimize the existing engine in place?
- When should we use database-native Materialized Views vs. scheduled staging table rebuilds?
- How do we eliminate redundant 1,000+ line CTE evaluation across analytical dashboards?
This repository documents the rationale, architectural decision paths, and operational consequences of solving dashboard latency bottlenecks on a columnar analytical engine (ClickHouse) serving Looker Studio dashboards.
| ADR | Title | Decision | Impact |
|---|---|---|---|
| ADR-0001 | Optimize Columnar Engine in Place vs. BigQuery Migration | Rejected migration to BigQuery; optimized existing ClickHouse cluster in place. | Avoided double cloud spend, eliminated redundant infrastructure overhead, and maintained sub-second SLA by fixing SQL bottlenecks. |
| ADR-0002 | Scheduled Staging Tables vs. Engine Materialized Views | Chose scheduled truncate-and-reload staging models over native ClickHouse Materialized Views. | Prevented silent data drift caused by in-place upstream CRM edits that bypass trigger-on-insert Materialized Views. |
Before migrating warehouse engines to solve dashboard latency, inspect the execution plan of the physical queries. In this case study, latency was driven by un-materialized, redundantly-evaluated live CTEs calculating multi-source attribution across 30+ charts—not engine limitations.
Columnar database Materialized Views often trigger exclusively on INSERT blocks. When raw data sources (such as CRM records or Bitable tables) support in-place mutations or column schema adjustments, Materialized Views become liability hazards. Full or partition-based scheduled staging reloads provide robust idempotency.
MIT © Ihsan Wanda