Engineering Time in Data Platforms — A 3-Part Series From Snapshots to Deterministic Timelines: Engineering Time in Data Platforms Most data platforms believe they are historized. They are not. They store history — but they do not engineer time. In this three-part MDDE deep dive, we will move from: Basic SCD2 storage to bi-temporal standardization (this article) Timeline compression and deterministic integration (Part 2) Valuation reconstruction with impact-driven refresh (Part 3) This first article lays...

Series Context Merging Multiple SCD2 Timelines Without Explosion: ASOF Sampling, Gaps-and-Islands, and PIT Tables In Part 1, we standardized heterogeneous sources into controlled bi-temporal SCD2. Now comes the hard part. Once everything is SCD2… Joining them becomes dangerous. Each SCD2 table has its own change points. Consider a simple 360-degree customer view requiring three SCD2 sources: Combined timeline: 9 change points from just 3 sources with 3 changes each. Each change fragments the timeline. Add...

How metadata-as-code principles extend beyond model generation to manage safe, automated, and traceable schema evolution in production How metadata-as-code principles extend beyond model generation to manage safe, automated, and traceable schema evolution in production In my previous article, we explored how conceptual and logical metadata can automatically generate physical data models — turning business meaning into structure and SQL. After publication, Kristian Poulsen asked a question that perfectly captures the next step in this journey:...

Structure versioning deals with the shape of the data: tables, columns, relationships, and schemas. Content versioning deals with the meaning of the data: business rules, filters, derivations, and how values are interpreted. Both are equally important: Structure changes affect how systems read/write the data. Content changes affect how the same data values should be understood. The platform’s physical core (historization + integration layers) is a superset of all structures ever modeled. Columns and tables are...