From metadata to automation — designing intelligent, governed, and explainable data platforms
Why Model-Driven Thinking Changes Everything
Data engineering is evolving — from hand-coded pipelines to metadata-driven platforms.
The future isn’t about writing more SQL; it’s about building systems that can generate, validate, and explain themselves.
Model-Driven Data Engineering explores how to make that transition.
It’s about replacing manual data manipulation with architecture-as-code, metadata-as-design, and models-as-truth.
In this publication, we turn the principles of modeling and metadata management into a practical engineering discipline — one that blends automation with human understanding.
What You’ll Find Here
Each article explores how models, metadata, and automation reinforce one another — turning data platforms into self-documenting, governed ecosystems.
Topics include:
- Metadata-driven data architecture and governance
- Business-friendly mapping and integration
- Historization and temporal modeling (Dual SCD2, PIT, and bitemporal structures)
- Migration and SQL optimization patterns
- Delivery and consumption layer design
- Visualization, lineage, and metadata tooling
- The evolution of modeling tools — from PowerDesigner to VS Code, CrossBreeze, and OpenMetadata
It’s a hands-on look at how to engineer data with design principles — not just pipelines.
1. From Coding to Modeling
The shift from writing ETL scripts to defining reusable metadata.
- From Coding to Modeling: The Rise of Model-Driven Data Engineering
- From Whiteboards to Working Models: How We Built a Model-Driven Data Warehouse at de Volksbank with CrossBreeze
- CrossBreeze and the Future of Model-Driven Data Engineering
- How Model-Driven Data Engineering, Business-Friendly Mapping, and Business-Oriented Data Modeling Reinforce Each Other
These stories show how metadata and models become the new source code — accelerating delivery while preserving meaning.
2. Influences & Collaboration
Model-driven design stands on the shoulders of giants.
This section honors and connects the people who shaped the practice:
- Hans Hultgren and Remco Broekmans: Ensemble Modeling Evolved
- Rick van der Lans and the SuperNova Mindset
- Roelant Vos and Data Engine Thinking
- Simon Späti’s Architectural Influence
- Patrick Cuba’s Data Vault Guru Perspective
- Honoring George McGeachie: PowerDesigner, Metadata, and Model-Driven Data Engineering
Together, they form the living lineage of model-driven thinking.
3. Metadata-Driven Architecture & Governance
Metadata is more than documentation — it’s the foundation of governance and automation.
- Metadata as Data — Controlling and Governing Data Models
- Publish Your Metadata Story to Confluence
- OpenMetadata in a Metadata-Driven Data Architecture
- Metadata-Driven Data Architecture: Leveraging draw.io and VSCode for Scalable Design
- Data Contracts: Anchoring Reliability in Model-Driven Data Engineering
These articles show how metadata can manage quality, lineage, and compliance — all while driving code generation.
4. Business-Oriented & Universal Modeling
Keeping architecture aligned with business meaning through shared structures and universal patterns.
- Unlock Your Business’s True Potential: The Power of Business-Oriented Data Modeling
- The Case for a Business-Shaped Integration Layer
- Bill Inmon’s Lasting Impact — Why 3NF and Universal Data Models Still Matter
- From Kimball to Metadata: How Dimensional Thinking Still Shapes Modern Data Architecture
Here, the focus shifts from pipelines to meaningful design — where every model tells a business story.
5. Source Models & Business-Friendly Mapping
Bridging raw data and business understanding through metadata-driven mapping.
- Reverse Engineering Sources
- Source Access Views — Bridging Physical and Logical Worlds
- Cooking with Data: How Business-Friendly Mapping Brings Clarity to ETL
- Business-Friendly Mapping Meets dbt
These articles show how mappings become living transformations — stored, versioned, and explainable through metadata.
6. Historization & Temporal Modeling
Capturing change over time is where design meets discipline.
- Dual SCD2 and PIT Tables: A Technical Deep-Dive
- How Dirk Lerner’s Work on Temporal Data Strengthens Dual SCD2 Architectures
- Why Snapshots Alone Don’t Give You True History
- Adding Surrogate Keys to Dual SCD2 Tables to Enable PIT Tables
Historization transforms raw data into auditable knowledge — the time dimension of trust.
7. Delivery & Consumption Layers
Designing exposure views and delivery schemas that make historized data consumable.
The delivery layer turns engineering into experience — where governed data becomes insight.
8. Migration & SQL Best Practices
Migrating from legacy SQL and monolithic ETL to modular, metadata-driven pipelines.
- How to Migrate Legacy SQL ETL to a Dual-SCD2 Platform
- From Monolith to Modules: Optimizing Legacy SQL
- SQL Best Practice: Modular Query Design with CTEs
- Why You Should Never Use Inline UNION ALL in SQL
Here, code becomes cleaner, safer, and aligned with the metadata that describes it.
9. Modeling Tools & Visualization
Tools evolve, but the principle stays the same: models and metadata must stay synchronized and visible.
- Automating draw.io Diagram Generation from Metadata
- Parsing Diagrams Back into Metadata
- CrossModel Meets Model-Driven Data Engineering
- Beyond PowerDesigner: Modern Alternatives for Metadata-Driven Engineering
The visual layer closes the loop between design, metadata, and delivery.
10. Events & Community
Conversations that shape the field — insights from summits and peers who share the same mission.
This publication celebrates the shared journey of practitioners building the next generation of data architecture.
The Unifying Theme
Model-Driven Data Engineering is not just about automation — it’s about understanding.
When models, metadata, and meaning align, data platforms become predictable, explainable, and trusted by design.
That’s the heart of model-driven thinking.
Join the Movement
Follow this publication to explore how metadata, modeling, and automation redefine modern data engineering.
If you believe that platforms should be as elegant as the models that describe them — welcome home.