How to maintain traceability and automation between business concepts and implementation models using YAML A series exploring how modern data platforms connect business meaning and technical implementation using metadata-as-code. 1️⃣ From Business Concepts to Database Blueprint — bridging conceptual and logical models inspired by Alec Sharp. 2️⃣ Linking Conceptual and Logical Data Models in Metadata — maintaining traceability and automation between business concepts and implementation models using YAML. 3️⃣ Generating Logical Models Automatically from Conceptual Metadata — (coming soon): implementing Alec Sharp’s...

How to keep conceptual, logical, and physical data models synchronized and collaborative in modern metadata-as-code environments 🧩 Metadata-Driven Modeling Series A continuing series exploring how modern data platforms connect business meaning and technical implementation using metadata-as-code. 1️⃣ From Business Concepts to Database Blueprint — bridging conceptual and logical models inspired by Alec Sharp. 2️⃣ Linking Conceptual and Logical Data Models in Metadata — maintaining traceability and automation between business concepts and implementation models using YAML. 3️⃣ Scaling Metadata-Driven Data Modeling — this...

Reproducible Bi-Temporal SCD2 Using _hash(business_columns)_ + _row_number()_ per Hash 🧩 Part of the Model-Driven Data Quality series — exploring reproducible, metadata-driven patterns for historization, deduplication, and bi-temporal data management. In a metadata-driven data platform, historization is the foundation of reproducibility and auditability. Our bi-temporal SCD2 design records both valid-time (business reality) and system-time (warehouse load time), allowing us to reconstruct any dataset as it was known at any point in time. Most implementations assume that the source defines a primary key — but in reality, many source systems don’t. They simply deliver...

Follow-up to “From ERDs and Lineage to Executable Metadata” When I published From ERDs and Lineage to Executable Metadata, a reader asked: “Do you advocate generating config and code from diagrams_, or generating diagrams_ from metadata_? And if it’s the latter, what does that metadata look like — and why not use standards like UML, BPMN, or XMI to capture meaning graphically?”_ A great question — and one that gets to the heart of the shift from descriptive diagrams to executable...

A modern metadata-driven development workflow where data modelers define logic and semantics declaratively, while data engineers operationalize these definitions in SQL and Jinja macros. Powered by VS Code, JSON validation, and language-server intelligence, this approach unites modeling and engineering into a single semantic DevOps loop. Summary Treat models as code. Data modelers express business meaning, structure, and mappings in YAML; data engineers build reusable macros and SQL templates that translate these definitions into executable code. Both collaborate...

Traditional data diagrams show what data looks like and how it moves. Model-Driven Data Engineering unites both — turning metadata into an executable design that powers automation, governance, and understanding. Entity Relationship Diagrams (ERDs) visualize structure. Data Lineage visualizes movement. Model-Driven Data Engineering combines both — using metadata as code to define, generate, and validate everything from models to mappings, SQL, and documentation. Instead of drawing what already exists, we now design once and generate many. Press enter or click...

A Metadata-Driven Perspective on Bridging Business Semantics and Data Engineering When Bill Inmon introduced the Business Language Model (BLM), he envisioned a bridge between business language and data — a model that could translate unstructured text into structured insight. Two decades later, model-driven data engineering is realizing that same vision: by treating business meaning as metadata, not just documentation. Press enter or click to view image in full size Since the dawn of data warehousing, one fundamental...

How we’re extending AI assistance from SQL migration to metadata-driven modeling, documentation, and collaboration. 💡 Summary Our rule engine and metadata framework taught us something important: AI can do more than refactor SQL — it can help build data models, mappings, and documentation if we give it the right context. The next step in our journey is to bring this capability directly into the workspace where business developers and data modelers collaborate every day: VS Code with Copilot. By...

Why VS Code, Python, AI, and Git-based metadata workflows might replace PowerDesigner, ERwin, and ER/Studio. 💡 Summary For decades, enterprise data modeling relied on heavyweight tools like PowerDesigner, ERwin, and ER/Studio — specialized environments with rich diagramming and repository features. But as metadata, automation, and AI evolve, we’re seeing a quiet shift: modeling is becoming code. In this article, I explore whether we still need these classic tools — and how a modern, open workflow using VS Code,...

Track A (Business) — Part 3 of the Entity‑First series Series navigation: A1 — Why We Need a Business‑Friendly Data Transformation Model • A2 — Entities, Subject Areas & Transformation Steps • B1 — How dbt & SQLMesh Organize Metadata and Code • B2 — Designing an Entity‑First YAML → SQL Generator • A3 (you are here) This playbook turns the Entity‑First approach into day‑to‑day practice. It covers naming conventions, ownership/RACI, change workflow, and quality gates so that modelers and engineers ship reliable data products with the same rules,...