Jaco van der Laan

From Data Dictionary to Model-Driven Data Engineering

26/09/2025  ·  Data Architecture  ·  Data Engineering  ·  Data Modeling

‹ ›

Why documenting your Power BI datasets is just the first step — and how a model-first, metadata-driven approach unlocks true scalability, governance, and automation.

🌟 Summary

Microsoft’s step-by-step guide to creating a data dictionary in Power BI shows an important truth: metadata matters.

But most teams stop there — producing a static catalog or dictionary that helps with understanding, but doesn’t drive the platform itself.

In Model-Driven Data Engineering (MDDE), metadata isn’t just documentation; it’s the engine.
Your models define how data flows, how it’s historized, and even how cost and governance are enforced by design.

This article explores how a simple data dictionary can evolve into a model-first, metadata-driven architecture that powers automation, cost control, and governed delivery layers.

Data Dictionaries: A Good Start

The Power BI article walks through:

  • Extracting table and column information.
  • Adding descriptions, data types, and business context.
  • Publishing a readable, searchable dictionary for analysts.

This step is valuable:

  • It creates a shared vocabulary.
  • It reduces onboarding friction.
  • It improves trust in reports.

But if the dictionary lives only in Excel or a wiki, it’s static. When schema changes, the docs drift. When governance rules evolve, they’re not enforced.

Model-First, Metadata-Driven Data Engineering

Over the past years I’ve written extensively about moving from hand-crafted ETL to model-driven automation:

Core principle:

The model is not documentation — it is the single source of truth that drives the platform.

Key capabilities:

  1. Metadata as Code
    Store table definitions, lineage, and mappings in a structured repository (YAML, JSON, or a metadata database).
  2. Automatic Pipeline Generation
    Frameworks turn those models into SQL/ELT pipelines — SCD2 historization, PIT tables, business-friendly mappings.
  3. Governance by Design
    Data contracts, column-level lineage, and access policies are generated and enforced automatically.
  4. Cost Awareness at Design Time
    (see Designing for Cloud Cost Control with Model-Driven Data Engineering): predict storage & compute cost for every new dataset before deployment.
  5. Continuous Documentation
    Your metadata drives live dictionaries in Confluence, OpenMetadata, or Power BI — always in sync with what’s actually running.

How Power BI Fits In

  • Power BI is the last mile.
    Even in a model-driven platform, analysts still need clear, trusted datasets.
  • Generate, don’t manually curate.
    Instead of building the dictionary by hand, publish it from your metadata store to Power BI (via the REST API or tools like OpenMetadata).
  • Close the loop.
    Analysts’ feedback (renamed fields, better descriptions) can flow back into the model repository, keeping everything aligned.

Practical Integration Pattern

  1. Model your semantic layer and transformations (SCD2, business mappings) in a metadata repository.
  2. Generate data pipelines (dbt, Delta Live Tables, Lakeflow) automatically from the model.
  3. Publish dictionary content into Power BI’s data model documentation or a Confluence portal.
  4. Synchronize — if the model changes, both pipelines and documentation stay up to date.

This is the same mindset that powers Rethinking EA Mapping with DuckDB & Python and Metadata-Driven Data Architecture: Leveraging draw.io and VSCode for Scalable Design.

Why It Matters

  • Consistency: one model feeds both engineering and analytics.
  • Speed: no manual re-work when schemas evolve.
  • Governance: lineage, contracts, and access policies stay accurate.
  • Cost control: designs are evaluated before they hit the cloud bill.
  • Empowered analysts: Power BI remains the business-friendly face, but backed by a governed, automated core.

Related Reading

Key Takeaways

  • A Power BI data dictionary is a valuable first step but shouldn’t be static.
  • Model-Driven Data Engineering turns metadata into an engine — driving pipelines, governance, cost control, and live documentation.
  • Power BI fits beautifully as the consumer layer when fed by a metadata-driven backbone.
  • Move from “document after” to design first, generate everything — including your dictionaries.