Lessons from Data Vault: Principles Without the Dogma
“Data Vault isn’t broken — but the way most people implement it is.”
Don’t implement Data Vault literally with Hubs, Links, and Satellites.
The real value lies in its principles: separation of concerns, business key orientation, historization, and agility.
Apply those ideas flexibly in modern architectures (data lakes, streaming, data mesh).
Borrow the principles, skip the dogma.
![[Pasted image 20251007202417.png]]
Most teams hear “Data Vault” and think of hubs, links, and satellites. In practice, implementing those physical structures literally often leads to complexity and frustration.
The real value of Data Vault lies in its principles — ideas that remain highly relevant in modern architectures such as data lakes, streaming platforms, and data meshes. By focusing on principles instead of rigid table patterns, you can build a platform that is robust, auditable, and agile.
Here are the key concepts worth keeping — and how to apply them without falling into the HL&S trap.
1. Separation of Concerns
- Idea: Break complex business concepts into business keys, relationships, and descriptive context.
- Practice: Keep these concerns logically separate in your platform, even if not as separate tables. This modularity makes change easier and clarifies purpose.
2. Business Key Orientation
- Idea: Natural business keys (Customer ID, Product ID, Order ID) are stable anchors for integration.
- Practice: Always tie your data model back to these business-driven identifiers. Technical keys (UUIDs, surrogates) are fine for storage, but enterprise-wide integration depends on the natural keys.
3. Historization and Auditability
- Idea: Capture the full history, not just the latest snapshot.
- Practice: Store immutable versions, add as-of timestamps, and preserve record sources. Techniques like versioned datasets or event sourcing ensure lineage and accountability.
4. Agility and Adaptability
- Idea: A data platform should easily absorb new sources and respond to change.
- Practice: Use schema-on-read for raw layers, modular pipelines, and flexible ingestion. Avoid designs that require heavy re-engineering when adding attributes or sources.
5. Enterprise Integration and Central View
- Idea: The ultimate goal is a unified, enterprise-wide view of data.
- Practice: Define a conceptual Enterprise Information Model and reinforce it with governance, glossaries, and consistent definitions — regardless of how you physically store the data.
6. Layered Architecture
- Idea: Clear layering prevents chaos.
- Practice:
- Staging/Landing: raw, temporary mirrors of sources
- Raw Layer: persisted, auditable source-centric data
- Integrated Layer: reconciled and aligned around business keys
- Consumption Layer: curated marts or views, disposable and easily recreated
7. Handling Diverse Data
- Idea: Data comes in many shapes — structured, semi-structured, unstructured.
- Practice: Land diverse data flexibly in a lake or raw zone, apply schema later, and isolate context so changes don’t ripple across the platform.
8. Master Data Alignment
- Idea: Data Vault concepts align well with Master Data Management.
- Practice: Reconcile variations in master data, track multiple interpretations, and flag discrepancies — instead of enforcing rigid constraints that reject dirty but useful data.
9. Model-Driven vs. Data-Driven
- Idea: Sometimes schema-on-write is right, sometimes schema-on-read.
- Practice: Use data-driven approaches in raw layers (fast and flexible) and model-driven approaches in curated layers (usable and performant).
10. Reference Data Handling
- Idea: Reference data is contextual, not a gatekeeper.
- Practice: Store reference codes separately, but don’t enforce strict foreign keys in the raw/integrated layer. Let exceptions surface for reporting instead of silently dropping them.
Final Thoughts
Data Vault isn’t about copying hubs, links, and satellites into your warehouse. It’s about thinking differently: separating concerns, honoring business keys, preserving history, and designing for agility.
If you adopt those principles — while resisting the temptation to turn them into dozens of rigid tables — you’ll capture the true strengths of Data Vault without drowning in unnecessary complexity.
The lesson is simple: borrow the ideas, skip the dogma, and build a platform that works for your business today and tomorrow.