Don’t Deploy the Whole Platform. Deploy Exactly What You Need.
Don’t Deploy the Whole Platform. Deploy Exactly What You Need.
Most metadata platforms suffer from feature bloat. You adopt one capability and suddenly inherit 200 database tables. The MDDE Adoption Framework maps modules to metadata — so you deploy only what your use case requires.
The Metadata Bloat Problem
You want to implement data lineage. Just lineage.
You install a metadata platform and discover it created:
- 47 tables for “core metadata”
- 23 tables for “governance”
- 15 tables for “operations”
- 12 tables for “collaboration”
- …
97 tables for lineage?You needed maybe 5.
This is metadata bloat. Platforms ship everything because they can’t articulate which metadata supports which capability. Every table is “required” because no one mapped the dependencies.
The Module-Metadata Matrix
The MDDE Adoption Framework introduces a clear mapping:every module declares its metadata dependencies.
┌─────────────────────────────────────────────────────────────────────────────┐
│ MODULE-METADATA MATRIX │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ MODULE REQUIRED ENTITIES CHILD MODELS │
│ ───────────────────────────────────────────────────────────────────────── │
│ │
│ yaml_models model, entity, attribute core │
│ sql_generation model, entity, attribute core │
│ │
│ entity_lineage entity, entity_mapping core, lineage │
│ column_lineage attribute, attr_mapping, core, lineage │
│ entity_mapping │
│ │
│ quality_checks entity, attribute, core, governance │
│ simple_check │
│ │
│ data_vault model, entity, attribute, core, reference │
│ stereotype │
│ │
│ unity_catalog model, entity, attribute, core, integration │
│ layer │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
If you only need lineage, you deploycore+lineagechild models — perhaps 15 entities instead of 380+.
If you only need documentation, you deploycore— maybe 5 entities.
Use Case to Metadata: The Minimal Subset
Each use case maps to a specific set of modules, which maps to a specific set of metadata entities.
Documentation Use Case
You want to document existing data assets. Nothing more.
from mdde.planning.adoption_framework import get_minimal_entity_set, UseCase
result = get_minimal_entity_set([UseCase.DOCUMENTATION])
Result:
use_cases: [documentation]
modules_needed: [yaml_models, discovery, ai_documentation, erd_diagrams]
child_models_needed: [core]
total_entities: 8
entities_by_child_model:
core:
- model
- entity
- attribute
- relationship_def
organization:
- diagram_def
- style_def
collaboration:
- collaborative_comment
- collaborative_glossary_link
8 entities.Not 380. Eight.
Lineage Use Case
You need to track data flows for regulatory compliance.
result = get_minimal_entity_set([UseCase.LINEAGE])
Result:
use_cases: [lineage]
modules_needed: [entity_lineage, column_lineage, impact_analysis, lineage_diagrams]
child_models_needed: [core, lineage]
total_entities: 18
entities_by_child_model:
core:
- model
- entity
- attribute
- relationship_def
lineage:
- entity_mapping
- attribute_mapping
- entity_lineage
- function_param_mapping
- entity_mapping_key_mapping
- relationship_key_mapping
organization:
- diagram_def
- style_def
18 entities.Full column-level lineage with impact analysis.
Quality Use Case
You want to implement data quality checks.
result = get_minimal_entity_set([UseCase.QUALITY])
Result:
use_cases: [quality]
modules_needed: [quality_checks, quality_gates]
child_models_needed: [core, governance]
total_entities: 14
entities_by_child_model:
core:
- model
- entity
- attribute
governance:
- simple_check
- quality_gate
- quality_indicator
- observability_config
- entity_freshness_observation
- observation
Regulatory Banking Use Case
BCBS 239 compliance requires lineage, quality, governance, AND bi-temporal modeling.
result = get_minimal_entity_set([
UseCase.REGULATORY,
UseCase.LINEAGE,
UseCase.QUALITY,
UseCase.GOVERNANCE
])
Result:
use_cases: [regulatory, lineage, quality, governance]
modules_needed:
- yaml_models
- entity_lineage
- column_lineage
- quality_checks
- quality_gates
- dual_scd2
- kde_tracking
- data_requirements
child_models_needed: [core, lineage, governance, reference]
total_entities: 42
entities_by_child_model:
core:
- model
- entity
- attribute
- layer
- relationship_def
lineage:
- entity_mapping
- attribute_mapping
- entity_lineage
- entity_timeline_consolidation
- entity_timeline_source
# ...8 more
governance:
- key_data_element
- kde_attribute_link
- requirement
- stakeholder
- simple_check
- quality_gate
# ...4 more
reference:
- stereotype_def
- time_profile_def
42 entities for full BCBS 239 compliance.Still far less than the full 380+ entity model.
The Adoption Advisor
TheAdoptionAdvisorclass generates personalized recommendations based on your profile:
from mdde.planning.adoption_framework import (
AdoptionAdvisor,
AdoptionProfile,
LifecycleStage,
ResourceLevel,
UseCase,
)
# Define your context
profile = AdoptionProfile(
name="Regional Bank Risk Platform",
industry="banking",
lifecycle_stage=LifecycleStage.DEVELOPMENT,
resource_level=ResourceLevel.MODERATE, # 3-5 people
use_cases=[UseCase.MIGRATION, UseCase.LINEAGE, UseCase.QUALITY],
regulations=["BCBS_239", "GDPR"],
)
# Get recommendations
advisor = AdoptionAdvisor()
subset = advisor.recommend(profile)
print(f"Modules recommended: {len(subset.modules)}")
print(f"Setup days: {subset.total_setup_days}")
print(f"Learning hours: {subset.total_learning_hours}")
print(f"Metadata entities needed: {len(subset.required_entities)}")
Output:
Modules recommended: 12
Setup days: 18.5
Learning hours: 52
Metadata entities needed: 28
The advisor:
- Finds matching modules— based on your use cases and regulations
- Resolves dependencies— if you need
column_lineage, you automatically getentity_lineage - Filters by constraints— if you’re at “minimal” resource level, enterprise-grade features are excluded
- Prioritizes— “must have” vs “recommended” vs “optional”
- Calculates metadata— exactly which entities you need to deploy
Quick Start Profiles
For common scenarios, use pre-configured profiles:
from mdde.planning.adoption_framework import get_quick_start_profile
# Documentation-only adoption
profile = get_quick_start_profile("documentation")
# → LifecycleStage.DISCOVERY, ResourceLevel.MINIMAL, [UseCase.DOCUMENTATION]
# Platform migration
profile = get_quick_start_profile("migration")
# → LifecycleStage.DEVELOPMENT, ResourceLevel.MODERATE,
# [UseCase.MIGRATION, UseCase.DOCUMENTATION, UseCase.LINEAGE]
# Data quality implementation
profile = get_quick_start_profile("quality")
# → LifecycleStage.DEVELOPMENT, ResourceLevel.MODERATE,
# [UseCase.QUALITY, UseCase.GOVERNANCE]
# Banking regulatory compliance
profile = get_quick_start_profile("regulatory_banking")
# → Industry: banking, Regulations: [BCBS_239, GDPR],
# [UseCase.REGULATORY, UseCase.LINEAGE, UseCase.QUALITY, UseCase.GOVERNANCE]
# Data mesh foundation
profile = get_quick_start_profile("data_mesh")
# → LifecycleStage.OPERATIONS, ResourceLevel.ENTERPRISE,
# [UseCase.DATA_MESH, UseCase.GOVERNANCE, UseCase.SELF_SERVICE]
The Module Catalog
The framework includes 22 modules across 8 categories:
Core Modules
Discovery & Documentation
Lineage
Quality
Temporal
Governance
Integration
Methodology
Entity Annotations
Every metadata entity can be annotated with its module relevance:
from mdde.planning.adoption_framework import get_entity_annotation
annotation = get_entity_annotation("entity")
print(annotation)
Output:
EntityAnnotation(
entity_name='entity',
child_model='core',
modules=['yaml_models', 'sql_generation', 'model_validation',
'discovery', 'ai_documentation', 'entity_lineage',
'column_lineage', 'impact_analysis', 'quality_checks', ...],
use_cases=['documentation', 'modeling', 'migration', 'governance',
'quality', 'lineage', 'devops', 'ai_enablement'],
is_core=True # Used by 5+ modules
)
This enables:
- Schema annotations— document which modules use which entities
- Documentation— auto-generate module-entity relationship docs
- Governance— understand impact of schema changes
Generating Adoption Roadmaps
The advisor can generate a complete phased roadmap:
roadmap = advisor.generate_adoption_roadmap(profile)
Output:
profile:
name: "Regional Bank Risk Platform"
lifecycle: "development"
use_cases: ["migration", "lineage", "quality"]
summary:
total_modules: 12
total_setup_days: 18.5
total_learning_hours: 52
required_metadata_entities: 28
required_child_models: ["core", "lineage", "governance"]
phases:
- phase: 1
name: "Foundation"
duration_days: 5.5
modules:
- name: yaml_models
display_name: "YAML Modeling"
reason: "Core for modeling, documentation"
- name: quality_checks
display_name: "Quality Checks"
reason: "Core for quality, governance"
- name: entity_lineage
display_name: "Entity Lineage"
reason: "Core for lineage, governance"
- phase: 2
name: "Expansion"
duration_days: 13
modules:
- name: column_lineage
display_name: "Column Lineage"
reason: "Supports lineage"
- name: impact_analysis
display_name: "Impact Analysis"
reason: "Supports lineage"
# ...
metadata_requirements:
entities:
- model
- entity
- attribute
- entity_mapping
- attribute_mapping
- simple_check
# ...
child_models:
- core
- lineage
- governance
Minimal Deployment Example
Here’s how to deploy only what you need:
Step 1: Generate Your Minimal Subset
# For documentation + quality
result = get_minimal_entity_set([UseCase.DOCUMENTATION, UseCase.QUALITY])
print(f"Deploy these child models: {result['child_models_needed']}")
print(f"Total entities: {result['total_entities']}")
Step 2: Export Required Schema
# Generate DDL for only the required child models
mdde schema export \
--child-models core governance \
--output minimal_schema.sql
Step 3: Validate Your Deployment
# Verify all required entities exist
mdde validate deployment \
--required-entities model,entity,attribute,simple_check
IP Sharing: Only Share What’s Relevant
The Adoption Framework also enablesselective IP sharing.
When working with customers, you don’t need to share the entire MDDE codebase. You can identify exactly which modules are relevant to their use case:
# Customer needs migration + lineage
modules_for_customer = advisor.get_use_case_modules(UseCase.MIGRATION)
modules_for_customer += advisor.get_use_case_modules(UseCase.LINEAGE)
# Generate module list for sharing
for module in modules_for_customer:
print(f"- {module.display_name}: {module.description}")
This clarity enables:
- Scoped licensing— license only the modules customers use
- Focused training— train only on relevant capabilities
- Lighter deployments— fewer entities = faster setup
The Bidirectional Map
The framework maintains a bidirectional mapping:
Entity → Modules
from mdde.planning.adoption_framework import get_entity_modules
# Which modules use the 'entity_mapping' entity?
modules = get_entity_modules("entity_mapping")
# → ['entity_lineage', 'column_lineage', 'impact_analysis', 'lineage_diagrams']
Module → Entities
from mdde.planning.adoption_framework import get_module_entities
# Which entities does 'column_lineage' require?
entities = get_module_entities("column_lineage")
# → ['attribute', 'attribute_mapping', 'entity_mapping',
# 'function_param_mapping', 'entity_window_param_mapping']
This bidirectionality enablesimpact analysis— if you modify an entity, you know which modules are affected.
Integration with Use Case Profiles
The Adoption Framework integrates with theUse Case Profilesarticle:
# use-case-profile.yaml
organization:
industry: banking
regulations: [BCBS_239, GDPR]
ambition:
level: transformational
mdde_preferences:
features_of_interest:
must_have:
- column_lineage
- dual_scd2
- quality_checks
The profile feeds into the Adoption Framework:
# Parse profile and generate recommendations
profile = parse_use_case_profile("use-case-profile.yaml")
subset = advisor.recommend(profile)
# Output includes exactly what's needed for BCBS 239 banking
assert "dual_scd2" in [m.module.name for m in subset.modules]
assert "column_lineage" in [m.module.name for m in subset.modules]
Why This Matters
For Platform Implementers
- Deploy faster— no unnecessary tables
- Reduce complexity— fewer moving parts
- Lower costs— less storage, compute, maintenance
For Consultants
- Scope clearly— know exactly what a project requires
- Estimate accurately— setup days and learning hours are documented
- Avoid over-engineering— don’t deploy features that won’t be used
For Customers
- Start small— adopt incrementally
- Understand dependencies— know what’s required before you start
- Grow when needed— add modules as use cases expand
Summary
The MDDE Adoption Framework answers the question every platform adopter asks:
“What do I actually need to deploy?”
Instead of shipping 380+ entities “just in case”, the framework:
- Maps modules to metadata— every module declares its dependencies
- Maps use cases to modules— documentation needs different modules than regulatory compliance
- Calculates minimal subsets— exactly which entities for your use case
- Generates roadmaps— phased implementation with effort estimates
- Enables selective sharing— share only what’s relevant to each customer
Stop deploying everything. Deploy what you need.