Compliance Scoring & CI Gates — Turning Architectural Intent into Enforceable Engineering Signals

The Problem

---

Compliance Scoring & CI Gates — Turning Architectural Intent into Enforceable Engineering Signals

The Problem

Most data platforms have architectural principles.

• No SELECT *
• Deterministic joins
• Naming conventions
• Explicit keys
• Reusable logic
• No implicit cartesian joins
• Stable projection ordering

They live in slides.
Or wiki pages.
Or Confluence.

But they do not live in the system.

And when architecture is not executable, it becomes advisory.
Advisory architecture erodes under delivery pressure.

---

The MDDE Position

Metadata-Driven Data Engineering treats architecture as queryable structure.

Once SQL is parsed into metadata:

• Entities exist
• Attributes exist
• Lineage exists
• Joins are modeled
• Filters are normalized
• Functions are extracted

At that point, compliance is no longer philosophical.
It becomes measurable.

---

From Rules to Scoring

MDDE introduces four structural components:

• metadata.compliance_rule
• metadata.optimizer_diagnostics
• metadata.proposal
• metadata.entity_compliance_score

Compliance is not binary.
It is structured.

---

Rule Definition

Example rule:

INSERT INTO metadata.compliance_rule (  
    rule_id,  
    rule_name,  
    severity,  
    category,  
    sql_condition  
)  
VALUES (  
    'CR_SELECT_STAR',  
    'SELECT * usage is forbidden',  
    'error',  
    'projection',  
    'EXISTS (  
        SELECT 1  
        FROM metadata.attribute a  
        WHERE a.is_star = TRUE  
    )'  
);

Rules operate on metadata, not raw SQL text.
That distinction matters.

---

Diagnostics as First-Class Records

When the optimizer runs, it does not print warnings.
It writes structured diagnostics.

Example:

SELECT  
    entity_id,  
    diagnostic_type,  
    severity,  
    message  
FROM metadata.optimizer_diagnostics  
WHERE severity = 'error';

Example diagnostic:

Now architecture becomes queryable.

---

Scoring the Model

Compliance scoring is computed per entity and per model.

Example scoring query:

SELECT  
    e.entity_name,  
    SUM(CASE WHEN d.severity = 'error' THEN 1 ELSE 0 END) AS error_count,  
    SUM(CASE WHEN d.severity = 'warning' THEN 1 ELSE 0 END) AS warning_count  
FROM metadata.entity e  
LEFT JOIN metadata.optimizer_diagnostics d  
    ON e.entity_id = d.entity_id  
GROUP BY e.entity_name;

Score calculation can follow weighted logic:

• Error = -5
• Warning = -2
• Info = 0

Final model score:

SELECT  
    100  
    - (5 * SUM(error_count))  
    - (2 * SUM(warning_count)) AS compliance_score  
FROM metadata.entity_compliance_score;

Architecture is now quantifiable.

---

CI Gates: Where Enforcement Happens

Observation and enforcement are separate.
That principle remains intact.

But CI is where enforcement becomes appropriate.

Example pipeline:

Parse → Analyze → Optimize → Score → CI Gate

Example CLI execution:

python -m mdde.scripts.analyze_model mdl_sales --verbose  
python -m mdde.scripts.optimize_model mdl_sales --verbose  
python -m mdde.scripts.score_model mdl_sales --fail-on-error

Fail condition:

if model.error_count > 0:  
    sys.exit(1)

Now GitHub Actions, Azure DevOps, or GitLab CI can enforce:

• No blocking errors
• Score above threshold (e.g., ≥ 85)
• No unresolved lineage
• No ambiguous joins

---

Example CI Gate Configuration

name: MDDE Compliance  
on:  
  pull_request:  
jobs:  
  compliance:  
    runs-on: ubuntu-latest  
    steps:  
      - uses: actions/checkout@v3  
      - name: Run MDDE Pipeline  
        run: |  
          python -m mdde.scripts.full_pipeline mdl_sales  
      - name: Enforce Compliance  
        run: |  
          python -m mdde.scripts.score_model mdl_sales --min-score 90

If the score drops below 90, the pull request fails.

Not because someone “disagreed”.
Because structure violated policy.

---

What This Enables

1️⃣ Architectural Drift Detection

If someone introduces:

• A cartesian join
• A new implicit filter
• A reordered projection
• A missing key

The metadata model changes.
Diagnostics appear.
Score drops.
CI fails.

---

2️⃣ Migration Safety

When migrating engines:

• Parse both versions
• Compare metadata
• Compare compliance scores

You are no longer diffing text.
You are diffing structure.

---

3️⃣ Governance Without Meetings

Rules are not debated repeatedly.
They are encoded once.

Architecture becomes reproducible.

---

The Key Design Principle

MDDE does not enforce by default.
It observes.

CI enforcement is optional.

This avoids:

• Premature rigidity
• Over-engineering
• Blocking experimentation

Teams can run diagnostics in “informational mode” for months before activating gates.

---

A Real Example: Projection Stability

Rule:

All generated SQL must be deterministically ordered by ordinal position.

Metadata check:

SELECT entity_id  
FROM metadata.attribute  
WHERE ordinal_position IS NULL;

If any row exists → violation.

That guarantees:

• Stable diffs
• Reproducible generation
• Clean Git history

Small rule.
Huge impact.

---

Compliance Is Not About Policing

It is about making intent durable.

The moment architecture is encoded in metadata:

• It can be validated.
• It can be scored.
• It can be versioned.
• It can be enforced.

Without tribal knowledge.

---

Key Takeaways

• Architecture without enforcement decays.
• SQL text cannot be reliably validated by string rules.
• Metadata enables structural validation.
• Compliance scoring turns principles into signals.
• CI gates turn signals into engineering guarantees.
• Observation and enforcement must remain separate.

---

Next in the Series

In the next article:

AI-Assisted Modeling — Realistic, Non-Hype Metadata Copilots

We’ll explore how LLMs can assist metadata evolution without hallucinating architecture.