Modeling the Common Integration Layer for Financial Services

For data engineers and architects designing a data mesh platform where financial, risk, and regulatory data must remain reliable and auditable.

Part 5 (final) of the series: Designing a Common Integration Layer (CIL) for Data Mesh Platforms

After covering fundamentals, managing change, delivery models, and governance, we finish with a financial services deep dive.

We explain concepts like contract-level vs event-based accounting, daily vs end-of-period closing, corrections & adjustments, non-financial risk, and how sign-off and evidence become non-negotiable in regulated reporting.

This case study shows the CIL in action where compliance and trust are paramount.

Press enter or click to view image in full size

Why Finance Needs Special Attention

Data mesh promotes domain ownership and self-serve data products, but finance has unique challenges:

Accounting precision — every number must tie back to contracts and transactions.
Regulatory traceability — supervisors and auditors need to follow data lineage.
Daily & event-based reporting — numbers change as positions revalue and adjustments arrive.
Strict control frameworks — sign-offs, evidence, and redeliveries must be managed.

The challenge: federate ownership while ensuring trust and compliance.

Core Financial Modeling Concepts (With Examples)

Contract-Level Accounting — store & reconcile per contract (e.g., each mortgage, policy, trade).
Event-Based Accounting — immutable events like TRADE_EXECUTED or FEE_ACCRUED.
Daily Accounting — rebuild balances from events each day for accuracy.
Corrections & Adjustments — late trades and fixes added as delta events, never overwrite.
Preliminary vs Final Datasets — early deliveries for analysis vs formally signed-off sets.
Redeliveries — controlled publication of corrected finals with versioning.
Non-Financial Risk — operational/compliance incidents with lineage for audit.

Platform Layers in Finance

┌───────────────────────────────┐
│ Consumption Layer │
│ – Regulatory reports │
│ – Risk & finance models │
│ – Calculation applications │
├───────────────────────────────┤
│ Common Integration Layer │
│ – Contract-level IDS │
│ – Event & daily accounting │
│ – Adjustments & corrections │
├───────────────────────────────┤
│ Acquisition Layer │
│ – Core banking / trading │
│ extracts │
│ – G/L & sub-ledger feeds │
└───────────────────────────────┘

Governance & Control

Control Framework — checks for completeness, reconciliation, DQ, lineage.
DQ Checks — automated validations: row counts, code sets, reconciliation.
Operational Sign-Off — technical confirmation that pipelines succeeded & data is fresh.
Formal Sign-Off — business approval that numbers & semantics meet requirements.
Evidence for Sign-Off — DQ dashboards, reconciliation reports, lineage graphs.
Controlled Data Delivery — certified, governed channels (APIs, curated tables, delivery sets).

🏦 Case Study — Monthly Close of a Trading Book

Goal: produce accurate month-end P&L and regulatory filings for trading.

Data Flow

Explanation:

Acquisition Layer

Trading systems emit TRADE_EXECUTED, AMEND_TRADE, MARK_TO_MARKET.
GL feeds provide balances.

Common Integration Layer

Contracts: trades keyed by TRADE_ID.
Events: mark-to-market, fees, adjustments.
Daily accounting: rebuilds balances, reconciles with GL.

Preliminary Delivery

6 AM: trading_positions_prelim for early risk VaR.
Operational sign-off confirms completeness & pipeline success.

Final Delivery & Sign-Off

After reconciliation, Finance formally signs off trading_positions_final_2025-10.
Evidence: DQ dashboards, lineage, reconciliation logs.

Redelivery Scenario

A late FX adjustment triggers trading_positions_final_2025-10_v2.
Consumers see lineage and change notes.

Consumption Layer

Risk engines, finance reports, regulatory filings consume signed-off sets confidently.

Practical Advice

Model contracts & events as first-class citizens.
Keep facts immutable; append corrections.
Separate preliminary & final clearly.
Automate DQ & reconciliation inside pipelines.
Version & document redeliveries.
Generate sign-off evidence automatically.

How This Fits Data Mesh

Domains own their trading, ledger, or risk data products.
The CIL integrates them into contract-level, event-driven, harmonized sets.
Governance & sign-off give trust without reverting to a monolithic warehouse.

Wrapping Up the Series

This article concludes our series on designing a Common Integration Layer (CIL) for modern, mesh-style data platforms.

Across the five parts we have:

Laid the Foundations — understanding what a CIL is and why it matters in a data mesh context.
Explored Managing Continuous Change — using transition states, interface models, and regression testing to evolve safely.
Defined Data Products & Delivery Models — how to package integrated data for reuse and controlled consumption.
Embedded Governance & Trust — automating data quality, sign-off, and evidence for compliance and audit.
Applied the Patterns to Financial Services — where contract-level accounting, risk, adjustments, and regulatory sign-off put these concepts to the test.

The key message:

A well-designed CIL doesn’t fight the mesh — it enables domains to interoperate, modernize, and comply while keeping change manageable and data trustworthy.

You can now:

Speak a shared language about integration concepts (IDS, UCS IDS, exposure datasets, controlled delivery).
Manage platform and modeling evolution without big-bang rewrites.
Build governed, evidence-driven data flows that satisfy both internal and regulatory consumers.
Apply these patterns in highly regulated industries like financial services.

What’s next for you

Use the CIL patterns to guide platform modernization and mesh adoption.
Introduce transition states and controlled delivery early to avoid costly rewrites later.
Build your control framework & sign-off process into CI/CD from day one.

Thank you for following the series — I hope it helps you design a CIL that’s both flexible and trusted, no matter how your data platform evolve.