Semantic Integration: Connecting Sources Without Flattening Their Meaning
By Arshad Ansari
Short answer: do not merge sources into one shared meaning. Map each source to your model with explicit, written rules. Keep the differences that matter to the business.
Most integration work fails in a quiet way. The pipeline runs. The tables join. The numbers are wrong because two systems used the same word for different things.
The flattening problem
Take "customer". In a billing system, a customer is whoever pays the invoice. In a CRM, a customer is a person you are talking to. In a support tool, a customer is whoever opened a ticket.
The usual fix is a canonical customer table. You union the three sources, deduplicate, and ship it. Now the dashboard says "customers" and everyone reads it their own way.
Three things go wrong:
- Real differences vanish. A payer who is not a user becomes a user.
- Disagreements hide. When the sources conflict, a rule you never wrote down decides who wins.
- Changes spread. When billing changes its meaning, every report built on the merged table shifts.
This is flattening. You got one table, and you lost the facts that told the sources apart.
Contexts are real
Domain-driven design has a name for this. Each area of the business is a bounded context. Inside it, a word has one meaning. Across contexts, the same word may mean something else, and that is fine.
The goal is not to remove those differences. The goal is to connect the contexts and say, in writing, how they relate.
Explicit mappings
An explicit mapping is a record of how a concept in one source relates to a concept in your model. It is data, not a comment in someone's head.
A useful mapping states four things:
- the source concept and its owner;
- the target concept in your model;
- the relationship: same as, narrower than, broader than, or related to;
- the rule that applies it, and what it drops.
The relationship is the part people skip. "Billing customer is a narrower case of party" is a different claim from "billing customer equals CRM customer". The first is safe. The second causes the quiet failure above.
Keep mappings in version control, next to the pipeline code. Review them like code. When a source changes, the diff shows which mappings are affected.
The anti-corruption layer
An anti-corruption layer is where mappings are applied. It sits between each source and your model. Its job is translation, and it has one rule: the source's vocabulary stops at the boundary.
In a data platform this is usually a thin staging layer per source. It does not clean up the business. It only renames, retypes and re-expresses source facts in your terms. Everything past that layer speaks your model.
This gives you three benefits:
- A source can change its schema. You fix one layer, not every report.
- A source with poor design cannot shape your model.
- You can read the layer to see what each source really claims.
Keep the differences that matter
Not every difference needs to be preserved. Ask one question: would a decision change if the two meanings were swapped?
If the answer is yes, keep both concepts and give each a clear name: paying_account, crm_contact, support_requester. Link them with a mapping. Let the report say which one it means.
If the answer is no, merge them and record the merge as a mapping. Merging is fine when you chose it. The problem is merging by accident.
What this looks like in practice
When I review a data platform, I look for the places where one word covers several meanings. I ask where the mapping between sources is written down. Often the answer is a join condition inside one SQL file.
Moving that join into a named, reviewed mapping is small work. It makes the meaning of the numbers checkable. That matters more once an AI workflow reads those tables, because a model will treat "customer" as if it had one meaning.
Who this is not for
If you have a single source of truth and one team that owns it, you do not need this. A written glossary is enough. Do not add a mapping layer for a problem you do not have.
Check your own platform
The free Data Platform Teardown is a checklist you can run yourself. It helps you see where your platform stands before you decide on any larger step.
Common questions
- What is semantic integration?
- It is joining data from different sources while keeping track of what each source means by its fields. You map meaning, not only column names.
- What is an anti-corruption layer?
- It is a translation layer from domain-driven design. It sits between a source and your model, so the source's concepts do not leak into your own.
- Why not merge everything into one canonical model?
- A single model forces every context to agree on one meaning. Where they truly differ, you lose information or invent a false agreement.
Get new posts by email
Data engineering notes like this one — what breaks and what it costs, in production.
What breaks and what it costs — pipelines, warehouse bills, and the failures that only show up in production. A few a month, never padded to hit a schedule. No sequence, no pitch deck. Reply 'stop' once and you're off — it reaches me, not a queue.
Want the whole playbook?
If this was useful, the long version is my book. Local-First Analytics — 313 pages, runnable code for every chapter — is the full build: DuckDB, Parquet and Arrow, from install to production. On Amazon, and chapter 1 is free to read here.
Get the bookNot ready to buy? Read chapter 1 free — the whole chapter, no email required.
Rather talk it through? Book a free 30-minute call. No slot that suits your time zone? Email info@hikmahtech.in.