When Schema and Model Drifted Apart

Some bugs are big because their cause is big. Others are big even though their cause is tiny — and those are the more instructive ones. This story is about one of them. It is deliberately kept general — no project, no people involved, no point in time.

Situation

In a large undertaking with several teams, one bug refused to go away. It kept appearing without being easy to pin down, and it resisted the search for longer than its eventual explanation would have suggested. Many hands were working on the same system, and the bug affected an area that several of them touched.

Initial assumptions

A stubborn bug in a large system raises the expectation of a large cause: a deep design flaw, a subtle interaction, an elusive condition. The search aligns accordingly — you look for something complicated because the symptom looks complicated. That expectation initially led right past the actual cause.

Problem

The cause was small. The database schema had changed, but the mapping in the code — the place where the model translates the database into objects — had not been updated along with it. Database and model no longer said the same thing about the same thing. This tiny mismatch grew into a bug that surfaced far away from where it originated and was therefore hard to locate. The problem was not the size of the cause but the distance between cause and effect.

Root cause

The deeper cause was that a change needed to happen in two places but only happened in one. Schema and mapping are two representations of the same truth; when one changes, the other has to follow. Where that coupling is neither visible nor safeguarded — and in a large project with many hands it rarely is on its own — one can move while the other stays put. The two drifted apart, and nobody noticed the moment it happened.

Decision

The decision after the discovery was less about fixing this one bug — that was quickly done — than about making this whole class of bugs less likely: not leaving the agreement between schema and mapping to chance and to individual attentiveness, but making it visible and verifiable.

Implementation

That meant bringing the coupling between database and model to where it gets noticed: a divergence between schema and mapping should become visible early and loudly, not late and as a puzzling symptom. Instead of trusting that everyone who changes the schema also updates the mapping, the drift itself became something the system notices — close to the cause, not far away at the effect.

Trade-offs

This safeguard has a cost: it adds a check that needs maintaining, and it makes the moment of a schema change somewhat more cumbersome, because both now have to match before anything moves on. What the price buys is that a tiny drift no longer turns into a long, cross-team bug hunt. We would decide differently for a very small system where a single person can easily keep schema and model in view — there, the extra check is more effort than protection.

What we learned

Small mismatches between database and model cause disproportionately large problems because the effect surfaces far away from the cause. The effort of a bug hunt is not measured by the size of the cause but by the distance between cause and symptom — and a mapping that has drifted is a tiny cause with a large distance. That is why it pays to make the silent couplings in particular visible: the places where two representations of the same thing have to match but can be changed independently.

How this changed Batunet

Since then, we treat the agreement between schema and model as something the system itself should watch over, not something everyone has to keep in their head. We deliberately look for the silent couplings — the pairs that have to change together — and make any drift visible early. "Someone should have thought of that" has become "the system tells us."

Further reading

• Guide: Zero-downtime database migrations — making schema changes deliberately and safely.
• Guide: Business logic doesn't belong in controllers — designing the translation between database and domain deliberately.
• Guide: Observability is architecture — why a cause should be visible close to its effect.
• Decision: PostgreSQL: JSONB vs. tables — how the shape of the data shapes its coupling to the code.

It is grounded in the Batunet Engineering Method: make silent couplings visible, surface errors close to their cause, don't rely on attentiveness alone.

The size of a bug says nothing about the size of its cause. The most expensive ones are the tiny mismatches whose effect surfaces far away from where they began.