AM
All posts
Data & Digital Platforms2 min read

Open source is an architecture decision, not a cost decision 

The large data lakes I have built stand on open-source products because they compose better. The savings are a consequence of that choice, not the reason for it, and treating them as the reason leads you to build the wrong thing.

Open source gets sold internally as the cheap option. It is a persuasive argument and a corrosive one, because a decision justified purely on price is re-litigated the moment the price argument weakens, and it always weakens once you have staffed the thing properly.

The durable argument is composition.

Why composition matters at scale

A large data estate is never one problem. It is ingestion, storage, catalogue, transformation, governance, query, and serving, each with its own rate of change and its own idea of what good looks like. An integrated commercial platform makes a bet that these should evolve together, on one release cycle, under one set of design assumptions.

At small scale that bet pays. At large scale it stops paying, because your components stop changing at the same speed. Storage economics move on one timeline, query engines on another, and governance on whatever timeline the regulator sets. A stack you can recompose absorbs that. A stack you cannot forces you to hold the slowest component's pace across the whole estate.

Two ways to hold a data estate together

At small scale the left column is an advantage. The bet stops paying when your components stop changing at the same speed.

The costs that are real

Open source is not free, and pretending otherwise is how these programmes lose credibility in year two. The real costs are:

  • Integration work that a commercial vendor would have absorbed, which is now permanently yours.
  • Version and compatibility management across components that do not coordinate their releases.
  • Deep expertise in a handful of projects, which means real people with real market value.
  • The discipline to say no to components: a stack of thirty tools composes worse than a stack of eight.

Put those in the business case. A programme that has already named its costs is far harder to attack than one that has to discover them in public.

What it buys you

The point is not that you can read the source. It is that no single vendor's roadmap can hold your data estate still.

Optionality is the actual asset. When a component stops being the right answer (and at a long enough horizon, each one will), you replace that component rather than negotiating a migration off a platform. That is worth considerably more over a decade than the licence line you removed in year one, and it is the thing to lead with when the decision is being made.

Enterprise Technology Executive

Anurag Kumar Mishra

I lead large-scale transformation across telecom and enterprise technology, connecting technology strategy, operating models and capital allocation to measurable business outcomes.

LinkedIn

Keep reading

Next step

Let us talk about the transformation you are actually trying to fund.

Board-level conversations, CXO roles, advisory work, and speaking. If the problem involves large data, complex operations, and a number that has to move, it is worth a conversation.

Start a conversation