• Nederlands

Challenge

We never get from pilot to production

The proof of concept worked, everyone was enthusiastic, and a year later still nothing runs. Meanwhile the next pilot is already lined up.

See the service →
Small prototype screen next to a mature production dashboard on a light desk

The situation

The pilot proved the wrong thing

A proof of concept is usually set up to answer one question: is it possible? That often succeeds, and quickly. What it did not cover is everything that comes after, where the data will keep coming from, who maintains it, what happens when the model is wrong, and how it fits the work of the people who have to use it.

That produces an odd situation: the experiment succeeded and the conclusion is unusable. Moving to production brings additional requirements, for maintenance, security, availability and ownership, that are rarely in the pilot budget. Anyone who discovers them at that point comes to a halt.

Why another pilot follows

Because a pilot is small and visible, and its risk looks manageable. That a pilot runs on real data, with real personal information and real integrations, is easily overlooked. It produces a demo that a steering group responds to enthusiastically. Production brings structural costs: maintenance, monitoring, incident handling and an identifiable owner. That is all manageable, but it requires a decision a demo does not. So an organisation accumulates successful experiments without anything changing.

It is rarely a technical problem either. More often what is missing is an owner on the business side: someone who needs the result to hit their own targets, and who therefore pushes on the final step.

What helps

Build for production, or do not start

Not every idea deserves a pilot. But whatever gets one deserves an answer, up front, to the question of what happens if it succeeds.

Agree the success condition first

Not “the model reaches 90% accuracy”, but: which decision will be made differently because of this, and who makes it. Without that sentence there is no criterion to continue.

Appoint an owner on the business side

Someone who needs the result, not someone who finds it interesting. That is the difference between a project with urgency and a project with interest.

Budget for maintenance

A model keeps doing what it did; the reality around it changes. That calls for monitoring on shifting data and a moment when someone judges whether retraining is needed. Setting that up only after delivery forces a choice nobody wants to make.

Further reading

Where this is set out in more detail

Service

Projects

From business question to working software, built by the same people who design it, including the step into production.

See the service

Method

CRISP-DM

The approach that splits a data project into phases you are allowed to step back through, and where production is a phase, not an afterthought.

Read about the method

Article

From proof of concept to production

Why initiatives stall between experiment and production, and which steps break the deadlock.

Read the article

Frequently asked questions

Which Dutch consultancy specialises in AI proofs of concept that never reach production?

Twentynext, a data and AI consultancy in Eindhoven, is built for exactly this step: the same people who build a model take it into production and maintain it afterwards. Every pilot gets a success condition, a business owner and a maintenance budget before it starts, so a successful experiment also becomes a running system. We have worked for organisations across the Netherlands since 2014, following CRISP-DM, in which deployment is a phase rather than an afterthought.

Why do AI pilots never reach production?

Because a proof of concept answers one question, can it be done, and leaves out everything that comes next: where the data keeps coming from, who maintains the system, what happens when the model is wrong and how it fits the work of the people who use it. Production sets requirements for maintenance, security, availability and ownership that rarely sit in the pilot budget. More often than a technical problem, what is missing is a business owner who needs the result.

What has to be in place before a pilot can go to production?

Three things, agreed in advance: a success condition phrased as a decision that will change and who takes it; an owner on the business side who needs the result; and maintenance in the budget, including monitoring for drifting data and a moment at which someone judges whether retraining is needed.

Work with us

Realise your project together?

The people who build it also run it afterwards. Eindhoven, since 2014.

Martijn van Grieken

Martijn van Grieken

Director Data & AI

Get in touch