Building a data team: build it yourself, hire in or outsource?

Translated with AI. This article was translated by an AI model from an article by Twentynext in another language, and the translation was checked automatically against the original. Verify figures and sources before acting on them. See our disclaimer.

Almost every organisation that gets serious about data runs into the same question: who is actually going to do this? The vacancy for a data engineer remains open, the hired specialist leaves just as the project gains momentum, and the quote from that external agency raises the question of how much of the knowledge will remain in house afterwards.

The choice between building yourself, hiring in or outsourcing is not an either/or question, but a trade-off that can turn out differently per phase and per role. Below we set the options side by side, with the considerations and risks that go with them, so that you can make a choice that fits where your organisation stands right now.

Start with the problem, not the people

A common risk when building a data team is starting with recruitment. A data scientist is hired because “we need to do something with AI”, who then discovers that there is no clear brief and no process owner waiting for results.

The order should be the other way round. First: which decisions or processes do we want to improve with data? Only then: which capabilities do we need for that, and in what order? Coalesce sums up this principle concisely in its guide to data teams: structure your team around the problems you need to solve today, and leave room to evolve. Understand where you are in your data journey before you start recruiting.

Look first at the foundation. Data engineers build the infrastructure that brings in and transforms data so that analysts and data scientists can work with it. So ask yourself honestly whether that foundation is in place before you add analysis capacity.

The three basic roles you will need sooner or later

At its core, a working data team runs on three types of work, regardless of who performs them:

Data engineering: building the infrastructure that brings in data, transforms it and makes it usable for analysts and data scientists. Without this work, everything else comes to a standstill.

Analysis and reporting: translating data into insights the business can act on: dashboards, KPIs, ad hoc analyses. This is where the organisation sees the value.

Direction and ownership: someone who sets priorities and builds the bridge to the business. This role usually receives less attention in recruitment plans than the technical roles. Wrongly so: without direction, there is no one to determine what the technical capacity works on.

The order in which you fill these roles depends on your maturity. Our advice: keep the direction role as close to your own organisation as possible.

Option 1: building your own team

An in-house team suits organisations where data becomes a structural part of business operations. Knowledge stays in house and the team gets to know its own context. Whether it is also cheaper per hour than hiring in depends on staffing levels, employer costs and rates; calculate that rather than assuming it.

Be honest about the preconditions:

• Recruitment takes time and appeal. Ask yourself why a good data specialist would want to work for your organisation in particular, and how long you can wait for the right candidate.
• One person is not a team. There is a risk that the first hire has to do everything at once: infrastructure, dashboards, advisory questions. That makes the set-up fragile and the role unattractive.
• You need to be able to assess what you are hiring. Without subject-matter knowledge in house, it is difficult to recognise quality in a job interview, let alone to steer the work afterwards.
• Retention requires prospects. An isolated specialist without colleagues or a clear brief has little reason to stay. Think in advance about room to learn and visible impact.

Building yourself is therefore not a quick win. Expect a longer run-up, and consider how the organisation will achieve results in the meantime. That is exactly where the other two options come into the picture.

Option 2: hiring in specialists

Hiring in, whether freelancers or secondment, can suit temporary peaks, specialist knowledge or bridging a recruitment period. How quickly you actually have capacity depends on availability and contract form.

The risks are predictable and deserve attention in advance:

Knowledge can leave with the person. If the hired engineer is the only one who understands the data platform, you have not hired capacity but created a dependency. So make documentation and knowledge transfer part of the assignment, not an afterthought.

Hiring in can become structural. What starts as a bridging measure can continue unnoticed. Check periodically whether the flexibility you are paying for still matches what you need.

Without your own direction, no one steers. A hired specialist does what is asked. If no one in the organisation asks the right questions and guards priorities, a lot of hard work goes into the wrong things.

In our view, hiring in works best as a supplement to an in-house core, with an explicit end date and a plan for securing knowledge. As a complete replacement for an in-house team, the set-up quickly becomes fragile.

Option 3: outsourcing to an agency or partner

With outsourcing, an external party takes on a defined part of the work: building a data platform, delivering reports or running operations. Note: outsourcing is not by definition buying results. Contract forms range from an obligation to deliver results to an hourly basis with shared direction: make explicit in advance what you are buying and who is steering what.

A partner can bring experience from comparable projects and several roles at once. Test that during selection instead of assuming it: ask about the approach, who actually does the work and how knowledge is transferred.

The risks lie at the edges:

• Vendor lock-in. If the platform can only be managed by the supplier, your negotiating position weakens. Ask in advance how transferability is arranged: documentation, standard technology, exit agreements.
• Distance from the business. An external party does not automatically know your processes. Without active involvement from your organisation, even a technically sound platform delivers little.
• Direction remains yours. Even with far-reaching outsourcing, the question “what do we want to achieve with data?” remains an internal responsibility. You cannot buy that in.

Outsourcing can be a logical route for managing existing solutions, or for (semi-)public organisations where permanent staffing is difficult to expand. Treat it as a collaboration, not an order. You can read what that looks like with us on the page about outsourcing data consultancy.

In practice: think in phases, not flavours

The question “build, hire in or outsource?” suggests a choice between three flavours. It can also differ per role and per phase, with the mix shifting over time. One possible pattern:

Phase 1: laying the foundation. Working with a partner to build the data foundation. Internally: one owner who steers and learns along the way.

Phase 2: building the core. Hiring the first in-house staff, close to the business. The external party moves towards operations and specialist questions.

Phase 3: running independently. The in-house team carries the daily operation. External knowledge is deployed in a targeted way for peaks and new topics, with knowledge transfer as a standing requirement.

Keep teams compact. Guides on data platform teams commonly mention three to eight people per team, and when there is more work, prefer splitting by area of focus to growing larger. And be deliberate about reorganising: Snaptravel tried five data team structures in nine months before settling on a domain-based model, which dbt Labs itself calls “a lot” of change. Each change costs a team time and focus, so make sure the expected gain justifies it.

The questions that determine the choice

Before you post a vacancy or request a quote, answer these questions honestly:

1. Is this temporary or structural work? Structural work belongs in house in the long run; temporary work does not.
2. Can we assess and steer quality? If not, the first step is not recruiting but organising direction, if necessary with external guidance.
3. How quickly do we need results, and which route can realistically deliver that? Test lead times per option in your own situation. And bear in mind: speed without securing knowledge is an illusory gain.
4. What happens if this person or party leaves? If the answer is “then we grind to a halt”, the set-up is too fragile.
5. Where does the distinctive capability lie? A useful starting point: keep knowledge of your own processes and data internal, and consider placing generic execution externally. Test this per role rather than applying it as a fixed rule.

The first step

Do not start with an organisation chart or a vacancy text. Start with a list of the three to five concrete questions or processes where data needs to make the difference in the coming year, drawn up together with the process owners, not by IT alone.

Then set that list against the capabilities you currently have in house. The gap between the two determines what you need: which roles, in which order, and what makes sense to do yourself, hire in or outsource. Anyone who skips this exercise buys capacity without direction, and that is rarely the intention.