Data science teams are key to making AI work. They’re the ultimate architects of machine learning models, but they often rely on engineers to deploy and productise those models. 

And while companies are hiring more data scientists than ever, it’s estimated that about 90% of investment in AI initiatives is wasted as very few commercial AI models are actually turned into a product. The reason? Data and data science teams just aren’t being used in the right way. 

If you’re a fast-growing startup trying to build an AI-powered product and tearing your hair out, here are some of the problems you’re likely facing — and how to avoid them. 

Issues with data

The first hurdle to getting a return on your AI investment lies with data. Data fuels AI, but data within a business is often disjointed. Different teams use different data sources that are meant to contain the same information (and often don’t). That’s why understanding and agreeing on a single source of data truth is a crucial first step in the AI journey. 

What to look out for here: agreeing on a single source of truth and achieving “data maturity” can be time-consuming. It’s difficult to estimate how long it will take at the beginning of a project and even harder to justify that time spend to management. Whatever the pressure, this isn’t a stage that can be rushed. If the data used is inaccurate, then the outputs or recommendations generated by your data science team will be too.

Issues with tech

Complicated tech stacks are the sister hurdle to issues with data. Data often sits in many different places. Most are department-specific, some might be on the cloud, others in the server, and some essential data might even live in poorly-formatted spreadsheets; navigating that is a data science nightmare. 

In larger organisations, waiting for the business intelligence team to dig up the necessary data is an all too frequent blocker for the data science team. 

Issues with culture

Which brings us to culture. Technical teams tend to lack business understanding — this is particularly true for data science, still a relatively new field. Most practitioners haven’t had the experience to get to grips with commercial goals. They are reliant on support from business users who know the objectives they need a model to drive. Yet data science is typically isolated from the commercial teams it’s designed to support, and more than a few of those business users will view AI sceptically.

How to avoid the AI and data science traps?

Getting around all these problems means building a data science team that’s outcome-focused, collaborates closely with other functions in the business and isn’t obsessed with 100% accuracy. This new standard of data science prioritises shipping product, and there are a few practical things you can do to get there. 

“If you assume that solutions built by a data science team must get things right 100% of the time you’ll never create any solutions”

First, if you assume that solutions built by a data science team must get things right 100% of the time you’ll never create any solutions. Why spend a year trying to improve efficiency by 10%, when the initial machine learning model hit 90% accuracy and was built in only five months? Focusing on what you want to achieve, getting the model built, put into production and delivering value, without working endlessly for infallibility, speeds up time to value. Remember, AI solutions aren’t a silver bullet.

Next, nominating a non-technical “super user” from a commercial team to work alongside the data science team is a great start. The super user feels empowered to help shape the product, and can act as an advocate for it within their wider team. This approach also ensures data scientists understand how a solution will be used. We know from experience that data scientists often feel undervalued and struggle to see how their work is impacting their company; greater integration can be a great tool for engagement and talent retention. 

“Nominating a non-technical “super user” from a commercial team to work alongside the data science team is a great start”

Finally, start with the end in mind. Most data scientists have been trained to think from the bottom up, understanding what data they have and then deciding what they could do with it. This results in data science projects — hypothetical solutions to data problems, rather than AI solutions that improve business performance.

By focusing on outcomes from the very beginning, as well as understanding what is feasible with the data you have, you can meet somewhere in the middle; where the solution is possible to build (rather than an unrealistic idea) but also highly valuable (rather than something interesting but not useful). Prioritise getting an end-to-end solution live and generating value as quickly as possible, then focus on iterating and improving it. Solutions don’t need to start off hugely sophisticated.

AI will ultimately become the preeminent software for business and will open up new opportunities for individuals and organisations. But to get to this promised land, we need to stop seeking perfection and theory and take an iterative approach. Fix the data, centralise the tech and foster a collaborative culture — only then will AI begin to benefit your business.

Amy Sharif is head of data science at Peak