MIT Sloan Management Review Article on Avoid ML Failures by Asking the Right Questions

Machine learning solutions can miss the mark when data scientists don’t check their assumptions. Adopting a beginner’s mindset in any domain can help.

In our collective decades of experience building, leading, and studying companies’ machine learning (ML) deployments, we have repeatedly seen projects fail because talented and well-resourced data science teams missed or misunderstood a deceptively simple piece of the business context. Those gaps create obstacles to correctly understanding the data, its context, and the intended end users — ultimately jeopardizing the positive impact ML models can make in practice.

We have discovered that small mistakes and misunderstandings are much less likely to cascade into failed projects when development teams engage with colleagues on the business side and ask enough questions to deeply understand the process and the problem at hand. Asking questions might seem like a simple step, but that might not be part of a company’s, team’s, or an industry’s culture. Appearing to be in command of all the information needed may be one of the ways employees signal competence in the organization. And while data scientists might possess technical mastery, they can lack the soft skills to reach a deep, accurate mutual understanding with business partners.

About the Author

Dusan Popovic is head of data science at Anheuser-Busch InBev, Commercial Analytics Europe. Shreyas Lakhtakia is a graduate student at Stanford University. Will Landecker is the former AI ethics lead and data science tech lead at NextDoor. Melissa Valentine is an associate professor of management science and engineering at Stanford University.

Learn more about MIT SMR.