How do iterative work and shared team knowledge affect the performance of machine learning projects facing uncertainty?

Host: Welcome to A.I.S. Insights — powered by Living Knowledge. I'm Anna Ivy Summers.
Expert: And I'm Alex Ian Sutherland. Great to be here, Anna.
Host: Today, we're looking at how organizations can improve the chances that machine learning projects deliver useful solutions without consuming excessive time and resources. The study is titled “Knowledge Overlap and Iterative Development in ML Projects: An Information Processing View.” Alex, why is this an important problem?
Expert: Machine learning projects are often more uncertain than conventional software projects. Teams may not know at the outset whether a promising business problem can actually be solved with the available data and algorithms. And even once a model has been developed, its inner workings may be difficult to interpret. These two challenges—use-case uncertainty and model inscrutability—can make it harder to develop an effective solution efficiently.
Host: So the question is not just whether a team has enough technical expertise, but how it organizes its work around these uncertainties?
Expert: Exactly. The study uses what is called the Information Processing View. The basic idea is that projects perform better when their ability to process information matches the uncertainty they face. The researchers therefore looked at two kinds of capacity: coordination through iterative development, and knowledge overlap between data scientists and business-domain experts.
Host: How did they investigate this?
Expert: They surveyed participants from 141 machine learning projects in organizations with more than 250 employees. The study examined two outcomes: the efficiency of the project—things such as time, budget, and effort—and the effectiveness of the ML solution in meeting technical and user needs.
Host: Let’s start with iterative development. What did the study find?
Expert: Iterative development was associated with higher project efficiency. And this relationship was particularly strong when use-case uncertainty was high. In other words, when teams were still trying to figure out exactly what the ML system should do, working in short cycles and revisiting assumptions appeared especially useful for avoiding wasted time and effort.
Host: Did iterative development also make the final ML solution better?
Expert: Interestingly, the study did not find a significant direct relationship between iterative development and solution effectiveness. The strongest evidence was about efficiency: iteration seems especially useful for helping uncertain projects avoid heading too far in the wrong direction.
Host: What about knowledge overlap between the business and technical sides?
Expert: That is where the findings become particularly interesting. The researchers looked at knowledge overlap in both directions. First, they examined whether domain experts understood data science. Second, they examined whether data scientists understood the business domain.
Host: And did those two forms of knowledge overlap have the same effect?
Expert: No, and that is one of the most useful findings. Domain experts’ data science knowledge was associated with higher project efficiency overall. It also became particularly valuable for solution effectiveness when use-case uncertainty was high. When business experts understand enough about data, algorithms, and ML techniques to engage meaningfully with data scientists, they may be better able to help the team identify feasible and useful applications.
Host: So some technical literacy on the business side can be especially valuable when the project is still searching for the right use case.
Expert: Exactly. But the pattern is different for data scientists’ domain knowledge. The study did not find a general positive effect of domain knowledge across all projects. Instead, it became particularly valuable for solution effectiveness when the ML model was highly inscrutable—that is, when it was difficult to understand how inputs led to outputs.
Host: Why might that be?
Expert: When a model is difficult to interpret, data scientists who understand the business context may be better able to communicate with domain experts, interpret model behavior, and fine-tune the solution to fit operational requirements. The study's results are consistent with that explanation, although the survey itself does not directly test those underlying mechanisms.
Host: If you are a business leader or project manager, what should you take away from this?
Expert: I would highlight three things. First, consider building enough data-science literacy among domain experts for them to engage productively with technical specialists—particularly when the use case is still uncertain. Second, use iteration where it adds value. The study suggests that highly iterative development is especially useful when the project is still exploring what the right use case and requirements should be. When use-case uncertainty is low, its efficiency benefits appear much smaller. Third, if you are working with models that are difficult to interpret, pay particular attention to whether the data scientists involved understand the business domain. Their domain knowledge appears especially valuable for developing effective solutions under high model inscrutability.
Host: So the broader lesson is not simply “iterate more” or “give everyone more cross-functional knowledge.”
Expert: Exactly. The central lesson is that ML projects need the right capability for the uncertainty they face. Iteration helps most when the use case is uncertain. Domain experts’ technical knowledge helps especially when the business problem is unclear. And data scientists’ business knowledge becomes particularly valuable when the technical model itself is difficult to interpret.
Host: That is a much more nuanced message than saying there is one best way to manage every ML project.
Expert: Absolutely. The challenge is to match coordination and knowledge to the type of uncertainty the project actually faces.
Host: Thank you so much for sharing these insights today, Alex.
Expert: My pleasure, Anna.
Host: And thank you all for tuning in to A.I.S. Insights — powered by Living Knowledge. Join us next time as we continue unpacking the latest research shaping business and technology.