What challenges arise when deploying an internal AI chatbot across complex, specialized organizational departments?

Host: Welcome to A.I.S. Insights — powered by Living Knowledge. I’m Anna Ivy Summers. Today, we examine a study titled "Ground Truth Heterogeneity: Exploring Development and Non-Clinical Use Of Retrieval-Augmented Generation Chatbot In Healthcare." This study investigates what happens when an organization builds an internal AI chatbot using retrieval-augmented generation, or RAG, to support staff during a major IT transition. Joining me is Alex Ian Sutherland. Alex, welcome.

Expert: Thanks, Anna. It is great to be here.

Host: Many organizations implement RAG to link large language models directly to internal documents, hoping to eliminate hallucinations. But this study points out a major hurdle: the problem of ground truth in complex organizations. Can you break that down?

Expert: Absolutely. RAG connects a general model to an organization's internal files. The assumption is that this anchors the AI in objective reality. But in large enterprises—like the regional healthcare system examined in this study—knowledge is rarely uniform. Different units maintain localized, sometimes conflicting routines. When organizations try to establish a single source of truth, they run into what the authors term ground truth heterogeneity: a condition where distributed, specialized knowledge practices make it difficult to build a stable retrieval base.

Host: Let us look at how the researchers investigated this. What was their approach?

Expert: The researchers conducted a single-case study of a Swedish public healthcare provider overseeing around ten thousand employees. They analyzed fifteen semi-structured interviews with project leaders, technical staff, domain experts, and healthcare workers. They also examined ninety-three internal survey responses, case documents, and over thirty-seven thousand user interaction logs from the chatbot's first six months of operation.

Host: That provides a detailed view of both development and actual use. What did the findings reveal about system performance?

Expert: First, RAG does not eliminate hallucinations. Instead, it relocates the issue from model training to the knowledge base curation process. When internal documents contained conflicting or missing information, the chatbot produced inaccurate responses. Second, the developers faced an accuracy-versus-relevance trade-off. To stop errors, they removed localized routine documents from the knowledge base, keeping only centralized policies. While that reduced hallucinations, it meant the chatbot could no longer answer unit-specific questions.

Host: Which directly impacted user satisfaction across different roles, right?

Expert: Exactly. The study observed a clear divide. Managers, whose work depends largely on centralized, high-level administrative guidelines, reported high satisfaction. But frontline clinicians needed unit-specific details, such as local referral routines. For them, general answers were inadequate, leading several to abandon the tool after initial trials.

Host: The study also noted that prompt literacy played a significant role in retrieval failures.

Expert: Yes. Over eighteen percent of all queries triggered an automated message stating the chatbot could not answer. User logs showed that many employees searched with single keywords or informal phrasing rather than structured prompts. While the organization advised staff to speak to the bot like a colleague, colloquial language often resulted in retrieval mismatches.

Host: Looking at practical takeaways, what does this mean for leaders planning internal generative AI deployments?

Expert: The primary takeaway is that maintaining a RAG knowledge base is an ongoing sociotechnical process, not a one-time technical task. Organizations cannot simply index an existing file repository and assume the data is ready. Before deploying, managers should conduct a heterogeneity audit to map where operational routines diverge across units.

Host: And technically, how should organizations handle those divergent routines?

Expert: Rather than relying on a single, unified knowledge base, organizations with significant local variation should consider modular retrieval architectures that scope data by unit or role. Additionally, leaders must allocate resources for continuous domain expert validation and provide training to help employees ask questions in ways the retrieval system can process.

Host: In short, the success of internal generative AI depends just as much on human governance and knowledge architecture as it does on the underlying model. Alex, thank you for breaking down the study for us.

Expert: You are welcome, Anna.

Host: And thank you for listening to A.I.S. Insights — powered by Living Knowledge. Join us next time for more practical analysis of business and technology research.