How does trusting AI feedback compare to human feedback, and how does it impact team performance?

Host: Welcome to A.I.S. Insights — powered by Living Knowledge. I'm Anna Ivy Summers.

Expert: And I'm Alex Ian Sutherland. Today, we are discussing a study presented at the International Conference on Information Systems in 2026, titled "When AI Credibility Fractures: Human vs. AI Feedback Sources in Global Virtual Student Team Projects," by Dhruv Pratap SINGH and Vasyl Taras.

Host: Alex, this study examines whether Source Credibility Theory, which was originally developed to explain how people evaluate human communicators, transfers intact to generative AI feedback in collaborative digital environments.

Expert: Exactly, Anna. In both education and professional settings, people no longer rely on a single human evaluator. Teams routinely draft work, prompt generative AI systems like ChatGPT or Claude for feedback, consult mentors or supervisors, and integrate all those suggestions within tight timelines. The core problem is that we know very little about whether users evaluate AI credibility along the same dimensions as human feedback, or whether trusting AI feedback actually improves or hinders team performance.

Host: How did the authors investigate that question empirically?

Expert: The study analyzed data from 3,776 participants across 31 countries working in global virtual teams on an 11-week consulting project. At week nine, participants evaluated the credibility of three distinct feedback sources: course professors, program coaches, and generative AI tools. Then, at week 11, performance was measured through both self-evaluations and peer evaluations from teammates.

Host: Measuring outcomes two weeks later and incorporating peer evaluations helps separate perceptions from actual observed contributions. What were the key findings?

Expert: The first key finding concerns the structure of credibility itself. For human sources—both professors and coaches—the three traditional dimensions of competence, trustworthiness, and goodwill collapsed into a single, global evaluation. For generative AI, however, credibility remained fractured into three statistically distinct dimensions.

Host: So participants did not evaluate AI feedback as a single unified impression. How did those distinct AI dimensions connect to actual performance?

Expert: That is the central finding of the study. For professors, higher credibility consistently predicted both higher peer-evaluated performance, with a standardized beta of point one five five, and higher self-evaluated performance, with a beta of point one one three, both at p less than point zero zero one. But for generative AI, the dimensions pulled in opposite directions.

Host: In opposite directions?

Expert: Yes. AI competence was positively associated with peer-evaluated performance, with a beta of point one three eight at p equals point zero zero three. In contrast, AI trustworthiness was negatively associated with peer-evaluated performance, with a beta of minus point two six eight at p less than point zero zero one. AI goodwill had no significant relationship with peer evaluations.

Host: Why would viewing AI as trustworthy be negatively linked to how peers evaluate someone's performance?

Expert: The authors suggest that high AI trustworthiness reflects a form of reliance where users treat AI outputs as sufficiently dependable to reduce their own independent scrutiny and effort. When users recognize AI's competence, they use it productively as an idea generator. But when they trust it unconditionally, they may offload critical judgment. Importantly, when the researchers modeled AI credibility as a single overall score, it masked this conflict and showed only a misleading net negative association.

Host: The study also examined whether actually using the source changed these dynamics, right?

Expert: Yes. Across all feedback sources, credibility associations were stronger or only statistically detectable among participants who actively engaged with the source, demonstrating that credibility operates as a behaviorally activated mechanism rather than an abstract opinion.

Host: What does this mean in practice for educators, managers, and system designers?

Expert: The primary takeaway is to aim for calibrated reliance rather than generic trust. In collaborative work, teams should be encouraged to utilize AI for its competence while maintaining independent verification. Organizations should avoid letting AI replace human mentorship, as human feedback retained a consistently positive link to performance. Finally, workflows should require users to document how they test, modify, or reject AI recommendations rather than accepting them on faith.

Host: Calibrated reliance instead of blanket confidence. Thank you, Alex, and thank you to our listeners for tuning in to A.I.S. Insights — powered by Living Knowledge.