How does the level of AI writing assistance affect whether people engage with or ignore suggestions?

Host: Welcome to A.I.S. Insights, powered by Living Knowledge. I'm Anna Ivy Summers.

Expert: And I'm Alex Ian Sutherland. Today, we're looking at a recent study titled "When Less is Worse Than More: AI Assistance Intensity and the Disengagement Zone in Writing," authored by Dominik Siemon, Triparna de Vreede, and Gert-Jan de Vreede, presented at the International Conference on Information Systems.

Host: This study examines how different levels of AI writing assistance shape user behavior and engagement modes in real-world settings. Alex, generative AI tools are everywhere now, but how does delegating writing tasks affect how people actually work?

Expert: That is the central question. Generative tools speed up drafting, but writing capability develops through revision and effortful struggle. Prior research often assumes that user interaction can be captured by a single aggregate acceptance rate. However, that metric obscures whether someone is actively evaluating suggestions, copying the entire text without thinking, or ignoring the system altogether.

Host: So looking only at overall acceptance misses the actual cognitive process. How did the authors test this dynamic in practice?

Expert: They conducted a randomized between-subjects field experiment embedded directly in a publicly accessible online writing platform. Sixty-six participants were randomly assigned to one of three assistance conditions without being told their assignment: light assistance, which only fixed grammar and spelling; balanced assistance, which added sentence clarity and word choice; and deep assistance, which offered structural rewriting. The platform logged interaction events directly, capturing whether users applied changes, copied wholesale, or ignored recommendations.

Host: What did the interaction logs reveal about how writers responded to these three tiers?

Expert: Looking at the 36 sessions where AI suggestions were generated, the authors found a non-linear pattern. Per-change acceptance was highest under light assistance at roughly 45 percent, intermediate under deep assistance at around 20 percent, and lowest under balanced assistance at zero percent.

Host: Zero percent in the balanced condition? That sounds surprising. Why would moderate help produce less engagement than deep rewriting?

Expert: The authors propose a three-zone framework to explain this. In the light condition, corrections preserve the writer's voice, so users evaluate and accept changes. In deep assistance, the rewriting is extensive, offering a major polish exchange that some users decide to adopt. But in the balanced condition, the AI alters phrasing and style—touching on the writer's voice—without providing enough perceived transformation to justify the loss of ownership. Out of 17 sessions in that balanced condition, 15 fell into the ignore mode. The authors call this the "displacement-without-exchange zone."

Host: It's important to note, though, that this is an ongoing study with preliminary data from a small sample, so these findings are directional. The engagement mode distribution across conditions had a p-value of point zero seven one, which is marginally significant.

Expert: Exactly. But it provides an empirical signal that users respond differently based on how much the tool intervenes. Another interesting finding was that writer confidence measured at onboarding significantly predicted whether a user returned to the platform for future sessions.

Host: For business leaders and product teams integrating AI into workflows, what are the primary takeaways?

Expert: First, more automation is not a simple linear dial. Designing an assistant that offers moderate, middle-of-the-road edits might actually trigger user disengagement if it interferes with tone without delivering transformative value. Second, organizations evaluating AI tools should look beyond simple adoption or acceptance percentages. Tracking whether employees actively evaluate suggestions versus passively copying or abandoning them can reveal whether tools support skill retention or bypass critical thinking.

Host: Matching the level of assistance to the user's task and expertise appears critical for sustained adoption and capability development. Thank you, Alex, for breaking down this study.

Expert: My pleasure, Anna.

Host: And thank you for tuning in to A.I.S. Insights, powered by Living Knowledge. Join us next time as we explore the practical realities of emerging technology.