Key question New
AI-processed · Author-approved
Can completed coursework still prove independent competence when students have access to generative AI tools?
Loads Spotify's player; Spotify may set cookies. This device remembers your choice.
Key findings
- In one of the programming courses, 45% of students earning top marks on weekly coursework scored zero on a monitored final exam, showing a weak correlation between coursework and exam performance (r = 0.18).
- Adding exam monitoring to an identical programming test in another higher education institution resulted in top grades falling significantly from 73% to 24%.
- Source-code comparisons showed that student submissions exhibited code inflation and premature programming constructs not yet taught, with AI-stylistic markers detected in approximately 18% of student-authored code.
- Survey results revealed that international and domestic student cohorts reported differing modes of AI use and perceived utility, reflecting varying social and institutional pressures.
- Banning AI via conduct rules proved insufficient to preserve assessment validity, highlighting the need to shift toward supervised exams and tasks that require visible cognitive effort.
What it means for you
CIO / IT Executive
Monitor digital exam environments because unproctored take-home assignments cannot be trusted anymore during the age of AI since it is easy to skip them.
IT Manager
Deploy secure, proctored digital testing platforms across institution-wide coursework to ensure assessment integrity.
Business Strategist
Reallocate institutional resources away from futile AI-ban enforcement toward scalable, supervised assessment infrastructure.
Researcher
Start evaluating how students perceive their own skills when using AI, and design new learning pathways that accept AI usage while ensuring students still learn.
Policymaker
Prohibiting AI tools is impossible, as we cannot monitor what students are doing at home and what kind of assignments they return, and thus, we should think how to make more supervised assessments possible and scalable as just plain oral exams in large student groups are not a feasible method.
Summary
Across three Finnish higher education institutions and over 1,000 students, the authors examine assessment grades, analyze 883 paired student and AI source-code submissions, and evaluate start-of-semester survey responses from 602 students.
Problem
Higher education assessment relies on the assumption that submitting a completed task demonstrates the underlying cognitive skills the task was designed to teach and evaluate. Generative AI undermines this link by enabling students to produce functional solutions without engaging in required problem-solving processes. Institutional policies banning AI fail to restore assessment validity because rules alone cannot change the practical capabilities the technology makes available.
Transcript
Expert: Hello, Anna. The study evaluates a fundamental premise of assessment: the assumption that submitting a functioning task demonstrates that the person understands the underlying concepts. Generative AI dissolves that connection. Today, someone can generate working software or reports without engaging in the cognitive steps that the task was designed to measure.
Host: Many organizations and universities responded to these tools by simply writing policies that ban generative AI. Did the study find that rules resolve the problem?
Expert: The data indicate that policy bans do not restore assessment validity. As the authors note, an administrative ban cannot remove the practical capability—or affordance—that the technology provides. When individuals face evaluation pressure or deadlines, rules alone do not prevent the use of accessible tools.
Host: Let's look at the methodology. How did the researchers gather their data?
Expert: They used a mixed-methods approach across three Finnish institutions. They examined grades and exam formats across cohorts of over one thousand students, analyzed eight hundred and eighty-three paired student and AI source-code submissions, and collected survey data from six hundred and two students at the start of the semester.
Host: And what did the performance metrics reveal once monitoring was introduced?
Expert: In one programming course, forty-five percent of students who earned top marks on unsupervised weekly coursework received a score of zero on a monitored final exam. The statistical correlation between weekly coursework scores and monitored exam performance was very weak, at r = 0.18.
Host: That is a stark separation between take-home output and monitored ability. Did other institutions show similar results?
Expert: Yes. In another institution using an identical programming test, adding exam monitoring resulted in top grades falling from seventy-three percent down to twenty-four percent. When examining the source code itself, the researchers observed code inflation and advanced programming constructs that had not been taught yet, identifying AI-stylistic markers in approximately eighteen percent of student-authored code.
Host: The study also gathered survey data. What did that tell us about student behavior?
Expert: The survey of six hundred and two students revealed that international and domestic cohorts reported different modes of AI use and perceived utility. These differences reflect distinct social, financial, and institutional pressures, showing that student engagement with AI is not uniform across a population.
Host: For business professionals, recruiters, and training managers, this has direct operational implications. How should leaders apply these findings?
Expert: The direct takeaway is that unmonitored take-home assessments can no longer be trusted as evidence of individual competence. Whether evaluating job applicants or measuring employee training, relying solely on completed deliverables creates a risk of selecting candidates who can prompt a tool but cannot independently troubleshoot, explain, or maintain systems under pressure.
Host: What should organizations do instead?
Expert: Organizations need to shift toward supervised evaluations or synchronous technical interviews where candidates demonstrate their problem-solving in real time. If generative AI tools are permitted during assessments, evaluation criteria must focus on the process—such as how an individual validates, debugs, and explains the output—rather than treating the final artifact as proof of skill.
Host: Verifying the reasoning process rather than assuming the deliverable speaks for itself. Alex, thank you for detailing these findings. And to our listeners, thank you for joining us on A.I.S. Insights — powered by Living Knowledge.
Generative AI, Assessment validity, Affordance theory, Introductory programming, Assessment governance, Higher education