New
European Journal of Information Systems (2024) AI Processed Human Approved
Design principles for artificial intelligence-augmented decision making:An action design research study
This study investigates how organisations can effectively design and deploy artificial intelligence-augmented decision making (AIADM) systems. Using an action design research methodology within an online fashion retail company, the authors developed and evaluated an AIADM system to support decisions across customer segmentation, customer retention, and product portfolio redesign.
Problem
Traditional decision support systems rely on predefined rules and human agentic primacy, making existing design knowledge difficult to apply directly to adaptive AI systems. Organisations face significant challenges in adopting AIADM systems, including stochastic outputs, ambiguous decision authority, regulatory concerns, and resource constraints.
Outcome
- Formulate a strategic data roadmap aligning AIADM use cases with business model requirements, resources, and capabilities.
- Ensure synergy across input data, AI models, and visual outputs to demonstrate tangible business value.
- Implement ethical AI governance frameworks and auditing mechanisms to ensure compliance and build stakeholder trust.
- Maintain human involvement in design and execution to integrate domain expertise and retain decision accountability.
- Embrace an iterative design mindset focused on continuous learning, adaptation, and system improvement over time.
- Utilize open-source tools and external expert partnerships to mitigate talent and resource limitations.
- Ensure synergy across input data, AI models, and visual outputs to demonstrate tangible business value.
- Implement ethical AI governance frameworks and auditing mechanisms to ensure compliance and build stakeholder trust.
- Maintain human involvement in design and execution to integrate domain expertise and retain decision accountability.
- Embrace an iterative design mindset focused on continuous learning, adaptation, and system improvement over time.
- Utilize open-source tools and external expert partnerships to mitigate talent and resource limitations.
What it means for you
- CIO / IT Executive: On Monday morning, schedule a 30-minute kickoff meeting with your heads of data science, infrastructure, and business intelligence to review the company's current AIADM initiatives and identify one high-priority business model requirement for AIADM to address within the next quarter.
- IT Manager: On Monday morning, initiate a preliminary audit of existing data pipelines and AI model repositories, specifically noting any inconsistencies or gaps in documentation that hinder synergy between input data, AI models, and visual outputs.
- Business Strategist: On Monday morning, draft a concise one-page document outlining at least two specific AIADM use cases that directly align with the company's current strategic objectives, identifying the key data inputs and desired decision outcomes for each.
- Researcher: On Monday morning, review the internal documentation for the AIADM system currently in use, and identify one specific instance where human domain expertise could have significantly improved the AI's output or decision recommendation.
- Policymaker: On Monday morning, begin drafting a preliminary checklist of essential components for an ethical AI governance framework, focusing on elements relevant to data privacy, bias detection, and clear lines of decision accountability for AI-augmented decisions.
Transcript
Host: Welcome to A.I.S. Insights — powered by Living Knowledge. I'm your host, Anna Ivy Summers, and today we're diving into a crucial topic for modern business leaders: how to successfully build and deploy artificial intelligence to enhance executive decision-making. We're looking at a fascinating new study titled "Design principles for artificial intelligence-augmented decision making: An action design research study." Joining me to break it down is our lead analyst, Alex Ian Sutherland. Welcome, Alex!
Expert: Thanks, Anna. It's great to be here. This study is particularly exciting because it tackles one of the biggest hurdles companies face today: moving AI from theoretical hype into actual, day-to-day business value.
Host: Exactly. Now, to give our listeners some context, this study looks at how organizations can effectively design and deploy artificial intelligence-augmented decision making—or AIADM—systems. But Alex, before we jump into the solutions, why is this such a big problem in the first place? Don't companies already use traditional Decision Support Systems?
Expert: That's a great question, Anna. Traditional decision support systems relied on fixed, predefined rules. You input data, and the system gave you a deterministic, predictable result where human agency remained front and center. Modern AI is fundamentally different. It learns, adapts, and generates stochastic—meaning probabilistic—outputs.
Host: Which means the outcomes aren't always guaranteed or easy to explain!
Expert: Precisely. That creates major organizational friction. Leaders struggle with ambiguity around who is actually accountable for a decision—the human or the machine? On top of that, companies face talent shortages, regulatory risks, and high failure rates when trying to scale AI. Extrapolating old IT playbooks to adaptive AI systems just doesn't work.
Host: So how did the researchers in this study approach finding a solution?
Expert: They used a methodology called Action Design Research. Instead of sitting in an academic lab, the researchers embedded themselves directly within a real-world business—a digital fashion brand called TBô Clothing—over a period of 28 months. Together, they built, deployed, and tested AI models across three real use cases: customer segmentation, customer retention, and product portfolio redesign.
Host: That hands-on approach sounds incredibly practical. What were the main findings or design principles that emerged from this study?
Expert: The study distilled six core design principles. First, design for alignment. You must create an actionable strategic data roadmap that aligns specific, measurable AI use cases directly with your business model and existing resources.
Host: Right, so you're not just adopting AI because it's popular, but because it solves a defined business problem. What's second?
Expert: Second, design for synergy in inputs, models, and outputs. To prove tangible business value, you need to combine rich datasets and pair model accuracy with clear visual outputs, like explainable AI tools that show managers *why* a prediction was made.
Host: That leads right into the third and fourth points, doesn't it? How do human involvement and ethics fit in?
Expert: Exactly. The third principle emphasizes ethical AI governance—establishing auditing mechanisms and adhering to regulatory guidelines to build trust with customers. The fourth, and perhaps most critical principle, is designing for human involvement. The study strongly advocates for decision *augmentation* over full automation. You keep human domain experts in the loop so they can catch model errors, input tacit business knowledge, and retain ultimate decision accountability.
Host: I love that emphasis on keeping humans accountable. What about managing costs and technical challenges?
Expert: That brings us to the final two principles. Principle five is designing for continuous learning and adaptation. AI implementation isn't a one-time setup; it's an iterative cycle where models must be updated as data shifts. And principle six is leveraging open knowledge and external partnerships—utilizing open-source tools, pre-built libraries, and academic collaborations to overcome internal talent and resource limitations.
Host: That makes total sense. So Alex, looking at the big picture, why does this study matter so much for business professionals today?
Expert: It matters because many leaders treat AI deployment as a standard software upgrade, which leads to abandoned projects and wasted capital. This study provides a battle-tested blueprint. By focusing on human-AI collaboration, strategic alignment, and transparent governance, organizations can minimize risks and reliably turn complex data into profitable decisions.
Host: A clear, actionable guide for navigating the AI revolution. Alex, thank you so much for breaking down this study for us today!
Expert: My pleasure, Anna.
Host: And thank you to our listeners for tuning into A.I.S. Insights — powered by Living Knowledge. Be sure to subscribe for more deep dives into the technology shaping the future of business. Until next time!
Expert: Thanks, Anna. It's great to be here. This study is particularly exciting because it tackles one of the biggest hurdles companies face today: moving AI from theoretical hype into actual, day-to-day business value.
Host: Exactly. Now, to give our listeners some context, this study looks at how organizations can effectively design and deploy artificial intelligence-augmented decision making—or AIADM—systems. But Alex, before we jump into the solutions, why is this such a big problem in the first place? Don't companies already use traditional Decision Support Systems?
Expert: That's a great question, Anna. Traditional decision support systems relied on fixed, predefined rules. You input data, and the system gave you a deterministic, predictable result where human agency remained front and center. Modern AI is fundamentally different. It learns, adapts, and generates stochastic—meaning probabilistic—outputs.
Host: Which means the outcomes aren't always guaranteed or easy to explain!
Expert: Precisely. That creates major organizational friction. Leaders struggle with ambiguity around who is actually accountable for a decision—the human or the machine? On top of that, companies face talent shortages, regulatory risks, and high failure rates when trying to scale AI. Extrapolating old IT playbooks to adaptive AI systems just doesn't work.
Host: So how did the researchers in this study approach finding a solution?
Expert: They used a methodology called Action Design Research. Instead of sitting in an academic lab, the researchers embedded themselves directly within a real-world business—a digital fashion brand called TBô Clothing—over a period of 28 months. Together, they built, deployed, and tested AI models across three real use cases: customer segmentation, customer retention, and product portfolio redesign.
Host: That hands-on approach sounds incredibly practical. What were the main findings or design principles that emerged from this study?
Expert: The study distilled six core design principles. First, design for alignment. You must create an actionable strategic data roadmap that aligns specific, measurable AI use cases directly with your business model and existing resources.
Host: Right, so you're not just adopting AI because it's popular, but because it solves a defined business problem. What's second?
Expert: Second, design for synergy in inputs, models, and outputs. To prove tangible business value, you need to combine rich datasets and pair model accuracy with clear visual outputs, like explainable AI tools that show managers *why* a prediction was made.
Host: That leads right into the third and fourth points, doesn't it? How do human involvement and ethics fit in?
Expert: Exactly. The third principle emphasizes ethical AI governance—establishing auditing mechanisms and adhering to regulatory guidelines to build trust with customers. The fourth, and perhaps most critical principle, is designing for human involvement. The study strongly advocates for decision *augmentation* over full automation. You keep human domain experts in the loop so they can catch model errors, input tacit business knowledge, and retain ultimate decision accountability.
Host: I love that emphasis on keeping humans accountable. What about managing costs and technical challenges?
Expert: That brings us to the final two principles. Principle five is designing for continuous learning and adaptation. AI implementation isn't a one-time setup; it's an iterative cycle where models must be updated as data shifts. And principle six is leveraging open knowledge and external partnerships—utilizing open-source tools, pre-built libraries, and academic collaborations to overcome internal talent and resource limitations.
Host: That makes total sense. So Alex, looking at the big picture, why does this study matter so much for business professionals today?
Expert: It matters because many leaders treat AI deployment as a standard software upgrade, which leads to abandoned projects and wasted capital. This study provides a battle-tested blueprint. By focusing on human-AI collaboration, strategic alignment, and transparent governance, organizations can minimize risks and reliably turn complex data into profitable decisions.
Host: A clear, actionable guide for navigating the AI revolution. Alex, thank you so much for breaking down this study for us today!
Expert: My pleasure, Anna.
Host: And thank you to our listeners for tuning into A.I.S. Insights — powered by Living Knowledge. Be sure to subscribe for more deep dives into the technology shaping the future of business. Until next time!