Key question New
AI-processed · Author-approved
How effective is generative AI in promotional messaging compared to human copywriters, and when does it work best?
Loads Spotify's player; Spotify may set cookies. This device remembers your choice.
Key findings
- Both generic and personalized AI-generated promotional messages significantly outperformed human-written copy and coupon-only controls in increasing purchases.
- AI-personalized messages increased daily spending by 14.3% and order quantities by 2.5%, remaining consistently effective across all contexts regardless of cognitive load.
- AI-generic messages increased daily spending by 10.4% overall and showed significantly stronger effects when customers ordered to a workplace or during working hours.
- Human-written generic copy showed no statistically significant improvement in spending or order volume compared to sending a coupon alone.
What it means for you
CIO / IT Executive
Approve and prioritize enterprise-wide deployment of generative AI tools integrated directly into the CRM and marketing automation platforms to scale automated promotional copy generation.
IT Manager
Configure the marketing platform's campaign scheduler to automatically trigger AI-generated generic promotional messages during standard business hours and to workplace delivery addresses.
Business Strategist
Reallocate promotional content budgets away from human copywriters entirely, shifting focus to deploy AI-personalized copy for high-value segments and AI-generic copy for B2B/workplace lunchtime campaigns.
Researcher
Design a follow-up field experiment testing the precise linguistic features and psychological triggers in AI-generated copy that enable it to maintain persuasion under high cognitive load.
Policymaker
Draft mandatory disclosure guidelines requiring digital food delivery platforms to clearly label when promotional copy and personalized discounts are generated by artificial intelligence.
Summary
This study evaluates the effectiveness of generative AI-generated promotional copy compared to human copywriters and coupons alone. Partnering with a restaurant chain on a major food delivery platform, the authors conducted a randomized field experiment examining actual purchasing behavior across 300 customers over two weeks.
Problem
While marketers rapidly adopt generative AI to produce promotional content, causal field evidence demonstrating whether AI copy actually drives purchasing behavior remains scarce. Additionally, little is known about how consumers' cognitive constraints, such as being at work or ordering during business hours, moderate the persuasive effectiveness of AI versus human messaging.
Transcript
Expert: Hello Anna. It is great to be here. This study evaluates how generative AI promotional messages influence actual purchasing decisions compared to copy produced by professional human copywriters and simple discount coupons.
Host: Marketing teams have rapidly adopted large language models to draft promotional content. What specific gap does this study address?
Expert: While adoption is widespread, causal field evidence demonstrating whether AI copy actually translates into higher sales in commercial settings has been limited. Furthermore, prior research has not established how contextual cognitive constraints—such as receiving messages while at work—moderate the persuasive impact of AI versus human messaging.
Host: To investigate this, the authors partnered with a restaurant chain operating on Meituan, a major food delivery platform in China. Alex, how was the experiment structured?
Expert: The researchers conducted a randomized field experiment across five restaurant locations during the launch of new menu items. Customers who placed orders were assigned to one of four conditions: a control group receiving only a discount coupon, a group receiving the coupon with human-written generic copy, a group receiving an AI-written generic message, and a group receiving an AI-personalized message tailored to the customer's prior order history. The AI copy was generated using DeepSeek-R1. The primary analysis tracked purchasing behavior among 300 confirmed message readers over a two-week period.
Host: Let us look at the findings. How did the different messages perform?
Expert: Both AI-generated formats significantly outperformed the human-written copy and the coupon-only baseline. AI-personalized messages delivered the strongest results, increasing daily spending by 14.3 percent and daily order quantities by 2.5 percent. AI-generic messages also showed substantial gains, increasing daily spending by 10.4 percent and order quantities by 1.9 percent.
Host: And how did the professional human copywriters compare to the baseline?
Expert: The human-written generic messages did not show a statistically significant difference in spending or order volume compared to sending the coupon alone.
Host: That is a striking comparison. The study also examined cognitive constraints, measured by whether customers ordered to a workplace or during business hours. What did that moderation analysis show?
Expert: The two AI conditions exhibited distinct behavioral patterns. AI-generic messages were significantly more effective in high cognitive load environments, generating a 16.5 percent increase in daily spending when delivered to workplace addresses. In contrast, AI-personalized messages maintained consistent effectiveness across both workplace and non-workplace settings, showing no significant variation by location or time of day.
Host: Drawing on the Elaboration Likelihood Model, how do the authors interpret these differences?
Expert: When cognitive resources are constrained, consumers tend to rely on heuristic processing. The clear, structured format of AI-generic copy appears well-suited to quick evaluation during busy workdays. Personalized AI copy, by incorporating specific past preferences, lowers processing friction directly, allowing it to succeed regardless of the consumer's available cognitive bandwidth.
Host: What are the key practical takeaways for digital platforms and retailers?
Expert: The findings suggest a clear segmentation framework. In high-load contexts, such as workday lunchtime promotions, cost-effective AI-generic messaging provides a strong return. When targeting consumers across diverse or unobserved contexts, deploying personalized AI messaging offers dependable, context-independent performance despite higher data and computational requirements.
Host: In summary, the study provides causal evidence that generative AI copy can improve promotional outcomes over traditional copy and discount-only baselines, with cognitive context serving as a practical guide for deployment. Thank you, Alex.
Expert: You are welcome, Anna.
Host: And thank you for listening to A.I.S. Insights — powered by Living Knowledge. Until next time.
Generative AI, Product Promotion, Cognitive Load, Field Experiment, Digital Marketing, Consumer Behavior