Sustained Impact of Agentic Personalisation in Marketing: A Longitudinal Case Study
An 11-month real deployment shows AI agents can maintain marketing engagement gains without human oversight — but humans still drive the biggest wins.
The Thesis
Most marketing automation today relies on rules a human wrote last quarter. This paper tests what happens when you hand the wheel to an AI agent — and then walk away. The result, drawn from a real consumer app over nearly a year, is reassuring but not triumphant: autonomous agents preserved performance gains, but human-curated phases produced the steepest improvements. The practical implication is a 'train then release' workflow, where marketers invest intensively in strategy and content, then let agents maintain those gains at scale. The catch is that this is a single-company case study, which limits how broadly the findings apply.
Catalyst
Large language models and agent orchestration frameworks have matured enough to be embedded inside commercial CRM (Customer Relationship Management) pipelines without requiring a team of ML engineers to babysit them. At the same time, rising customer acquisition costs are forcing marketers to extract more value from existing users, making scalable personalisation economically urgent in a way it wasn't three years ago.
What's New
Traditional CRM personalisation relied on static segmentation — a marketer draws up audience buckets and message templates, and the system applies them mechanically until someone updates them. Earlier adaptive systems, such as multi-armed bandit algorithms and rule-based recommendation engines, could adjust which message to send but still required humans to continuously refresh the content library. This paper tests a fully agentic layer that selects, sequences, and recombines content autonomously from a fixed library, and measures whether the lift holds across an extended passive period — something prior published work had not demonstrated in a longitudinal, production setting.
The Counter
This is a single case study from a single unnamed consumer app. We have no way to know whether the product category, user base, or content library was unusually well-suited to this approach. The 'passive phase' result — that agents sustained lift — is interesting, but the paper does not report whether lift slowly decayed across those months or held flat, which matters enormously for real-world deployment decisions. The fixed content library used in the passive phase is a significant constraint: in practice, content goes stale, seasonal context shifts, and competitors respond. An agent working from last season's creative is arguably just a fancier rule engine. Finally, 'positive lift' relative to a pre-agent baseline is not the same as beating a well-resourced human team running continuously — and the paper explicitly shows humans outperform the autonomous phase when actively engaged.
Longs
- SALESFORCE (CRM) — core CRM vendor most exposed to agentic personalisation upgrade cycle
- BRZE (Braze) — customer engagement platform that competes directly in this workflow
- HUBS (HubSpot) — mid-market CRM with active AI feature investment
- ADBE (Adobe) — Experience Cloud sits squarely in AI-driven personalisation
- BOTZ (Global Robotics & AI ETF) — broad exposure to autonomous agent deployment
Shorts
- Legacy ESP (email service provider) vendors such as Mailchimp and Constant Contact — their value proposition rests on human-designed campaign workflows that agentic layers make redundant
- Marketing consultancies and CRM agencies — if agents sustain performance passively, the billable hours for ongoing campaign management shrink
- Rule-based A/B testing platforms — agentic personalisation replaces sequential experiment design with continuous autonomous optimisation
Enablers (Picks & Shovels)
- LangChain / LangGraph — open-source agent orchestration frameworks that make agentic CRM pipelines buildable without custom infrastructure
- Segment (Twilio) — customer data platform that feeds the audience signals agents need
- AWS Bedrock / Azure AI Studio — managed model hosting that lets CRM vendors embed LLM-based agents without owning model infrastructure
- Braze SDK — mobile engagement infrastructure that surfaces the push and in-app channels these agents optimise
Private Watchlist
- Movable Ink — AI-driven content personalisation for email and push
- Attentive — mobile marketing automation with AI optimisation layer
- Persado — AI-generated marketing language optimisation
- Simon Data — customer data platform with ML-based campaign orchestration
Resources
The Paper
In consumer applications, Customer Relationship Management (CRM) has traditionally relied on the manual optimisation of static, rule-based messaging strategies. While adaptive and autonomous learning systems offer the promise of scalable personalisation, it remains unclear to what extent ``human-in-the-loop'' oversight is required to sustain performance uplift over time. This paper presents a longitudinal case study analysing a real-world consumer application that leverages agentic infrastructure to personalise marketing messaging for a large-scale user base over an 11-month period. We compare two distinct periods: an active phase where marketers directly curated content, audiences, and strategies -- followed immediately by a passive phase where agents operated autonomously from a fixed library of components. Our results demonstrate that whilst active human management generates the highest relative lift in engagement metrics, the autonomous agents successfully sustained a positive lift during the passive period. These findings suggest a symbiotic model where human intervention drives strategic initialisation and discovery, yet autonomous agents can ensure the scalable retention and preservation of performance gains.