How automation, AI, and dashboarding can help scale experimentation programs If you are running 10+ experiments monthly across multiple teams, you know how important efficiency is. Moreover, available resources may not scale uniformly, making automation increasingly necessary to streamline processes. The challenge intensifies with execution quality when coordinating programs across diverse teams. While having clear standards and disciplined workflows is beneficial, integrating automation, leveraging AI, and utilizing dashboards significantly enhances any experimentation program. This not only addresses scalability issues but also provides stakeholders with more time to focus on the core aspects of their work. This webinar will be your script to plan and build automation, leverage AI and effective dashboarding to scale experimentation programs. Sign up now!

1. 1
Q4-2023
GAP Analyses.
Mark de Winter
Mark de Winter – Clickvalue - 29-1-2024

2. 2
COMPANY’S
VELOCITY
TARGETS
CRO
RESOURCES

3. 3
COMPANY’S
VELOCITY
TARGETS
CRO
RESOURCES
Problems when scaling
experimentation programs:
o Resources don’t grow at same pace as
(change) ambitions.
o Quality becomes an issue >10
experiments / month over multiple
teams.
o Cumbersome tasks are boring, and
demotivating.

4. 4
About ClickValue.

5. 5
Concepting & creation
Data & Analytics
Conversion
optimization
Online marketing
B2B market
Online marketing
B2C market
E-mail marketing &
marketing automation
24
CXO Experts (incl.
developers)
1000
AB tests / year
+15
Years of experience in
CXO
50
Countries

6. 6
Our Mission:
“Growing our clients’ business by supporting them in
making better decisions on their customers’
experiences”

7. 7
Our Clients:

8. 8
Trends 2024.

9. 9
CXO Trends in 2024*
(UX) Research
✓ Sustainability
✓ Digital accessibility
✓ Customer centricity
✓ Predictive analysis
✓ Centralized insights
Solution Design
✓ Immersive experiences
✓ Mobile first CX
✓ Optimizing voice
interactions
✓ VR / AR
Experimentation
✓ Automation
✓ Exp. data quality &
Dashboarding
✓ Hyper personalization
PM / Strategy
✓ Scaling of experimentation
in organizations
✓ Leveraging AI
✓ Data privacy & trust
✓ Omnichannel harmony
*Sources: Contentsquare / CMSWire / + Clickvalue benchmarks

10. 10

11. 11
Step 1: A structured process.

12. 12
“If you don’t have a process, you don’t know what you’re doing”
E. Deming

13. 13
Step 1: Start with a structured process
1) Ad-hoc experimentation 2) Dedicated experimentation 3) Advanced experimentation 4) Embedded experimentation
Maturity state
Single touchpoint Multiple touchpoints Full customer journey All (marketing & product related)
Experimentation scope
1 person Small team(s) Multiple organized teams Embedded (no CXO team)
Teams
Basic Structured process Advanced / automated Democratized
Process
1: Start structuring the
program

14. 14
Step 1: Start with a structured process
Ensure a solid workflow

15. 15
Step 1: Start with a structured process
Templates Subtasks Sprint management

16. 16
Step 2: Start automizing standardized
tasks
1) Ad-hoc experimentation 2) Dedicated experimentation 3) Advanced experimentation 4) Embedded experimentation
Maturity state
Single touchpoint Multiple touchpoints Full customer journey All (marketing & product related)
Experimentation scope
1 person Small team(s) Multiple organized teams Embedded (no CXO team)
Teams
Basic Structured process Advanced / automated Democratized
Process
2: Take your first
automation steps

17. 17
Automated Program Dashboarding:
Filter out all results
and experiment data
Quickly find all
your reports
based on
advanced
filters
View results from whole
program or specific team(s)
Succesdata
Compare
success ratios
to understand
what works
and what not

18. AI to increase efficiency
Generate quick ideas or substantiate
existing ideas with generative
content or create your own
database for trend & meta-analyses
1. AI Ideation
Generative AI can help create
assumptions on how to interpret
data and create hypotheses.
2. AI enriched
opportunities
AI can help with inspiration and quickly
generate multiple versions (viewports /
devices) of existing designs
3. Faster design
AI can help to create assumptions on
how the data can be interpreted in user
behavior and attitude to generate follow
up ideas.
4. Rich validation

19. 19
19
Example:
Help with
creating
assumptions
in ChatGPT
https://chat.openai.com/g/g-9CEOma1P1-cx-cognitive-insight
Prompt with: “What
can explain the
effect…”

20. 20
More
ChatGPT
examples:
20

21. 21
Step 3: Automation by default
1) Ad-hoc experimentation 2) Dedicated experimentation 3) Advanced experimentation 4) Embedded experimentation
Maturity state
Single touchpoint Multiple touchpoints Full customer journey All (marketing & product related)
Experimentation scope
1 person Small team(s) Multiple organized teams Embedded (no CXO team)
Teams
Basic Structured process Advanced / automated Democratized
Process
3: Automation at the
core of the program

22. 22
Experiment data management & monitoring:
Basic decision
support on
what to do.
Only show
the
experiments
you need
Splitwatch+ shows all
experiments and its relevant
data at one glance
See all
running
experiments
and select KPI
to call

23. 23
Experiment database:
Summary of results +
option to download
full report
Build custom filtering
& dashboarding for
specific research topics
Multiple variables available
to easily search through it
A database
with all
experiments

24. 24
Experiment database:
Job
Completion
Surveys Funnel
analysis
Return
rate
analysis
Customer
Journey
Mapping
A/B test
analysis
Landing
page traffic
analysis
Refined
opportunity
Moderated
usability
study
Expert
review
Competitor
scan
Prototype
test results
Funnel
analysis
Exit
survey
results
Preference
test results
1: One place for the
entire organisation to
log insights
2. A uniform way to
add new insights using
a clearly defined data
model & forms
3: The ability to easily find
and combine insights.

25. 25
Team training:
Individual training
tracks: Analyst (Asana)
Knowledge base in
Document360

26. 26
26
Example: AI
customer
database

27. 27
CXO Ecosystem:
Research
inspiration
Better predictions
due to prior data
& insights
Design
inspiration
Automated Program
Output overview
Splitwatch+
automated data &
monitoring tool
Automated
reporting
Structured
Opportunity’s,
Areas, Solutions
and their
relations (OST)
Faster “meta-
analyses”
Experiment repository stores all
important experiment data

28. 28
Summary
1) Ad-hoc experimentation 2) Dedicated experimentation 3) Advanced experimentation 4) Embedded experimentation
Maturity state
Single touchpoint Multiple touchpoints Full customer journey All (marketing & product related)
Experimentation scope
1 person Small team(s) Multiple organized teams Embedded (no CXO team)
Teams
Basic Structured process Advanced / automated Democratized
Process
1: Start structuring the
program

29. 29
Summary
1) Ad-hoc experimentation 2) Dedicated experimentation 3) Advanced experimentation 4) Embedded experimentation
Maturity state
Single touchpoint Multiple touchpoints Full customer journey All (marketing & product related)
Experimentation scope
1 person Small team(s) Multiple organized teams Embedded (no CXO team)
Teams
Basic Structured process Advanced / automated Democratized
Process
2: Take your first
automation steps

30. 30
Summary
1) Ad-hoc experimentation 2) Dedicated experimentation 3) Advanced experimentation 4) Embedded experimentation
Maturity state
Single touchpoint Multiple touchpoints Full customer journey All (marketing & product related)
Experimentation scope
1 person Small team(s) Multiple organized teams Embedded (no CXO team)
Teams
Basic Structured process Advanced / automated Democratized
Process
3: Automation at the
core of the program

31. 31
Benefits of automation in experimentation
√ Scaling will become easier and less resource heavy
√ Less quality loss due to automated data-prep and decision support
√ Better, clearer overview of all CXO output, results, impact and ROI
√ Save time for deeper analyses, designs, interpretations and follow ups.
√ More fun! As repetitive tasks are mostly automated.
But
➢ Beware of rubbish in – rubbish out: Prioritize clear process standards
➢ Standardisation v.s. customization
➢ AI isn’t trustworthy (yet)

32. 32
COMPANY’S
VELOCITY
TARGETS
CRO
RESOURCES

33. 33

34. 34
Thanks!
mark@clickvalue.nl