This document summarizes the steps an organization took to increase conversion rates on its paid content from less than 0.5% to a target of over 5% by implementing machine learning and improving its culture. It began with a classic metered paywall with low conversion (Phase 1). It then developed a flexible rules engine, improving conversion to 1.2% (Phase 2). Next, it iterated on this rules engine, raising conversion to 2.5% (Phase 3). It then introduced propensity scoring with machine learning to further improve targeting (Phase 4). Its final goal is an "OmniGate" system to optimize the user experience across the entire lifecycle (Phase 5). Key to its success has been establishing

1. Increasing paid conversion 5-fold
with machine learning and culture
Steven Neubauer, Managing Director NZZ AG
@neubauersteven
Increasing paid conversion 5-fold
with machine learning and culture
Steven Neubauer, Managing Director NZZ AG
@neubauersteven

3. Most trusted media brand in Switzerland
150 m CHF in revenues
>150.000 paying subscribers

4. REACH ENGAGEMENT
anonymous lead
INMA Media Subscriptions Summit, April 19, 2018
SALES INCREASE
identified lead
REG-GATE PAY-GATE

5. REACH ENGAGEMENT
anonymous lead
INMA Media Subscriptions Summit, April 19, 2018
SALES INCREASE
identified lead
REG-GATE PAY-GATE
Conversion
on PayGate
x5 in last
3 years
Personalized
content
recommend-
ations
>10.000
registrations
per month

6. PHASE 1 (2012 to 2014):
Classic Metered Paywall
Conversion
rate
<0.5%

7. PHASE 2 (2014 to mid 2017):
Developing flexible rules engine
Conversion
rate
1.2%Dimension
Reg-
Prompt
Order-
Prompt
Landing
page
# of articles
Reading behavior
Prompt ON/OFF
Call-to-action
Format
Personal greeting
Time of day
Placement of prompt
Offering

8. EXAMPLE:
Personal greeting
Conversion +25%

9. PHASE 3 (2nd half 2017):
Dynamic Paygate v0.9 – Improving rules engine
Conversion
rate
2.5%
Iterate to improve rules engine
Hypotheses-based,
data-informed A/B-tests
à Dozens landing page
templates
à Random distribution
for 1 month
REGISTRATION
200 CHF
20 CHF
10 CHF ORDER
Pattern recognition
via machine learning
Tagging registered
users
Patterns driving
conversion

10. EXAMPLE:
Pattern recognition for pricing preferences

11. EXAMPLE:
Pattern recognition for call-to-action

12. EXAMPLE:
Differentiating the flow
Inactive, registered
users
Email: Read the rest
of the month for free
# of articles
read in previous
month
Individualized
subscription
prompt after ...
5
articles
8
articles
11
articles
13
articles
≤5 6-8 9-11 ≥12≤1
No
prompt

13. Phase 4 (2018):
Dynamic Paygate v1.0 – Propensity Scoring Conversion
rate
target
>2.5%
Propensity
score
top 20%
No
Use standard
rule set
A/B-test
Yes
Use standard
rule set
Use specialized rule set,
e.g., directly show
individualized order
prompt with highest
conversion probability

14. EXAMPLE:
Propensity scoring with machine learning
Propensity
score
à Time since registration
à Time since last visit
à # devices used
à # newsletters
à # active days
à ...
Random forest

15. Success criteria
§ Central, scalable platforms for product and
marketing automation
§ Unified data warehouse
§ Data and data science resides in marketing
§ Culture of experimentation and continuous
improvement
§ …

16. Organizational experiments
Self-governed, agile Digital Conversion Team
§ Purpose
§ Autonomy
§ Mastery

17. What did we do wrong?
§ Not enough focus on core products
§ Acceleration trap
§ We did not fail fast enough
§ No common understanding
of agility and no agile-ready
technology architecture
§ …

18. Phase 5 (2019):
«OmniGate»
Anonymous user
ORDER
home-
page
show
reg-gate
classify
Engagement?
action action
Engagement?
Send
personalized
push message
send
offer via
email
action ?
Paying subscriber
RETEN-
TION
observe
Engagement?
action
propensity
to churn?
email personalized
reading
recommendations
courtesy
call
action
ORDER
Conversion
rate
target
>5%

19. thank you