Personalized LiveOps Offers Improve LTV 52

Improving LTV with Personalized LiveOps Offers
Approaches & Hill Climb Racing 2 Case Study
Ivan Trancik & Jakub Marek
Warsaw 17-18 June 2019

Warsaw | 17-18 June 2019
Jakub Marek
Head of Big Data & Monetization
jakub.marek@superscale.com
Ivan Trancik
CEO & Founder
ivan.trancik@superscale.com

“We’re forming growth partnerships with world’s top developers
to scale their games to maximum potential.”

● 60+ World Class Experts In-House
○ (Self-)Publishing Infrastructure
i. Business Intelligence
ii. Analytics & Data Science
○ Monetization
i. LiveOps Optimization
ii. Game & Monetization Design
○ User Acquisition
i. Creatives
ii. UA Campaign Management
iii. ASO
● Founded in 2016
● Bratislava, London, Berlin, Helsinki &
Prague oﬃces
First Light Games
Our partners
Games

LiveOps Optimization
Motivation

How can we improve monetization in games?
Huge topic - lots of angles
● Core Game design
● Meta game - retention systems
● Better onboarding
● Economy Balancing
● New content pipeline
...
● LiveOps Oﬀers - focus of this talk

Each player gets either the same price and content in an oﬀer or gets to
choose from 3-5 similar oﬀers.
There is no consideration of player’s:
● payment potential
● spending habits
● in-game behavior
This results in lower conversion
& lower satisfaction of players
Offers in most games are not maximizing revenue.
= missed opportunity => lower revenue.

The Question
How can we improve the monetization through special oﬀers
● using already existing content in the game?
● without impacting existing monetization systems?
=> by delivering a relevant content at a relevant time for a relevant price
How to actually do it?
● utilize segmentation - targeting players on a personal level

Approach: better monetization of existing game content through
personalised pricing, content and delivery
+52%
LTV Uplift from baseline
Sneak peek: results from Hill Climb Racing 2

LiveOps Optimization
Approach

Base model for LiveOps offers
STP Model
Well describes the process in
these questions:
● WHO?
● WHAT?
● HOW MUCH?
● HOW OFTEN?

Segmentation - Naive approach
Simple rule-based
segmentation based on LTV
Advantages:
● Easy to understand
● Most widely used
● Enables basic targeting

Recency, Frequency and Monetary value
● Directly models purchasing behavior
● Great for predicting purchase eﬀectiveness
● Easy to understand and apply in content
creation process
● Basis for targeted oﬀers
● Still no answer to WHAT?
RFM segmentation
Segmentation - Business Analytics Approach (Purchasing Behavior)

Segmentation - Business Analytics Approach (Purchasing Behavior)
Example application
Recency
Freq. $ 1 2 3
1 1 $15 $17 $14
1 2 $42 $41 $42
1 3 $169 $122 $143
2 1 $53 $54 $53
2 2 $140 $143 $145
2 3 $477 $425 $437
3 1 $192 $255 $274
3 2 $644 $708 $911
3 3 $2,288 $2,369 $3,109
Recency of last order
321
1. Group users into equally
sized buckets by
180_days_ago 90_days_ago 7_days_ago
Frequency of orders
1 2 3
1_Order 3_Orders 5_Orders 7+_Orders
Monetary value (average)
2. Further group users with each
possible combination of RFM
3. Find the Average Past Revenue-
per user in each of the buckets
Frequency
Recency
$
21
$0.00 $10.00 $20.00 $30.00 $40.00
3

Segmentation - Business Analytics Approach (Gameplay Interaction)
Gameplay Interaction Segmentation
Players in the game are not the same -
● They unlock different content
● Progress to different points in the story
● Different split between game modes
● Different spending habits

Best results = Combining both segmentation approaches
Purchasing Behavior Segmentation Gameplay Interaction Segmentation

Case Study: Hill Climb Racing 2
Game intro

Case study: Hill Climb Racing 2
● Fingersoft
● Released Q4 2016
● iOS & Android
● Multiplayer racing game
● 150+ million players
.. and how we delivered +52%
true LTV uplift.

Case study: Hill Climb Racing 2 - core game

Case study: Hill Climb Racing 2 - car collection & upgrading

Case study: Hill Climb Racing 2 - gacha (tuning parts)

Case Study: Hill Climb Racing 2
Overview

Pre-personalization monetization in HCR2
● direct gem pack purchases
● rank-based offers (tied to progression)
● special offers (offers with skin and good value, rotating in shop)
● seasonal offers (Halloween, Christmas, Back to school ...)

Goals
1. Increase revenue through personalization
2. Deliver to players a relevant content at a relevant time for a relevant price
3. Do not harm user retention
4. Minimize “cannibalization” of future proﬁts

What are we actually aiming for?
Minimize
additional value
(value multiplier)
Increase revenue
per user
Price
Content distribution
Value
Availability
Oﬀer sets and their
sequence
Visual aspects
Optimization
=> The value is in the personalization!

Business analytics
(BQ + Periscope + Python/R)
BI analyses
(monetization and gameplay
data)
Machine learning models
(probability modeling +
XGBoost + others)
Exporting segments
(offer config + user ID
pairings in BigQuery)
Analytics server LiveOps server Game client
Downloading offer config +
user ID from analytics
server
Distributing correct offer
configs to users
Displaying personalized
popup offers
Sending analytics data to
BigQuery
(popup + IAP data)
Evaluation of results
Automatic setup of popup
offer definitions
Defining target payer
segments & offer contents
LiveOps Optimization - Infrastructure

LiveOps Optimization - Modules
Players profile
(custom pipeline)
Vehicles progression
(mining or more diverse, frequency,
recency ..)
Models Offers sequence
method/models
Pricing model maximizes
revenue per impression
● Adjust conversion per price point
● Weights impression and purchases
over time
● Calculates revenue coefficient for
each player
● Adapts over time
● Automatic upselling
Predefined
sequence
Based on player’s
preference
(best converting sequence of
offers based on particular player
preference)
Behavioral preference model
● Takes into account all behavioral
data which defines future behavioral
● Creates set of personalized features
to choose from to increase variability
of offers
Upgrading preferences
(vehicle, amount, recency.. )
Tuning parts usage
(frequency,amount coins and scraps,
vehicle ..)
Purchasing behavior
(products distribution, price,frequence,
recency, amount.. )
Currency spending
(both soft and hard currency)
Rank progression
(rank, seasonal rank, teams)
Automatic value definition
model
● Automatic value definition (needed
for deeper personalization)
Modes preference
(challenges, adventures, events ..)
A/B testing tools
Behavioral based
(vehicle chests distribution,
amount of hard currency ..)
Monetary based
(waterfall, upselling ..)
Support tools
Automatic evaluation of
iterations
(ARPU, ARPPU, conversion, revenue,
impressions ..)
Subsegment definition
● Resources
(hard and soft currencies)
● Visuals
(backgrounds, titles,
colors, value multipliers)
User interface
based
(different value multiplier,
discount..)
Automatic offer
generation

Progression defines needs of players
Players progress through various flows - for example:
Step 1
Vehicle purchase
with soft currency
Vehicle upgrading until all
soft currency is depleted
Step 2
Step 3
Joining event and seeing what
vehicles winners are playing
Step 4
Opening race chests
and seeing rare card
Purchase
Player purchases bundle with
new vehicle from event, skin
and additional vehicle chest
Can we find these patterns and learn from our existing data?

Top challenges when designing offers
1. Understanding player behavior
○ What are player’s needs/preferences at the moment? What about their payment potential?
2. Creation of offers
○ How to keep things fresh and rotate through similar offers over and over again?
3. Evaluation, AB testing and continuous improvement
○ How to work towards long-term success?
4. Avoiding “cannibalisation” of future profit
○ How much additional value players should get?

First personalization model - rule based
RFM
(recency + LTV + avg. purchase)
Behavioral segmentation
(using only 2 major features - most
played & upgraded vehicle)
In ﬁrst iterations we used data to understand player behavior, distributions and created rule
based segments which led to 40% improvement over the baseline during oﬀer weekends.

First machine learning implementation
Monetary segmentation
(Modiﬁed RFM + highest previous
purchase)
Behavioral segmentation
(Agglomerative hierarchical clustering)
Prediction of chest type
(Random forest - feature importance)
Understanding monetary possibilities
of players
Understanding players preferences
in soft currency spending
Additional personalization of
chest content
Introduction of machine learning models led to even higher improvement at 108% improvement over
the baseline during oﬀer weekends.

Current data science model (+52% true LTV uplift)
Content deﬁnition based on behavior in
a game
Extreme gradient boosting (XGBoost)
Combination of soft and hard currency spending
with upgrading behavior, tuning behavior and
purchasing behavior
RFM
BI method
Probability modelling
Price conversion modelling with adaptation and
revenue maximization function
Understanding payment potential
through monetary parameters of
player
Understand which price points and sequence
of price points maximizes revenue from each
player individually

Takeaways (segmentation)
1. LTV & frequency of purchasing are more important than recency
2. Setting price of oﬀer based on player’s previous highest purchase is a good starting
point
3. Finding a way how to oﬀer player content he uses/interacts with the most without
breaking your existing systems is essential (and should be your starting point as well)

Elements of offers to optimize
Amount of
resources
Additional value
Oﬀer price
Availability
Type of chests
Visuals & copy

Takeaways (creation of offers)
1. Changing the “theme” of offer (gacha focus, currency focus, variety of offered items) to keep offers
from feeling stale (on pictures are offers from the same iteration)
2. Varying the value multiplier (while overall high value multiplier introduces cannibalization there is a
segment of players which reacts only to high-X-value offers)

Iterative improvement of performance - AB testing
AB test categories
● Pricing
○ New $80 price point introduced (top payers LTV +36%)
● Additional value multiplier optimization
○ Better personalization + lower value = kept good performance
● Segment deﬁnition
○ Waterfall model testing (moving players between price points)
○ Adaptation of personalization to unique play-styles (e.g. current vehicle or
game mode preference)

Iterative improvement of performance - ad hoc analyses
Continuous improvement that does not negatively affect other aspects of the game.
Monitoring cannibalization
development over time on different
level of detail
(segment/subsegment/offer etc.)
Monitoring community
discussions on different platforms
(e.g. Discord)
Monitoring performance over time,
comparing iterations of offers with
different content distribution per
segment/subsegment

Takeaways (evaluation & continuous improvement)
1. Ask the right questions from data
if the answer doesn’t change what you will do ask a different question
2. AB tests are expensive - test for major things first
new segmentation method is more important than testing new background
3. Your community is probably active somewhere - try to find those players and see
how they perceive offers they are getting

Result after 33 iterations
+41%
Total revenue growth
But how much have we impacted the baseline?
Is there some cannibalization?

Evaluation - True LTV uplift calculation
● True LTV uplift = LTV uplift after consideration (and deduction) of baseline cannibalization
→ For the correct calculation of the LTV uplift we need to establish long-term control
group(s) (i.e. to run a long-term AB test)
● Preceding every iteration we split new payers 90/10
=> 90% of the new payers start receiving personalized offers
=> 10% will never receive any personalized offer, but they will still be receiving all other offers
(seasonal, rank-up offers etc.)
● We do the same for non-payers
before each iteration we find all new players who joined the game and split them 90/10 as well

Evaluation - Payers - 49% true LTV uplift
LTV calculation
LTV = cumulative revenue
of a group / cumulative
number of distinct players
+ 49% true
LTV UPLIFT
21% Baseline
cannibalisation
(payers w/o oﬀers out
of control revenue)
+84%
LTV uplift

Evaluation - Non-payers (new players) - 58% true LTV uplift
+ 58% true LTV UPLIFT
+ 25% baseline LTV over time
Hypothesis
1. Personalized oﬀers increase
engagement of players over time
2. Coins spending analysis supports this,
overall upgrading and tuning of vehicles
increased for players who are purchasing
personalized oﬀers
LTV calculation
LTV = cumulative revenue of a group /
cumulative number of distinct players

Takeaways (evaluation & cannibalization)
1. Cannibalization is not something you can ignore
test it through proper AB testing
2. Offering hard currency in offers can introduce huge cannibalization of the existing
monetization systems
proceed carefully or drop all together
3. Making players happy with their first purchase might cause zero cannibalization and
even increase LTV further

Thank you for attention.
Questions?

Personalized LiveOps Offers Improve LTV 52

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Personalized LiveOps Offers Improve LTV 52

Similar to Personalized LiveOps Offers Improve LTV 52 (20)

More from GameCamp

More from GameCamp (20)

Personalized LiveOps Offers Improve LTV 52