Early Churn Prediction and Personalised Interventions in Top Eleven game
•Als PPTX, PDF herunterladen•
3 gefällt mir•1,858 views
Churn alone, it is a widely known term in many industries, including banking, telecommunications and gaming. When people think about churn, it is mostly about veteran or late churn, however, nobody ever speaks about early churn. In gaming, there can be many definitions of early churn, and they vary based on the game, but some general definition is that early churn is when a user comes into your game, plays the game for a limited short period of time, and then leaves the game, never to return again.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (13)
Ähnlich wie Early Churn Prediction and Personalised Interventions in Top Eleven game
Ähnlich wie Early Churn Prediction and Personalised Interventions in Top Eleven game (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Early Churn Prediction and Personalised Interventions in Top Eleven game
- 1. Early Churn Prediction and Personalised Interventions in TOP 11 Miloš Milošević, Nordeus
- 2. About me Data engineer at Nordeus. Main focus: Predictive machine learning pipelines; Their deployment and maintenance.
- 3. About Nordeus ● Award winning gaming company ● Offices in London, Dublin, San Francisco, Skopje and Belgrade ● Creators of Top Eleven
- 4. About TopEleven ● Our flagship product ● 110+ million registered users ● Global ● 5 years old
- 5. Agenda IDENTIFY - Predict early churners using machine learning TARGET - Construct a personalized message for every churner INTERVENE - Send messages via a scalable notification system
- 6. Early churn ● What is it? ● Why is its reduction important? ● Can we predict it and how?
- 7. When to engage the prediction algorithm?
- 8. We tried 3 things
- 9. Features
- 10. Models ● Best model - Gradient Boosting Trees ● Model we use - Logistic Regression ● Model to play with - Recurrent Neural Network
- 11. And what do we do then? ● Optimize backend ● Target churners ● Stay prediction
- 12. Targeting Messages need to be meaningful, otherwise...
- 13. What kind of notifications work well on what type of user? How to find the perfect match? Use the user’s first day activity to create meaningful personalized notifications. Targeting
- 15. Examples
- 16. Measuring results Split users into 3 groups: ● Control ● Baseline ● Test
- 17. Our results ● On average, baseline group achieved 30% better retention ● Test group achieved 40% better retention than baseline
- 18. Notification engine Specs: ● Modular design ● Scalable ● On time delivery ● Near real time message creation ● Logging system ● A/B test support
- 19. The power of Spark
- 20. Conclusion ● Everyone can do early churn prediction ● Target with care ● When in doubt, Apache Spark
Hinweis der Redaktion
- Hi everyone and thanks for coming. My name is Milos Milosevic, I am from Nordeus and today I’m gonna talk to you about early churn prediction and personalized interventions in TopEleven
- So, first off, a little bit about me. I am a data engineer at Nordeus and my main focus lies in predictive machine learning pipelines, their deployment and maintenance in a live production setting.
- Regarding Nordeus, we are an award winning gaming company, founded in 2010. We have offices in London, Dublin, San Francisco, Skoplje and Belgrade. Also, we are the creators of Top Eleven.
- In case you aren’t familiar with TopEleven, it is a social online cross platform free to play football manager game. When you register, you are given a football team, you can play around with your team tactics, train your players, sell and buy new ones and of course play matches. You have 3 competitions per season, league, champions league and cup, while the season itself lasts 28 days. TopEleven is our flagship product. It is 5 years old, and global, with over 110 million registered users from every country in the world playing it.
- So, I am here today to talk to you about how to solve the problem of early churn. In order to do that, we will discuss how to identify and predict early churners using machine learning models. Furthermore, we will see how to target the churners and construct personalized messages that maximize reactivation. And in the end, we will touch upon the underlying infrastructure needed to reach out to these users and actually send these messages.
- Okay, so, can you tell me, what is early churn? Churn alone, it is a widely known term in many industries, including banking, telecommunications and gaming. When people think about churn, it is mostly about veteran or late churn, however, nobody ever speaks about early churn. In gaming, there can be many definitions of early churn, and they vary based on the game, but some general definition is that, early churn is when a user comes into your game, plays the game for a limited short period of time, and then leaves the game, never to return again. If we reduce early churn, we increase retention, and by increasing retention we can increase all other metrics. Think about it like this, if a user is to stay longer in the game, this increases the chances that the user invites his friends to the game or spends some money in the game. Now, early churn is mainly addressed in a game by creating a good tutorial, or by optimizing the first few levels of the game. If you would want something about it further, normally you would need to create and developed a new in game feature. Depending on your game, and the magnitude of the feature, actual implementation of it can take ages. But, what if there’s something, that we, as analytics people, can do, to reduce early churn, without burdening the product team with new features to create. This was exactly the idea we at Nordeus had that sparked our effort towards firstly identifying early churners. So we sat down and thought about how can we create a model that predicts this. And this led us to interesting question that we haven’t thought of before, and that question is ...
- … when to engage the prediction algorithm? So, this is a very important question and depending on your definition of early churn, the answer can vary. But also, depending on your particular use case for early churn prediction you may want to engage the algorithm sooner or later. When we started out, we knew we wanted to do targeting, but we still couldn’t answer this question. So what do you do when you’re not sure about something? You test. We made 3 slightly different models, predicting after 3 days, 2 days and 1 day after user registration, and here are the results.