Measuring the benefit effect for customers with Bayesian predictive modeling
•Als PPTX, PDF herunterladen•
1 gefällt mir•938 views
Strata 2015 London presentation http://conferences.oreilly.com/strata/big-data-conference-uk-2015/public/schedule/detail/39592 https://github.com/cojette/CompareImpact
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Andere mochten auch
Andere mochten auch (20)
Ähnlich wie Measuring the benefit effect for customers with Bayesian predictive modeling
Ähnlich wie Measuring the benefit effect for customers with Bayesian predictive modeling (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Measuring the benefit effect for customers with Bayesian predictive modeling
- 1. Measuring benefit effect for customers with bayesian prediction modeling Kwon, JeongMin (cojette@gmail.com) Strata + Hadoop World 2015, London
- 2. Offering
- 3. Offering • Popular way to promote products
- 5. Step 1: Customer Data Management Monitor customers’ actions Keeping track of customer data and information Defining goals
- 6. Step 2: Targeting Customer segmentation and selection with goals at step 1 Based on demographic information and log collections Statistical methods and data mining algorithms
- 7. Step 3: Offering Benefit (Campaign) • Delivering proper benefits to targeted customer groups • Methods: Promotion, Event, Advertisement and others • Measurement and prediction of campaign effects
- 8. How to measure and predict
- 9. How to compare offering effects
- 10. Multivariate Testing • Technique for testing a hypothesis with multiple variables • Issues for offering • Lack of long-term prediction • data, benefit limitations (Image from https://www.ownedit.com/features)
- 11. Bayesian Interpretation • Diachronic Interpretation • Probability of the hypotheses changes over time • Prior and posterior based on background information • Good for simulation, decision and prediction (Image from http://en.wikipedia.org/wiki/Bayes%27_theorem)
- 13. CausalImpact • Based on the paper [Inferring causal impact using Bayesian structural time-series models], Google, 2014 • CausalImpact Package in R • https://github.com/google/CausalImpact (Image from the paper)
- 14. CompareImpacts • Integration of bayesian time series prediction model with multivariate tests • For simple comparison of causal effects
- 15. Use Cases • Same offerings in various groups
- 16. Use Cases • Various offerings in a group
- 17. Basic Model • CausalImpact results
- 18. • Same offerings in three groups Use Case Results
- 19. • Offerings in a group with time differences Use Case Results
- 20. Office Hour: 15:25~16:05, Table A E-mail : cojette@gmail.com New Tool for Offering Comparison: Multivariate Test + Bayesian Time-Series Analysis
Hinweis der Redaktion
- Hello, everyone. I'm happy to give a presentation at Strata conference. Today I’d like to talk about "Measuring benefit effect for customers with bayesian prediction modeling" .
- The main keyword of this presentation is ‘Offering’. I don’t know whether you have ever heard about “ Offering” or not, // but I’m sure you have experienced it at least once. Nowadays there have been lots of 'offering benefit' events // in the mobile, online, and offline world.
- You may have apps in your smartphone // that give you free or discounted coupons. You could get personalized discount offers by e-mail, //and sometimes you may receive some paper coupons on the street. Technologies are being developed, // and the variety of offering is being increased, but the essence of offering is the same // - increasing whole customer value. Therefore, all these examples are in the range of offering. Of course, some of you may feel that offering is a kind of spam. Sometimes you may be annoyed by these offering events. But offering benefits is a well-known way // to inform customers about your products, // to differentiate your products from competitors // and to create value by meeting customers’ wider needs and deeper impressions //more than the competitors. In short, offering benefits has been a very classical but important // and widely-used strategy //for acquisition of new customers and churn management.
- The Offering Process is like this. Let’s suppose these colorful objects are customers, // and this gray square is a group,// for example, this is a whole customer group. Our strategy consists of three steps: // gathering of customer data , targeting relevant customers, and offering benefits.
- First step is the customer data management. In this step, we do not interact directly with the customers. We just monitor customers' actions and observe their needs, // and define goals //that we wish to achieve by interacting with these customers. We may keep track of customer data and information, // and set the dimensions and views for them.
- Second, the targeting step. In this step, we segment customers into groups // based on the purposes that we already set //in the previous step. Customer segmentation can be based on // demographic informations, their action log, payment log and other data. There are lots of methods for doing this // , mainly various statistical methods and algorithms // related to clustering and classification. Many of you must be familiar with algorithms // such as K-means, SVM, EM and others. These all algorithms can be used in the targeting step. Moreover, on the business side, // there have been lots of strategy improvements // through decades of research on //Customer Relationship Management. The customer segmentation has been the foundation of CRM for a long time.
- Final step is actually offering benefits, //also known as campaign. In this step,we try to deliver proper benefits to targeted customers // using promotions, events, advertisements and other methods. This is the most important and difficult step, // because 'offering' is completed // when the relevant targeted customers get the appropriate benefits // and they are satisfied. However, it is difficult to quantify and forecast // how much of the targeted customers will be satisfied // with the benefits they get.
- In this process, especially during the final step, // there is a huge problem of decision making. The most important and difficult question is this:// How do you measure and predict effects of benefits? Many decision makers suffer for various reasons, // such as the lack of effective methods // for measuring and predicting the effects of benefits, // insufficient customer data, the wide range of available benefits, // temporary availability of benefits due to business concerns and, //most importantly, the difficulty of predicting the customer reactions. Consequently, many just give up making decisions based on the data. Instead, they claim that they can only rely on their business instincts.
- Post hoc comparison of effects has even bigger problems. It is rare to host only a single campaign. Most companies would make various offerings, //refining their campaigns through trial and error //as they go along. For example, //we can refine groupings by dividing a group to smaller groups // and treat each subgroup with different offerings. However, in order to do this, //we need to make a precise comparison// between different groups, //which means that we need to control background variables. There are not that many techniques that easily allow this, //and they are usually very hard to use.
- We are beginning to see easier alternatives now. You may have heard of A/B test, A/A test, multi-armed bandit test and more- // in fact, most of you already have heard // great talks about them before this session. These are easily applicable and modifiable, //they are being widely used, //especially for functional and experience tests. However, there are several issues //when we try to use them for customer offering. For example, they lack of long term prediction methods; //there is only a limited scope of modifying the data //and comparing testing options quantitatively. They also need to be able to consider //the influence of external factors and treatments. And moreover, the patterns of peoples’ behaviors are hard to control. Therefore, other alternatives are necessary. Inferring causal effects of benefits in the practice has been very hard.
- What I want to suggest instead is Bayesian interpretation. You may know bayesian interpretation. The core concept of bayesian probability is diachroneity, //in other words, time flow. The probability of the hypothesis changes over time, // and prior and posterior can be computed// based on background information. Also, this concept provides inferences// without reliance on asymptotic approximation, //and we can use this concept //without any assumptions about the previous distribution. Therefore, this concept is very useful for real world simulation.
- Bayesian time series model has already been applied to business analysis. Google uses this model to AdSense for measuring ROI. After learning this,I thought that //this model would be effective// for measuring and comparing the effects of offerings.
- One approach developed at Google // is based on Bayesian structural time-series models. They used these models // to construct a synthetic control — // what would have happened to outcome metric // in the absence of the intervention. This method is flexible and modular, // and it is very useful //to measure and compare causal inferences of actions. They released the model and the related functions as an R package, //called CausaIlmpact. The paper about CausalImpact was hard to understand, // but the package was easy to download. You can just get it from Github.
- And I could integrate // this Bayesian time series prediction model and function // with multivariate tests. I developed the CompareImpacts function //with local functions in CausalImpact package // and a modified plotting function in this package // to compare effects of benefits at a glance // using functions in ggplot2 package. With these, we can create comparison reports and plots much more easily- // you only need to use a single R function, // and it will help people //comparing the predicted results of offerings more simply. Anyone can use the results, //without understanding the complicated Bayesian method // and the underlying R implementation. These functions will be shared through Github soon.
- This modification allows two offering types. One case is about providing the same offering //to different target groups. For example, some companies have offering with all customers, // and a few days later, // they divide their customers to some groups // and want to know // how those offerings effect to each customer segment.
- Another case is about trying various offers// for one targeted customer group// sequentially. Some companies do not segment their customers //and take offerings with all customers continuously. Others have offerings for only a single group, such as their VIPs. However, //if they want to know and compare effects of each offering, // it is very hard to measure. Therefore, those offerings usually become only one-use campaign and // the companies cannot use their results for the next offering. But with my design, you can compare causal effects //and use the result // to decide and to design offering more easily.
- Now, let’s see the causal effects of offering results. First, this is the original CausalImpact package usage. You can use it for testing each causal effect of offering, // but comparing causal effects may be difficult for some people // because of different informations of each data and complicated Bayesian concepts.
- These are the cases //with modified functions developed by me, //using Google’s Causallmpact R package and ggplot2 package as I mentioned. I’m going to show you // how to apply the functions //for the two real cases of offering data. First, this is a case of a promotion for three different target groups. We can see original data being plotted // in the first plot. And we can see different simulation results // and effects of the each benefit // with these dotted lines and color-shaded areas //in second and third plots In this example, I want to know which group is best for this promotion. However, // size, sales amount, visit periods and other data of each group are all different and // we don't know if they affect other factors. With this plot, //we can easily know and compare the impact for each group //without these consideration.
- Next, there are some promotions for one group with time difference. As time goes by, each data continuously changes differently//even though it is for the same group. Sometimes there are some statistically insignificant causal effect results. Then the plot is omitted like this. Originally, there were three promotion results, // but one result was omitted // because of insignificance. Unfortunately, it is very common to have a case like this. Of course these functions will be refined and dealt with more offering cases.
- Offering benefits is a very classical, proven business strategy, // and it has evolved with the technology over time. Nowadays we can gather related data more easily, // but it is still hard to use these data enough practically. But there are techniques //that can be used to develop offering data evaluation,// such as the Bayesian analysis that I presented. It is possible to wrap up these new techniques so that it is easy to use. That was my presentation. If you have any questions or comments, // please use the office hour after this session or send me an e-mail. I would be happy to discuss any aspects of this work. Thank you for listening.