How Machine Learning Can Transform The Customer Experience
•Als PPTX, PDF herunterladen•
5 gefällt mir•1,403 views
Aarthi Srinivasan talks at Product School, Silicon Valley
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (20)
Ähnlich wie How Machine Learning Can Transform The Customer Experience
Ähnlich wie How Machine Learning Can Transform The Customer Experience (20)
Mehr von Product School
Mehr von Product School (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
How Machine Learning Can Transform The Customer Experience
- 1. How Machine Learning Can Transform The Customer Experience Evren Korpeoglu, Data Science Aarthi Srinivasan, Product Management /Productschool @ProdSchool /ProductmanagementSV
- 2. Aarthi Srinivasan - Walmart Labs - 12+ years of combined experience in product management, consulting and engineering - MBA & MS in Computer Science www.productschool.com INTRODUCTIONS Evren Korpeoglu - Walmart Labs Data Scientist - Machine Learning, Big Data, Statistical Modeling & Optimization for powering real-time Experiences - Ph.D. Operations Research
- 3. What to Expect Today? 3 • What is machine learning ? • Why is it important ? • How do we use it ? • Technical Concepts • Examples
- 4. What is Machine Learning? 4 1. Science of getting computers to learn or recognize something without being explicitly programmed – Andrew Ng • Branch of Artificial Intelligence which is a branch of Computer Science • Give lots of data to the computer so that it can figure it out • One of the first examples is the computer checkers program by Arthur Samuel * - ref: Andew Ng Courses, Big data: A revolution 2. Distinguish big data & machine learning: Big data is the data seed for creating machine learning forests • Big data collects information based on our digital exhaust (crumbs we leave in the digital world) , demographics, preferences, health etc. • Machine learning will mine this data and model behaviors with interactive responses based based on this data
- 5. Why do we need this? 5 1. Tons of applications impacting human health, utility and simplification Health & Wellness Utilitarian Futuristic • DNA sampling & diagnosis • Health reminders & prevention through AI tools • Correlation studies • Personalized medicine tablets, diets • Real time optimized path maps • Search Ranking • Spam filter on email • News aggregators • Shopping Recommendation • Facebook face recognition • Age recognition (How-old.net) • Voice recognition – Siri, Alexa • Driverless cars • Home decoration
- 6. Key Terms 6 • A set of data used to predict relationships. Data and answers for each sample. • E.g. A diamond’s size, cut, color and clarity helps predicts the price. Training Set • Uses training set to make a prediction. • E.g. Model predicts diamond prices based on past prices. Supervised Learning • Provide data without suggesting anything so computer can identify patterns or groupings. • E.g. Customer segmentation, DNA groupings. Unsupervised Learning • Each distinct measurable data value you select in the training data set. • E.g. A diamonds’ size is one of the feature’s for predicting price. Features/ Variables / Attributes • Using the features provided in the training set make a prediction. Fit a curve using the data provided. • E.g. Price of diamond = X*Cut + Y*Clarity + Z*Size + other features… Supervised: Regression • A defined set of categories that can be labeled for placing new observations. • E.g. Presence of absence of cancer; Types of diabetes Supervised: Classification • Process of assigning observations into subsets. • E.g. Customer segment creations Unsupervised: Clustering
- 7. Learning Steps 7 Collect / Update User Data 1 Create / Update Training Set data 2 Create / Update algorithm for training data Update Algorithm Validate Algorithm 3 Create predictive model 4 New real-time observations A/B Test & Launch on production 5
- 8. Data Wrangling and Feature Extraction 8 Spam Email Detection Title Sender Domain # of Recipients Email content Country of Origin Non- dictionary Words Hyperlinks Address Book Length of email • Structured Data (Best) – RDBMS, columnar data – Strict Schema – SQL • Semi-Structured Data (Better) – JSON, XML – Enforce minimum schema – JSON, XML Parser • Unstructured Data – Text, Image, Raw email – No Schema – Batch processing – Regular expressions – Map Reduce GARBAGE IN GARBAGE OUT
- 9. Model Training 9 Feature Extraction (Feature vector) New Text documents User Activity Images Transaction history Feature Extraction (Feature vector) Labels Machine Learning Algorithm Training / Testing Text documents User Activity Images Transaction history Predictive Model Expected Label Model Evaluation
- 10. Supervised learning techniques 10 • Linear classifier (numerical functions) • Parametric (Probabilistic functions) – Naïve Bayes, Hidden Markov models (HMM), Probabilistic graphical models • Non-parametric (Instance-based functions) – K-nearest neighbors • Non-metric (Symbolic functions) – Classification and regression tree (CART) • Aggregation – Bagging (bootstrap + aggregation), Adaboost, Random forest, Ensemble models
- 11. Linear Classifiers 11 • Logistic regression – ) – w with minimum loss – Solve iteratively using gradient descent • Support vector machine (SVM) – Maximum margin classifier • Artificial Neural Networks – Inspired from how neurons work – Activation function (Sigmoid, ReLU etc.) – Deep Learning
- 12. KNN / CART 12 • K-Nearest Neighbors – Find K nearest training examples – Majority vote – Easy to implement – Not scalable for real time predictions • Classification and Regression Trees – Easy to interpret for small trees • Random Forests – Ensemble of decision trees – Usually performs very good
- 13. Unsupervised Learning 13 • Clustering – K-means clustering – Spectral clustering • Dimensionality reduction – Principal component analysis (PCA) – Factor analysis • Product Recommendations – Collaborative Filtering • Association Rules – Market Basket Analysis
- 14. Model Evaluation 14 • Measure model performance • Optimize model to improve prediction quality – Feature selection – Hyperparameter tuning • A/B Testing • Explore/Exploit • http://en.wikipedia.org/wiki/Precision_and_recall
- 15. Sample Architecture 15 -HADOOP - SPARK PREDICTION ENGINE REAL TIME DATA SQL / NO SQL Data Base CLIENT MACHINE LEARNING SYSTEM
- 16. Health & Wellness Sen.se Mother (iOT) 16
- 17. Amazon Echo & Personalization 17
- 18. Houzz Visual Match Deep Learning 18
- 20. Sample E-Commerce Applications 20 1. Segment customers (E.g. Millennial college grads, Moms, New Dads, etc.) 1. Personalize experiences for segments (Moms will see unique customer layouts and promotional items compared to dads or teens who love video games) 1. Personalize marketing e-mail and even timing of e-mail delivery 1. Trigger experiences based on customer information or local events (e.g. shipping preferences, Events like birthdays or concerts) 1. Create a personalized basket based on previous purchases or life stage 2. Use BOTs to provide relevant information to users 1. Augmented reality - Provide personalized information for sale items
- 22. Appendix 22
- 23. Sample Personalization Highlights Testing Results with test (2MM – 7MM pop) POV Personalized vs. No personalization Personalized POV increased conversion by x%. POV White listed personalization vs. Full automation Desktop conversion increased by x%. Layered POV vs. Static POV Conversion increased by x%, PVR increased by x%, Bounce reduced by x%. Personalized carousels on the home page Increased conversion on Desktop users by x% Personalized DTC vs. Curated Mother’s day carousel CTR increased by x%.
- 24. Upcoming Courses Silicon Valley October Cohort Weeknights: October 18th Weekends: October 15th Apply At www.productschool.com www.productschool.com
- 25. www.productschool.com Upcoming Workshops Rsvp On Eventbrite Sept 28: Product Management Course - Info Session Oct 5: From Building Products To Managing Them Oct 12: Risk Management while creating great products: Why no one really cares and what happens because of that Oct 19: Product Management Happy Hour
Hinweis der Redaktion
- Let me start today’s session with a question from a scientist Ricardo Santini “What is the amount of information (atoms) stored in a little baby?” It is like filling 2000 titanics with flash drives. Given that each human has 3 Billion DNA sequences (about 262,000 pages if we print it page by page), we can visualize the opportunity to use this data to personalize every detail to match taste of the user including the opportunity for personalized medicines Bina technologies is one of the companies doing this which got acquired by Roche For those who are familiar with mindfulness and meditation techniques which appears to be far from science, the basic essence is teaching the user to experience this constantly changing atoms in the human body with focus. I am sure if I continue on this topic you are going to ask for your money back so lets get back to Machine learning. Assume that we are delivering this presentation using mixed reality where the pace, tone and content adjusts automatically to your brain’s response to the content. How cool with that be? With that lets get to introductions
- Evren is a data scientist with experience on machine learning, big data, statistical modeling and optimization. Currently at Walmart Labs, he is focused on developing cutting-edge machine learning algorithms that positively impact business results. He has architected a highly scalable real time platform to personalize the omni-channel customer experience. Prior to this role, he was at HP Labs working on designing procurement auction algorithms and stochastic optimization models. He enjoys using latest technology to solve business-critical problems and develop next generation intelligent systems. He holds a Ph.D. in Operations Research from Bilkent University, Turkey. I enjoy building dynamic teams to launch interesting products. At Walmart Labs, I am focused on creating a personalized experience for key customer segments across front end digital touch points. Prior to that I was with Financial Engines where her teams launched the award winning Social Security and Income Planning solutions that eventually resulted in their CEO being invited to the white house. I was also a product manager at Intuit with a focus on behavioral science for customer conversion initiatives and has worked in enterprise applications in her earlier years. On the academic front, she holds an MBA from Wharton, MS in Computer Science from Stony Brook University and BS from Madras University. I enjoy motivating teams to combine voice of the customer, data analytics and lean testing to manage a portfolio of products.
- Today we will cover the topic of Machine learning. Evren will get into how the magic happens with engineering.
- Probability of winning board vs. loosing board is calculated 1 What is ML Branch of computer science and AI Lots of data to a computer so they can figure it out 2. It is important to distinguish between big data and machine learning. You need lots of seed data for creating a machine learning algorithm or related algorithms e.g. DNA data We can get this data from just who we are – demographics, psychographics, digital exhaust, our health records, education and the list keeps going. Ok so what – you collect this data and then ML will mine this data, find patterns and model behavior with interactive responses. 1. Amazon’s Dash and 23 &me collect a ton of data about their users which they can put to use for predictions with machine learning. Pillo is the in house health robot Now that we know what it is, let us look at why it is important
- Health & Wellness applications: DNA sampling Pillo the robot which reminds you to take the tablets Data correlated to diseases Personalized medicines that are based on our genes or microbiomes present in our body. Utilitarian: Google maps (How many of you used maps to get here?) Search ranking algorithm Futuristic: -
- Training set: Data and answers for each row e.g. Diamond prices are determined by its cut, clarity, color and size. Supervised Learning: Uses a training set to make the prediction for a new observation. For example, you give a new diamond’s cut clarity and size to the model and it will predict a price based on the other past training samples you have provided. Unsupervised Learning: Provide data without suggesting anything so computer can identify patterns or groupings. Next, lets talk about the high level steps in Machine learning.
- Steps 1 and 2 is all about data collection and creating a training set 3 & 4 are the crux which is the secret recipe or algorithm that will be built for the training data, validated and used for predicting results on observations. Finally A/B test this on production and update models with real time data Evren will share the magic behind these circles now.
- Thank you Evren – That was amazing and now you have made us product managers dangerous enough to talk technical with our engineering partners!! Lets take some examples. Mother is an IoT device that has wireless sensors to attach to pillows, pill boxes, doors, water consumption to collect data. Then you observe the data to find standard patterns of sleep, pill intake, activities at your door and water consumption to name a few. It is kind of like big brother watching but you can decide where you want these sensors. This is always the delicate balance so you don’t lose the humanity aspects in Machine learning. The example in Mother monitors kids home work or brushing teeth which was a bit too much for my balance. If the algorithm learns enough about you based on your data and notices anomalies it will notify you.
- Amazon is a pioneer for using machine learning in making the right recommendations for you. It has smartly invested in devices such as dash and echo that are wired to learn more about you. When you think about tide dash, the laundry equipment manufacturer eventually embed the dash like device so it automatically detects how many loads you have completed and reorder your detergent for you. Some manufacturers have started hiring for this type of role. Echo’s voice recognition Alexa is constantly capturing your questions, scheduled and preferences such as music. The most common use case in e-commerce companies is the Recommendation module which has carousels of relevant items. Based on your features such as: customer segment, previous clicks, current session data, order history or even browser used, gender, demographics and psychographic data the algorithm predicts which carousel and items to show you. A-mazon is also constantly testing the layout of the page based on information about the user. They do all these tests with Machine learning to determine which layout and carousels impact the e-commerce business metrics such as product view rate (PVR) and conversion positively.
- Upload a dream photo on Houzz Houzz will find suitable products that are similar to the ones in your picture from the inventory ------ Data set where it analyzes your ideal lamp and compares other lamps to your lamp to find the best possible match
- We are also testing on Wal-Mart and even simple tests prove that personalization is moving the needle positively for us. So a sample test journey would be is to go from a fully curated non-personalized page to a semi-personalized page to a completely personalized page that even changes layout and context based on the user. At every step of the way we should have a highly controlled test. We ran a test where this asset on the page was static and compared it with a personalized assets that were dynamic based on location (E.g. winter clothes in North east vs. California) Next we started testing different type of assets to personalize with products inside. Obviously we are also doing recommendation algorithms just like other e-commerce players.
- Segment customers (E.g. Millennial college grads, Moms, New Dads, etc.) Personalize experiences for segments (Moms will see unique customer layouts and promotional items compared to dads or teens who love video games) Personalize marketing e-mail and even timing of e-mail delivery Trigger experiences based on customer information or local events (e.g. shipping preferences, Events like birthdays or concerts) Create a personalized basket based on previous purchases or life stage Use BOTs to provide relevant information to users Augmented reality - Provide personalized information for sale items