Slides from sharing at Strata + Hadoop Singapore 2016 (http://conferences.oreilly.com/strata/hadoop-big-data-sg/public/schedule/detail/54542)
Ecommerce has enabled retailers to make all of their products available to consumers and consumers to access niche products not found in brick-and-mortar stores. This growth provides consumers with unparalleled choice. Nonetheless, the sheer number of products brings with it the challenge of helping users find relevant products with ease.
Lazada has tens of millions of products on its platform, and this number grows by approximately one million monthly. Lazada’s challenge: How can we help users easily discover good quality products they will like? How can we ensure product selection remains fresh and constantly updated?
One way to do this is through the ranking of products. Via ranking, Lazada helps customers easily find products that will delight them by ensuring these products appear in the first few pages. I’ll share how Lazada ranks products on our website. (Note: Google “how amazon ranks products” for some industry background)
Topics include how we:
* Develop methodology (and tricks) to solve not-so-well-defined problems
* Collect and store user-behavior data from our website and app
* Clean and prepare the data (e.g., handling outliers)
* Discover and create features useful features
* Build models to improve customer experience and meet business objectives
* Measure and test outcomes on our website
* Built this end-to-end on our Hadoop infrastructure, with tools including Kafka and Spark
4. Lazada Data Science
Data App Devs expose, integrate, platform-ize
Data Scientists explore, prepare, model
Data Engineers collect, store, maintain
Start from bottom up
17. Web Tracker
(JavaScript)
Mobile Tracker
(Adjust)
3rd
Party
(e.g. ,ZenDesk,
SurveyGizmo)
Kafka Queues
Bulk Loaders
(Spark)
Hadoop
Hadoop
Data
Exploration
+
Data
Preparation
+
Feature
Engineering
+
Modelling
(Spark)
Manual
Boosting
(Django)
Local
Validation
A/B
Testing
Product
Seller
Transaction
Product rankings
Split traffic and measure outcomes
(Category Managers)
(User devices)
18. Overall results
Better ranking improved conversion and revenue per session
Introducing new products improved new product engagement
Emphasizing product quality had neutral to positive outcomes
21. Problem
Lazada has millions of products—not easy to navigate
How to identify products that interest users in the future?
How do we measure interest?
24. Data preparation
Filter and categorize online behavioral events
(e.g., impressions, clicks, etc.)
Merge various views of product data (e.g. price, stock, etc.)
Exclude outliers and potentially fraudulent events
26. Modelling
(i.e., machine learning)
Predict future (tomorrow’s) product clicks/checkouts
Examine results against a benchmark model
Pandas + XGBoost is faster and more effective than
Spark + MLlib; assessing XGBoost4J-Spark
27. Boosting products
(manually)
Manually increase rank of certain products
(e.g., highly anticipated products, campaign tie-ups)
User-friendly interface to drag-and-drop products
Limits on how many products can be boosted
28. Validation and
A/B testing
Local validation is easy, but difficult to ensure
similar results via A/B testing
A/B test all updates before production
32. Problem
Products with strong engagement stay on top
Products without engagement don’t get traffic
How can we identify new products that are likely
to interest users?
33. Methodology (demand)
Find what people need
Measure needs through internal/external data
Rank new products in terms of demand
34. Methodology (supply)
Find products similar to top products
Measure similarity with top products
Rank new products based on similarity and top
product volume
35. Data preparation and
feature engineering
Parse (log) data to identify shoppers’ needs
Measure potential product demand
Model product similarity (Spark GraphX / ElasticSearch)
36. Validation and
A/B testing
Limited capability on existing A/B testing platforms
to track specific products
Measure performance of new products across
experimental groups using in-house tracker
37. Results
Increased new product click-thru rate by 30 – 80%
Increased new product add-to-cart by 20 – 90%
Expected overall conversion to decrease—increased
instead (though not statistically significant)
39. Intent
Improve customer experience throughout purchase journey
From online browsing to receiving of product
Product quality identified as key driver
41. Methodology (online)
Content (e.g., title quality, richness of content)
Reviews (e.g., average rating, negative reviews)
Performance (e.g., click-thru rate, browsing time)
42. Methodology (offline)
Perfect order rate (i.e., not cancelled, not returned, etc.)
Negative feedback (e.g., counterfeit, complaints, etc.)
Seller metrics (e.g., timely shipped-rate, return rate, etc.)
43. Data preparation and
feature engineering
Derive product features (e.g., title quality, image quality, etc.)
Measure content richness (e.g., attributes available,
grouping, etc.)
Measure delivery performance and customer feedback
44. Results
Improved quality of products displayed
Increased conversion by 3 – 5% for some countries
Small conversion change in other countries (non-significant)
45. Key takeaways
Data science is (i) team sport, (ii) partly R&D, (iii) iterative
How you use data to solve problems (methodology), data
preparation, and feature engineering > machine learning