Deep learning applications in e-commerce search: Dynamic talks Chicago 3/14/2019
•
0 likes•237 views
In this talk, we will discuss how recent advancements in artificial intelligence are transforming traditional search technologies. Deep learning-based image analysis and natural language processing open exciting new horizons for search and recommendation applications across the industries. We’ll talk about how deep learning models can help conventional search engines to achieve better relevance. We will share our experience implementing innovative solutions for online retailers, finance and high tech customers.
Recommended
Recommended
More Related Content
Similar to Deep learning applications in e-commerce search: Dynamic talks Chicago 3/14/2019
Similar to Deep learning applications in e-commerce search: Dynamic talks Chicago 3/14/2019 (20)
More from Grid Dynamics
More from Grid Dynamics (20)
Recently uploaded
Recently uploaded (20)
Deep learning applications in e-commerce search: Dynamic talks Chicago 3/14/2019
- 1. Deep learning in online commerce Privileged and Confidential Stanislav Stolpovskiy GD Engineer
- 2. Privileged and Confidential About me this is my robot:● Grid Dynamics engineer 2012-present ● 10+ years of Java experience ● Transitioned from Search to ML 3 years ago ● Graduated from St Petersburg IFMO University
- 3. Privileged and Confidential Agenda ● Case studies ● Foundational capabilities ● Deep dive: How to search by artistic style
- 4. Case studies
- 5. Privileged and Confidential How it all started ● Search accuracy hurt from bad catalog data ● Is this a common problem ? ● Initial approach: build an algorithm to recognize product color from the produce image ○ Works ok in most cases ○ But… not scalable ● Lack of scalability lead us to try a single ML model that could recognize all product attributes
- 6. Privileged and Confidential Automated attribution for leading department store chain LENGTH floor COLOR red HEM high-low SLEEVE sleeveless STYLE ruffled OCCASION evening Image embedding Text embedding Featurefusion Latent feature space AttributeClassifiers
- 7. Privileged and Confidential More like this for leading department store chain More Like This
- 8. Privileged and Confidential “Similar Art” feature for leading art retailer
- 9. Privileged and Confidential Pin recommendations for leading home improvements chain
- 10. Privileged and Confidential Part search for leading home improvements chain
- 12. Privileged and Confidential Deep learning: convolutional neural networks
- 13. Privileged and Confidential Visual recommendations
- 14. Privileged and Confidential Visual recommendations Image embedding Attribute embedding Featurefusion Latent feature space More Like This LENGTH floor COLOR red HEM high-low SLEEVE sleeveless STYLE ruffled OCCASION evening Nearest neighbors search
- 15. How to search by artistic style
- 16. Privileged and Confidential Visual search for art Similar by image contents: “peaches” Similar by artistic style: “old Dutch masters” Looking for something like this to decorate my room... ✅🚫
- 17. Privileged and Confidential “Similar Art” feature
- 18. Privileged and Confidential Deep learning: convolutional neural networks
- 19. Privileged and Confidential Feature extraction (vectoriser) G G G fusion style vector content vector fusion image vector
- 20. Privileged and Confidential General approach KNN - K-Nearest neighbor search
- 21. Privileged and Confidential High level architecture
- 22. Privileged and Confidential Dimensionality reduction ● Why? Image vectors are very large (~60K dimensions), need to reduce their size without losing much of features ● Randomized PCA is used (fbpca, sklearn) ○ fast, yet hogs memory ● Search space pruning allows to split into batches
- 23. Privileged and Confidential Approximate nearest neighbor search ● Why? Exact algorithm is too slow ○ With 2M vectors in memory, response time ~ 500 msec. ● Approximate KNN ○ Training: ~ 1.5h ○ Accuracy: ~90% ○ Response time: 5-10 ms
- 24. Similar image search on steroids
- 25. Privileged and Confidential Wild image issue Transfer learning from ImageNet is not working well for wild images 0.5 0.9
- 26. Privileged and Confidential Siamese network
- 27. Privileged and Confidential Triplet loss function
- 28. Privileged and Confidential Image feature extraction
- 30. Privileged and Confidential Find a chair in the room
- 31. Privileged and Confidential Find a chair in the room
- 32. Privileged and Confidential Find a objects in the room
- 33. Privileged and Confidential Object detection network
- 34. Privileged and Confidential High level architecture
- 35. Part search
- 36. Privileged and Confidential Real-world size problem ● Part search requires high dimensional precision ● Image doesn’t provide any size information ● The same objects can be made in range of different sizes ● We have to ask to use reference object
- 37. Privileged and Confidential The Object Detection approach to localization Pros: ● Detects multiple objects in an image ● Detects different classes in an image ● There are a lot of pre-trained models for different frameworks Cons: ● The bounding box around an object still contains background ● High recall must be kept. At the same time, avoiding false positives sometimes failed for real cases ● The Object Detection model is computationally heavy.
- 38. Privileged and Confidential UNet localization U-Net models classify each pixel that belongs to the object of interest.
- 39. Privileged and Confidential High level architecture
- 40. Appendix
- 41. Privileged and Confidential AI catalog management for leading department store chain Merchandiser AI assistant product attribute suggestion product clustering for mass edit category suggestions product name generation attribute verification
- 42. 3. Solution Solution is based on state-of-the art natural language processing model based on transfer learning of the powerful general purpose language model. Fine tuning is used to complete the training of the language model for specific domain, in this case - shopping. NLP model is able to accurately classify complex phrases which helps to correctly interpret customer intent in a complex dialog. Same approach can be used for wide range of phrase and text classification applications, such as customer support. Case study: Intent analysis for conversational platform Privileged and Confidential 42 1. Situation An online retailer is developing conversational commerce solution enabling discovery, selection, checkout and post-order functionality with smart voice devices such as Google Home and Alexa. Online retailer is looking for the solution which is able to accurately recognize shopping intents of for utterances such as “Can you suggest me an action camera?” 2. Scope and Goals Grid Dynamics is developing a conversational commerce platform capable of accurately classify customer intentions, understand the query and support a seamless dialog with the customer. can you ... ... ... action camera? word embedding phrase matrix LSTM LSTM ... LSTM ... ... pooling linear softmax backward phrase embedding LSTM ... ... LSTM ... LSTM forward phrase embedding CLASS PROB ACCEPT 0.01 DENY 0.001 NEXT 0.05 … … DISCOVERY 0.81 SELECTION 0.1 ORDER 0.05 SMALLTALK 0.02
- 43. Privileged and Confidential Deep learning: convolutional neural networks
- 44. Privileged and Confidential UNet localization
- 45. Privileged and Confidential 250-dim latent space Image features Feature fusion with CCA 1 x 2048 Text features 1 x 300 Distances to minimize Find a pair of linear transformations Wimage and Wtext which minimizes distances in latent space of a given size for all items in training set Wimage Wtext Wimage Wtext
- 46. Privileged and Confidential First neural network LENGTH floor COLOR red HEM high-low SLEEE sleeveless STYLE ruffled OCCASION evening Image embedding Featurefusion Latent feature space AttributeClassifiers We use CNN instead algorithm Works for all product types
- 47. Privileged and Confidential Results -- adding ML helped! ● Increased merchandiser productivity ● Improved relevancy of search ● Able to cover product attribute checking for an extremely large catalog ● Reduced catalog misattribution using this tool for the last 3 years
- 48. Privileged and Confidential Item identification for online dinnerware retailer Catalog image Wild image
- 49. Privileged and Confidential Shoppable content product_type:coat gender: women material: fur prod214380280 women fur coat similar to this
- 50. Privileged and Confidential The next way to improve search with ML LENGTH floor COLOR red HEM high-low SLEEE sleeveless STYLE ruffled OCCASION evening Image embedding Featurefusion Latent feature space AttributeClassifiers Opportunity to use image recognition as a search feature not just a product attribute checker