Ecommerce customers are accustomed to an accurate, smooth, and timely search experience. Behind the curtains is a chaotic battle to operationalize a hefty strategy, optimize a fickle infrastructure, and rally troops around a single vision. Hear how Nordstrom Rack | Hautelook braved this with help from Lucidworks Fusion.

2. My Journey
PA N K A J A N D H A L E
Senior Data Scientist
Nordstrom Rack | Hautelook
B U I L D I N G S E A R C H & P E R S O N A L I Z AT I O N J O U R N E Y
• Background : Computer Science Engineering + Machine Learning
• Experienced working on textual data enrichment leveraging Crowdsourcing approach and external data
sources to create quality datasets to enable Machine learning @Intuit
• Using scientific techniques to measure and tune the relevance of search algorithm @ Intuit
• Currently working on improving relevance by leveraging customer behavior & building recommender
systems

3. Outline
• Challenges
• Road to Relevance
• Fusion
• Results
• Q & A

4. Search Lenses
Relevance ExperiencePerformance

5. Search Quality

6. Challenge #1
Bad Customer Experience
• Untuned Solr 6
• Customer complaints
• No dedicated search team

7. Challenge #2
Data Driven Culture
• KPIs not defined
• No way to AB test performance of various
algorithms
• No dashboards to track quality,
performance, or surface opportunities
• How do we measure success?
Search
Visitors
Non
Search
Visitors
1x
2x
to
3x
Conversion Rates
(Industry)
1.67x

8. Challenge #3
Speed & Flexibility
• No synergy between Search and
Merchandising
• Simple tweaks required code changes and
deployments
• Slow to fix simple issues

9. Challenge #4
Data Management & Processing at
Scale
• Processing customer behavior & leveraging in
near real time
• Infrastructure did not support

10. Addressing
Challenges

11. • 1 Product Manager
• 1 Engineering Manager
• 1 Architect / Principle Engineer
• 1 Solr Engineer
• 1 QA
• 1 Data Scientist
Forming a Team
Road to Relevance

12. Fusion Platform
Road to Relevance
• Rapidprototyping
• Testing
• Deployment
• EasyAB testing
+
• Signalaggregation
• Versioning
• Managedrules
• Machine learning

13. Measuring Quality
Road to Relevance
• 42 KPIs and supporting metrics defined,
audited, and tracked
• Unlocked power of Google Analytics
• NewSearchDashboardautomatedtime-consuminganalysis
consuming analysis
• Democratized search domain analysis
• AB testing on iOS

14. Cross-Functional Feedback
• 30+ Merch Team issues fixed in two weeks
• Reciprocal sharing of knowledge led to
quick understanding
• Most new issues can now be resolved in the
same day
• Giant leap in business agility
MerchandisersRoad to Relevance

15. Signal Aggregation
Road to Relevance
• Query:red dress

16. Noteworthy Example of Relevance
Tuning
Road To Relevance
• Deep dive into the query logs
• Understanding the customer intent
• Leveraging the query pipeline stage
• Significant increase in the search count

17. Conversion Rate
Searchvs.Non-SearchSessions
Results
1.7x

18. Conversion Rate
Searchvs.Non-SearchSessions
Results
1.7x 2.9x

19. Search Quality
https://opensourceconnections.com/blog/2018/11/19/an-introduction-to-search-quality/

20. THANK YOU

21. Questions