1. AI in Retail
Subrat Panda, Principal Architect
Capillary Technologies
AI First - Thought Leader
up.AI Summit IIT Kharagpur

2. About Capillary
■ Incubated in IIT Kharagpur
■ Successful SaaS Companies
■ CRM – Loyalty, Campaigns, Omni-Channel engagement
■ Helping our customers engage better with their customers
through AI

3. Brief Introduction about me
● BTech ( 2002) , PhD (2009) – CSE, IIT Kharagpur
● Synopsys (EDA), IBM (CPU), NVIDIA (GPU), Taro (Full Stack Engineer), Capillary
(Principal Architect - AI)
● Applying AI to Retail
● Co-Founded IDLI (for social good) with Prof. Amit Sethi (IIT Bombay), Jacob Minz
(Synopsys) and Biswa Gourav Singh (AMD)
● https://www.facebook.com/groups/idliai/
● Linked In - https://www.linkedin.com/in/subratpanda/
● Facebook - https://www.facebook.com/subratpanda
● Twitter - @subratpanda

4. Industry 4.0
https://en.wikipedia.org/wiki/Industry_4.0
1. Interoperability
1. Information
transparency
1. Technical assistance
1. Decentralized
decisions

5. Knowledge is Power - Sir Francis Bacon
- Industry 4.0 enabled by IoT, BigData and AI
- IoT is the intelligent sensor
- BigData will enable processing huge volumes of data
- AI will make sense of the data in decision making
- AI helps transform raw data into power - AI will transform businesses
for sure
- Primarily Machine Learning and then the deeper aspects with Deep
Learning
AI is the bedrock on which Industry 4.0 relies on.

6. The AI landscape

7. Machine Learning – http://techleer.com

8. What AI can and cannot Do today ?
https://hbr.org/2016/11/what-artificial-intelligence-can-and-cant-do-right-now

9. Supervised Learning
1. Being able to input A and output B will transform many industries.
2. The technical term for building this A→B software is supervised learning.
3. The best solutions today are built with a technology called deep learning or deep neural
networks, which were loosely inspired by the brain.
4. Basically labelled data is the most important requirement for Supervised Learning.
If a typical person can do a mental task with less than one second of thought, we can probably automate it
using AI either now or in the near future. - Andrew Ng

11. Transfer Learning - http://ruder.io/transfer-learning/

12. Transfer Learning - http://ruder.io/transfer-learning/

13. Drivers of ML Success

14. Corporate Strategy - AI
Very important to understand where value is created and what’s hard to copy. The AI community is remarkably open, with
most top researchers publishing and sharing ideas and even open-source code.
Algorithm has become a commodity.
In this world of open source, the scarce resources are therefore:
● Data - Data, rather than software, is the defensible barrier for many businesses.
● Talent - Simply downloading and “applying” open-source software to your data won’t work. AI needs to be
customized to your business context and data. This is why there is currently a war for the scarce AI talent that can
do this work.
● Domain Expertise + Data + Software ==> Scalable AI strategy
● Quoting Andrew Ng here.

15. Data Strategy or Data Governance
- AI solutions won’t work for you unless you have a clear cut data strategy
- Data acquisition
- Data curation
- Data normalization
- Data protection
- Data ingestion
- Data Extraction at scale
Platforms need to be built and is very specific to the problem one is working on.

16. Talent Strategy
http://blog.belong.co/can-india-supply-her-companies-with-artificial-intelligence-talent

17. Talent Strategy
● Deep Domain Expertise - You already have an edge
● Upskill folks in AI - depending on the data that you are dealing with
● They should be able to do the following:
https://blogs.sas.com/content/subconsciousmusings/2017/04/12/machine-learning-algorithm-use/
● The size, quality, and nature of
data.
● The available computational
time.
● The urgency of the task.
● What you want to do with the
data.

18. When am I ready to adopt AI ?
1. Can the Use-case be enhanced by the use of AI ?
2. Am I/Customer comfortable with the probabilistic nature of AI solutions ?
3. Do I have enough data ?
4. Is the !/$ spent on AI based solution worthy of investment ?
5. Do I have the timeline to iterate and improve/experiment ?

19. Risks in AI Adoption
Yet the biggest harm that AI is likely to do to individuals in the short term is job
displacement, as the amount of work we can automate with AI is vastly bigger than
before.
You need to understand your data better.
If you cannot create differentiation with data it is a problem.

20. Traditional Applications of AI in Retail
■ Customer Segmentation
■ Inventory Management
■ Recommender Systems
■ Campaign Management
■ Insights

21. Neo-AI in Retail
■ Enhanced Inventory Management – Takes care of other factors
which could have been hard to decipher.
■ Enhanced Recommender Systems
■ Customer Engagement – Chatbots based interface (NorthFace –
using IBM Watson)
■ Consumer Insights – Deep Understanding of Stores/Customers
■ Logistics and Delivery – Robots and Drones

22. Using Computer Vision in Retail
Video analytics, derived through computer vision, helps retailers answer
many critical questions, including:
• How many shoppers entered the store?
• What are my shoppers’ gender and age ranges?
• Where do shoppers go in my store (and where do they not go)?
• Where do shoppers stop and engage with fixtures or sales
associates?
• How long do they stay engaged?
• Which are my most effective fixtures, and which ones are
Video analytics, derived through computer vision, helps retailers answer many critical
questions, including:
● How many shoppers entered the store?
● What are my shoppers’ gender and age ranges?
● Where do shoppers go in my store (and where do they not go)?
● Where do shoppers stop and engage with fixtures or sales associates?
● How long do they stay engaged?
● Which are my most effective fixtures, and which ones are underperforming?
● RetailNext integrates a variety of sensor technologies as part of its “technology
stack” in building its industry-standard retail analytics platform.
● Reference: https://retailnext.net/en/blog/computer-vision-sees-better-than-2020/

23. How we do this ?
• How many shoppers entered the store? – People Counting
• What are my shoppers’ gender and age ranges? – Demographic Analysis
• Where do shoppers go in my store (and where do they not go)? - Heatmap
• Where do shoppers stop and engage with fixtures or sales associates? – Shoppers
Tracking
• How long do they stay engaged? – Tracking and Identification
• Which are my most effective fixtures, and which ones are underperforming? – Peel
Off Counters
■ All of these can be solved using AI.

24. Computer Vision – The Sixth Sense in
AI Retail
■ Affectiva, an MIT Lab spinoff that have analyzed over 5 million faces, enables retailers to use facial
tracking to generate invaluable emotional insights the inform digital displays and in-store signage.
■ Sensing up to 7 human emotions (including anger, sadness, disgust, joy, surprise, fear and contempt)
up to 20 different facial expressions, age range, ethnicity and gender, their recognition technology
analyzes pixels in those regions to classify facial expressions and mapping them to associated
emotion emojis.
■ Building customer segmentations based on computer vision data and sentiment analysis empowers
retailers on a deeper level. It adds a layer of complex thinking to pass/fail decisions. It allows
retailers to understand the dynamics of a living lab store environment. On a basic level it gives
answers as to traffic patterns and dwell times, but on a more complex level it can drive true
personalization. It can empower sales associates to serve as personal concierges to each customer.
■ Reference: https://www.linkedin.com/pulse/why-computer-vision-sixth-sense-retail-melissa-
gonzalez

25. Computer Vision
■ Emotion plays a huge part in marketing and brand building
■ Technologies like AR, VR, 3D modeling are used to evoke those emotions.
■ Retailers can truly customize the consumer experience, advertise full
product ranges more effectively, and also design more engaging and
customer-friendly store layouts and displays to increase revenue
■ Popular in store #selfie marketing campaigns.
■ Reference :
https://channels.theinnovationenterprise.com/articles/computer-vision-
picturing-the-future-of-retail

26. Trax
Trax offers three computer vision-based products:
■ Retail Execution: This product enables field reps of consumer packaged goods (CPG) companies and
third-party auditors to capture shelf data with mobile phones and tablets and receive real-time
reports on corrective actions to take in the store.
■ Shelf IntelligenceSuite(by Traxand Nielsen): This product provides continuous and accurate retail
measurement and analysis based on category shelf and point of sale data
■ Retail Watch: This product delivers real-time store monitoring analytics for retailers to reduce
stockouts and improve planogram compliance.
■ Trax, a company that has developed a computer vision platform designed to provide data insights
for consumer packaged goods companies and retailers, has received $64 million in funding.
■ Reference: http://www.vision-systems.com/articles/2017/06/computer-vision-company-enabling-
retail-store-insights-receives-64-million-in-funding.html

27. NLP in Retail
■ Chatbots are ubiquitous
■ Customer engagement through contextual discussion
■ Different from normal FAQ based chatbots as context is lot relevant
■ Uses – preference elicitation, recommendation based on personal history,
enables long contextual interactions.
■ Luis from MS, WIT from FB, Watson’s Chatbot framework based in
Bluemix.

28. References
■ https://www.techemergence.com/artificial-intelligence-retail-10-
present-future-use-cases/
■ https://www.forbes.com/sites/kimberlywhitler/2016/12/01/how-
artificial-intelligence-is-changing-the-retail-experience-for-
consumers/#51422f6c1008

29. Q and A ?
- What were your key takeaways ?