In this talk I will briefly discuss the ubiquitous needs of video and video understanding across Alibaba and the challenges that are being addressed and solved at iDST, Alibaba's AI R&D division. Examples include mobile shopping on Taobao, video search and recommendation on Youku and Tudou, and real-time systems for Cainiao Logistics and City Brain.

1. The Quest for Video Understanding
Xiaofeng Ren
Institute of Data Science and Technologies (iDST), Alibaba Group

2. Alibaba
2

3. Alibaba Ecosystem
3
CLOUD COMPUTING OPERATING SYSTEM
DATATECHNOLOGY TEAM
Online and Mobile Commerce Platform
$
BUYERSSELLERS
Marketing
Financing &
Payment
Solution
Professional
Service Providers
Digital
Entertainment
Logistics
Mobile Browser
Location-BasedSocial Media
Weibo

4. Alibaba Mission
4
CLOUD COMPUTING OPERATING SYSTEM
DATATECHNOLOGY TEAM
Online and Mobile Commerce Platform
$
BUYERSSELLERS
Marketing
Financing &
Payment
Solution
Professional
Service Providers
Digital
Entertainment
Logistics
Mobile Browser
Location-BasedSocial Media
Weibo
make it easy to do business anywhere

5. (E-)(M-)(O-) Commerce
5

6. Text Image Video Interactive Media

7. Taobao @ 2009
7

8. Taobao @ 2017
8

9. Taobao Search: Text  Text
9
Title：茜舞2017春装新品 韩版修身公
主裙红色连衣裙 无袖打底背心裙春秋
Q：无袖蓬蓬裙连衣裙夏 a型
Title：美特斯邦威卫衣女士
春装字母拼色针织套头衫
Q：百搭灰色套头无帽卫衣女

10. Taobao Search: Text  Text
10
Title：茜舞2017春装新品 韩版修身公
主裙红色连衣裙 无袖打底背心裙春秋
Q：无袖蓬蓬裙连衣裙夏 a型
Title：美特斯邦威卫衣女士
春装字母拼色针织套头衫
Q：百搭灰色套头无帽卫衣女

11. Taobao Search: Text  Image
11
Title：茜舞2017春装新品 韩版修身公
主裙红色连衣裙 无袖打底背心裙春秋
Q：无袖蓬蓬裙连衣裙夏 a型
• Joint text and image embedding for description
• Leverage user click data to learn query-description similarity

12. Taobao Search: Image  Image
12
3 billion images over 100 million products, 12 million daily active users

13. Videos on Taobao
13

14. Videos on Taobao
14
• Promotional video
• Video guide on how to use product
• Documentary on how a product is made
… …

15. Live Streaming on Taobao
15
• Many top-selling stores have live streaming
• Allow user interaction (chat, FAQ)
• Conversion rate much higher than normal

16. Videos on Taobao
16
• Short videos are abundant on Xianyu (second-hand trading platform)
• Demonstrations of products; sometimes people showing skills
• Much richer information than text descriptions

17. Videos on Taobao
17
• Authoring tools
• Video enhancements
• Content analysis and tagging
• Video-to-text matching
… …

18. AR/VR on Taobao
18

19. Youku & Tudou: “Youtube-like”
19
Founded 2006
Founded 2005
IPO 2010
IPO 2011
Merged
2012
Joined Alibaba
2016

20. Youku: Video Hosting
20
30M paid members
~400M active users
700-min per month

21. Youku: Product Identification
21

22. Youku: Product Identification
22

23. Youku: Product Placement
23

24. Tudou: UPGC Sharing
24

25. Tudou: UPGC Sharing
25
Copyright protection
Authoring tools
Search & recommendation
… …

27. Cainiao Logistics
27
24 hours in China
72 hours globally
1B packages on 11/11Mission:

28. Robots @ Cainiao
Delivery Robots Delivery Drones
Picking Robots

30. Sample Video Applications
30
Digital
Entertainment
Logistics
CLOUD COMPUTING

31. Videos Everywhere
31
CLOUD COMPUTING OPERATING SYSTEM
DATATECHNOLOGY TEAM
Online and Mobile Commerce Platform
$
BUYERSSELLERS
Marketing
Financing &
Payment
Solution
Professional
Service Providers
Digital
Entertainment
Logistics
Mobile Browser
Location-BasedSocial Media
Weibo

32. DAMO Academy (including iDST):
$15B investment on R&D to solve “big problems”

33. Thank You!
iDST@Alibaba