ACCV 2014 Keynote Slides. See http://www.accv2014.org/ Title: How Changing Mobile and Media Technologies is Changing The Way We Create Innovations According to Schumpeter's definition of "Innovation," all the innovation instances are combinations of technologies that already exist. In that context, this talk covers the combination of progress of mobile network technologies and media understanding technologies. When the first mobile phones of 2nd Generation came out, people thought it was only for speech communication and texting. Then the phones got cameras, GPS. accelerometers, near filed communication devices together with fat communication pipes, people in all walks of life started using mobile phones in various opportunities. It is noteworthy that the progress of media understanding applications remarkably creates big data in a way of virtuous cycle by service and technology developments.　This talk also highlights big-data driven service developments and API strategies for mobile innovations as well as technologies.

1. © 2014 NTT DOCOMO, INC. All rights reserved.
How Changing Mobile and Media Technologies
is Changing The Way We Create Innovations
Mick Etoh
@mickbean
NTT DOCOMO
1

2. © 2014 NTT DOCOMO, INC. All rights reserved.
Outline
Big-Data, Big-Server, and Fat-Pipe
Solutions
Large-scale Signals from Wireless
Infrastructure
APIs for Federated Systems
Definition of Innovation
Progress of Cellphone and Network
Technologies
2

3. © 2014 NTT DOCOMO, INC. All rights reserved.
Who am I?
Digital VCR Development (1985)
Artificial Intellligence (1988)
Computer
Vision and PR
(1991)
Image Coding (1994)
Mobile Multimedia
(2000)
Data Mining and NLP (2007-)
Speech Recognition(2006) Machine Translation(2014-)
Mobile
Network
(2002-)
Investment(2012-)
3

4. © 2014 NTT DOCOMO, INC. All rights reserved.
Connecting the dots….
4
Those experiences gave me the ability of

5. © 2014 NTT DOCOMO, INC. All rights reserved.
5
Those experiences gave me the ability of
Connecting the technologies….

6. © 2013 NTT DOCOMO, INC. All rights reserved.
What is Innovation?
Joseph Schumpeter‘s
“neuer Kombinationen”
(Theorie der wirtschaftlichen Entwicklung)
6

7. © 2014 NTT DOCOMO, INC. All rights reserved.
7
“All technologies are combinations of
technologies that already exist.” —
• Combinatorial Explosion
• Creates Economy
W. Brian Arthur

8. Innovation Example
2.5G Internet connectivity,
DOCOMO
1998
Micro Browser,
Access Co.
1995
i-mode,
DOCOMO
1999
New Biz. Concept:
Content Provider Bazar
8
© 2014 NTT DOCOMO, INC. All rights reserved.

9. Wireless Technologies, the Enablers
Personal Storage
SNS/Comm. Tools
LTE:～75Mbps2010
+Big data synchronization
Application Market
(games, net contents, etc.)
3.5G: ～14Mbps 2006
+Smart Phone + Cloud
Mobile Multimedia
3G: 64-384Kbps 2001
+CODEC
i-mode
2G:9.6KbpsX3 1998+micro browser
9
© 2014 NTT DOCOMO, INC. All rights reserved.

10. © 2013 NTT DOCOMO, INC. All rights reserved.
Very Simplified Formula
Wireless
Broadband
Innovation X
?
10
❌

11. © 2014 NTT DOCOMO, INC. All rights reserved.
Distributed Speech Recognition(DSR)
in 2006
First Implementation of ETSI AURORA Project (2000-2003)
16 Khz Sampling MFCC feature vectors --> 5.6 Kbps
Mobile Terminal
Feature
Extraction
Result
Encode
Server
Decode Rec.
Engine
Dictionary
11

12. © 2014 NTT DOCOMO, INC. All rights reserved.
Lessons Learned From the DSR(2006)
12
Pros
• 16Khz sampling outperforms 8Khz Sampling.
• Dedicated Feature Extraction Front End outperforms
the legacy IVR (Interactive Voice Response) methods.
Cons
• Channel assumption was too narrow (i.e., Kbps-order).
• No background noise model (left intact)
Bandwidth is not crucial.

13. Mobile Issue: Battery
13
© 2014 NTT DOCOMO, INC. All rights reserved.
3X energy density evolution
40MHz
ARM(2001)
Mbps
Broadband
Quad-Core
1GHz+
ARM(2014)
250X Improved
+

14. © 2014 NTT DOCOMO, INC. All rights reserved.
Thin Client Architecture with Fat Pipe
and Big Server since 2007
14

15. © 2014 NTT DOCOMO, INC. All rights reserved.
LTE-Advanced Network （２０１６-）
Macro Cell Base Station
Small Cell Base Station
Centeraized Radio Access
Network
15

16. © 2014 NTT DOCOMO, INC. All rights reserved.
Evolution of
Wireless Technologies
16
10
kbps
100
kbps
1
Mbps
10
Mbps
100
Mbps
1
Gbps
10
Gbps
2020
5G
2015201020052000
LTE
LTE-­‐A
HSDPA,
HDRWCDMA,
CDMA2000
MIMO
Carrier
Aggregation/
MU-­‐MIMO
3D/Massive
MIMO
∼3.5GHz
∼60GHz
CDMA
OFDM/OFDMA
Small
Cell

17. © 2014 NTT DOCOMO, INC. All rights reserved.
Smart Phones as Information Hubs
WiFi
NFC
Bluetooth
Gyro
Microphone
CCD/CMOS Camera
GPS
Accelerometer
Cloud
Wireless Broadband
Smart Phone
17

18. © 2014 NTT DOCOMO, INC. All rights reserved.
Camera+Snapdragon+Vuforia(FastCV)+Unity
18
Ball-in-a-maze puzzle
See youtube video.

19. © 2014 NTT DOCOMO, INC. All rights reserved.
19
https://play.google.com/store/apps/
details?id=jp.co.sompo_japan.safetysight
iOnRoad
Safety Sight(2014)
http://www.ionroad.com/

20. © 2014 NTT DOCOMO, INC. All rights reserved.
History of Cellphone Sensors
20
Barometric
Magnetometer
Accelerometer
Fingerprint
Co-Processors
GPS
2005 2010 2015
Navigation
Power reduction
biometrics
Navigation Compass
altimeter, weather forecast
pedometer, game controler
Mar/03
July/03
April/04
July/06
June/12
May/13
Gyroscope Image StabilizationNov./09

21. © 2014 NTT DOCOMO, INC. All rights reserved.
Coprocessor solution (iphone 5s)
21
A6 Magnetometer
Accelerometer
Gyroscope
A7 M7 Magnetometer
Accelerometer
Gyroscope

22. © 2014 NTT DOCOMO, INC. All rights reserved.
Sensor Fusion
22
3 axis gyroscope 2000°/second precision
3 axis accelerometer +-50mg precision
3 axis magnetometer 6° precision
Pressure sensor +/- 10 Pa precision
Ultrasound sensors for ground altitude measurement
60 fps vertical QVGA camera for ground speed measurement
http://ardrone2.parrot.com/ardrone-2/specifications/

23. © 2014 NTT DOCOMO, INC. All rights reserved.
Flying Cell Phone: Androcopter
23
http://diydrones.com/profiles/blogs/androcopter-use-you-smartphone-as-an-autopilot

24. © 2014 NTT DOCOMO, INC. All rights reserved.
Siri 24
Big-Data, Big-Server, and Fat-Pipe Solution

25. © 2014 NTT DOCOMO, INC. All rights reserved.
“Shabette-Concier” Voice agent
• Launched Mar. 1, 2012
• Over 40 services in it
• Including chatting
• 10 million users
Shabette
Voice
=
Concier
Concierge
=
How may I help you?
25

26. © 2014 NTT DOCOMO, INC. All rights reserved.Touch the Concier.Q: “What is the height of Mt. Fuji?”A: “3,766m!”Q: “When is holding schedule of the Tokyo Olympic Games?”A: “It will hold in 2020.” 26

27. © 2014 NTT DOCOMO, INC. All rights reserved.
Basic Architecture 2010
Logging
Voice
Recognition
Task
Recognition
Logging
Voice
text
text contents
Service
Providers’ DB
contents
text
Text to speech
27
Fat-Pipe
Big-Servers

28. © 2014 NTT DOCOMO, INC. All rights reserved.
BELIEF: More data usually beats better
algorithms.
Speech
Speech
Data
Language
Data
Machine Learning Machine Learning
Acoustic
Model
Lexicon
Language
Model
Speech Recognition I want...
Text
Voice Recognition
28
Big-Data

29. Performance Improvement by Data
Precision(%)
40
50
60
70
80
90
Text Data Amount in Million Lines
1 10 100 1000
29
© 2014 NTT DOCOMO, INC. All rights reserved.

30. © 2014 NTT DOCOMO, INC. All rights reserved.
Virtuous Circle Enabled by Machine Learning
Improved
Performance
Better
User Experience More Data
Machine
Learning
30

31. Signal Processing in Mobile Comm.
Signals From Each Mobile Device about
audio, image, video, and extra sensors for
Applications: Information Retrieval, Navigation, etc.
31
© 2014 NTT DOCOMO, INC. All rights reserved.
Signals From Gigantic Social Infrastructure about
crowds, traffic, earthquake, and weather for
Applications: Large-scale Infrastructure
Optimization and Human Security

32. © 2014 NTT DOCOMO, INC. All rights reserved.
Earthquake Early Warning System(2007)
32
Mobile
Telecommunication
System
TV Broadcasting
System

33. © 2014 NTT DOCOMO, INC. All rights reserved.
Earthquake Early Warning System
33

34. © 2014 NTT DOCOMO, INC. All rights reserved.
PM 14:46, March 11th, 2011
34
http://pics.livejournal.com/druzifer/pic/004q2qar
Great Eastern
Japan Earthquake

35. © 2014 NTT DOCOMO, INC. All rights reserved.
35
Connecting the technologies
through APIs…

36. © 2014 NTT DOCOMO, INC. All rights reserved.
36
+ Character Recognition,Environment Sensor,Data Storage BOX,Location
Information, Mobile Payment, and more.
API as Catalysts

37. © 2014 NTT DOCOMO, INC. All rights reserved.
Yet another Google Goggle/Amazon Flow
37https://twitter.com/ksasao

38. © 2014 NTT DOCOMO, INC. All rights reserved.
38
API Aggregator
Mashape
is
the
largest
API
marketplace,
powering
thousands
of
APIs
and
Applications.

39. © 2014 NTT DOCOMO, INC. All rights reserved.
39
Rank.25
Rank.34
Rank.39
Rank.43
Rank.16 ICCV2013
Rank.6 Googleglass

40. © 2014 NTT DOCOMO, INC. All rights reserved.
40
Rank.51
Rank.57
Rank.62
Rank.77
Rank.85
Rank.87

41. © 2014 NTT DOCOMO, INC. All rights reserved.
Take aways for “connecting the dots.”
41
Wireless
Broadband
Innovation X?❌
Big-Server, Big-Data, and Fat-Pipe
for Smart Thin clients
Virtuous Circle by Machine Learning
Large-scale Signals from Wireless
Infrastructure
APIs as Connecters and Indicators of
your inventions.

42. © 2014 NTT DOCOMO, INC. All rights reserved.
42
Design of combination
from existing and
foreseeable
technologies
Research for Impact factor

43. © 2014 NTT DOCOMO, INC. All rights reserved.
Signals from GPS around Tokyo in
a normal and non-eventful day. 43
My dream is …to connect the
systems.
Innovation
X
Speech Dialogue
System
Computer Vision
System
unknown?