The Microsoft Cognitive Toolkit (CNTK) is a unified deep-learning toolkit that describes neural networks as a series of computational steps via a directed graph. In this directed graph, leaf nodes represent input values or network parameters, while other nodes represent matrix operations upon their inputs. The objectives of this presentation is to 1) describe what CNTK is, 2) present a comparative evaluation with similar technologies, 3) outline potential applications, and 4) demonstrate the technology with Jupyter Python examples.

2. With many contributors:
A. Agarwal, E. Akchurin, E. Barsoum, C. Basoglu, G. Chen, S. Cyphers, W. Darling, J. Droppo, A. Eversole, B. Guenter, M. Hillebrand, X. Huang, Z. Huang,
R. Hoens, V. Ivanov, A. Kamenev, N. Karampatziakis, P. Kranen, O. Kuchaiev, W. Manousek, C. Marschner, A. May, B. Mitra, O. Nano, G. Navarro, A. Orlov,
S. Pathak, M. Radmilac, A. Reznichenko, P. Parthasarathi, B. Peng, A. Reznichenko, W. Richert, F. Seide, M. Seltzer, M. Slaney, A. Stolcke, T. Will, H. Wang,
W. Xiong, K. Yao, D. Yu, C. Zhang, Y. Zhang, G. Zweig

3. Objectives
• Describe the Microsoft Cognitive Toolkit (CNTK)
• Comparative evaluation with similar technologies
• Outline potential applications
• Show Jupyter Python demos

4. Assumptions
• Intermediate knowledge of programming
• Intermediate knowledge of data science or machine learning
• Stay Calm: You may fill in gaps later

5. What is Cognitive Toolkit (CNTK)

6. The Microsoft Cognitive Toolkit (CNTK)
• Microsoft’s open-source deep-learning toolkit
• ease of use: what, not how
• fast
• flexible
• Supported on both Linux and Windows
• Fully open development framework

7. • CNTK is production-ready: State-of-the-art accuracy, efficient, and scales
to multi-GPU/multi-server.
• CNTK expresses (nearly) arbitrary neural networks by composing simple
building blocks into complex computational networks, supporting
relevant network types and applications.
The Microsoft Cognitive Toolkit (CNTK)

8. Microsoft’s historic
speech breakthrough
• Microsoft 2016 research system for
conversational speech recognition
• 5.9% word-error rate
• enabled by CNTK’s multi-server scalability
[W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke,
D. Yu, G. Zweig: “Achieving Human Parity in Conversational
Speech Recognition,” https://arxiv.org/abs/1610.05256]

9. Microsoft’s historic
speech breakthrough
• Microsoft 2017 research system for
conversational speech recognition
• 5.1% word-error rate
• “We introduced an additional CNN-BLSTM
(convolutional neural network combined with
bidirectional long-short-term memory) model for
improved acoustic modeling."
• Additionally, Microsoft enlisted its suite of AI
technologies called the Cognitive Toolkit (CNTK) in its
record-setting bid.

11. Comparative Evaluation

12. 28.2
25.8
16.4
11.7
7.3 6.7
3.6
0.0
5.0
10.0
15.0
20.0
25.0
30.0
ILSVRC 2010
NEC America
ILSVRC 2011
Xerox
ILSVRC 2012
AlexNet
ILSVRC 2013
Clarifi
ILSVRC 2014
VGG
ILSVRC 2014
GoogleNet
ILSVRC 2015
ResNet
ImageNet Classification top-5 error (%)
Microsoft had all 5 entries being the 1-st
places in 2015: ImageNet classification,
ImageNet localization, ImageNet
detection, COCO detection, and COCO
segmentation

13. CNTK Benchmarked by HKBU (September 2016)
• CNTK’s LSTM performance is 5-10x faster than the other toolkits.
• For convolution (image tasks), CNTK is comparable, but note the
authors were using CNTK 1.7.2, and current CNTK 2.0 beta 10 is
over 30% faster than 1.7.2.
• For all networks, CTNK's performance was superior to
TensorFlow performance.

14. Benchmarking on a single server by HKBU – September 2016
“CNTK is production-ready: State-of-the-art accuracy, efficient,
and scales to multi-GPU/multi-server.”
FCN-8 AlexNet ResNet-50 LSTM-64
CNTK 0.037 0.040 (0.054) 0.207 (0.245) 0.122
Caffe 0.038 0.026 (0.033) 0.307 (-) -
TensorFlow 0.063 - (0.058) - (0.346) 0.144
Torch 0.048 0.033 (0.038) 0.188 (0.215) 0.194
G980

15. Frames Processing
Source: https://github.com/Microsoft/CNTK/blob/master/Documentation/Documents/PerformanceChart.png
• The figure compares processing
speed (frames processed per
second) of CNTK to that of four
other well-known toolkits.
• The configuration uses a fully
connected 4-layer neural network
(see our benchmark scripts) and
an effective mini batch size
(8192).
• All results were obtained on the
same hardware with the
respective latest public software
versions as of Dec 3, 2015.

16. Benchmarking CNTK on Keras: is it Better at Deep
Learning than TensorFlow?
• Deciding the better Keras framework is not as
clear cut as expected.
• Accuracy is mostly identical between the two
frameworks. CNTK is faster at LSTMs/MLPs,
TensorFlow is faster at CNNs/Embeddings
• When networks implement both, it’s a tie.
http://minimaxir.com/2017/06/keras-cntk/

17. Potential Applications

18. Deep learning at Microsoft
• Microsoft Cognitive Services
• Skype Translator
• Cortana
• Bing
• Bing Ads
• Augmented Reality
• Microsoft Research

19. Intelligent Kiosk Demo

20. Intelligent Kiosk
• https://github.com/Microsoft/Cognitive-Samples-
IntelligentKiosk

21. CNTK Code

22. Jupyter
http://jupyter.org/

23. Github
• Equifax may have given
away my birth date, my
social security number, and
my credit cards
• But they’re not getting into
my Github

24. • Github: https://github.com/Microsoft/CNTK
• Wiki: https://github.com/Microsoft/CNTK/wiki
• Issues: https://github.com/Microsoft/CNTK/issues
Microsoft Cognitive Toolkit

25. Microsoft
Cognitive
Toolkit
Examples
https://github.com/Microsoft/CNTK

26. Microsoft MML Spark

27. Jupyter Python Demos

28. Apache Spark
• SPARK2017
https://www.wintellectnow.com/Home/Instructor?instructorId=Mark_Tabladillo

29. Summary of CNTK
• Microsoft’s open-source deep-learning toolkit
• Supported on both Linux and Windows
• Fully open development framework

30. Abstract
The Microsoft Cognitive Toolkit (CNTK) is a unified deep-learning toolkit
that describes neural networks as a series of computational steps via a
directed graph. In this directed graph, leaf nodes represent input values
or network parameters, while other nodes represent matrix operations
upon their inputs.
The objectives of this presentation is to 1) describe what CNTK is, 2)
present a comparative evaluation with similar technologies, 3) outline
potential applications, and 4) demonstrate the technology with Jupyter
Python examples.