TensorFlow open sourced recently by Google is one of the key frameworks that support development of deep learning architectures. In this slideset, part 1, we get started with a few basic primitives of TensorFlow. We will also discuss when and when not to use TensorFlow.

1. TENSORFLOW FOR DEEP
LEARNING – PART 1
Anantharaman Palacode Narayana Iyer
JNResearch
ananth@jnresearch.com
15 April 2016

2. REFERENCES

3. AGENDA
 Tensorflow overview
 When should we use it?
 High level steps
 Basic operations to get started

4. WHAT ISTENSORFLOW?
 From the whitepaper: “TensorFlow is an interface for expressing machine
learning algorithms, and an implementation for executing such
algorithms.”
 My perspective:
 Tensorflow is a framework with the following characteristics:
 A Framework that allows a developer to express his machine learning algorithm
symbolically, performing compilation of these statements and executing them.
 A programming metaphor that requires the developer to model the machine
learning algorithm as a computation graph.
 A set of Python classes and methods that provide an API interface
 A re-targetable system that can run on different hardware

5. WHY USETENSORFLOW?
 Suppose we are addressing a problem using a machine learning approach and our proposed
solution can be implemented by one of the standard classifier architectures. (say, an SVM)
 We often use one of the standard ML libraries (say, scikit) for the above situation instead of
implementing our own classifier
 If the existing ML models aren’t optimally suited for our problem, we might need to come
up with a more custom architecture that might give a higher accuracy. In such a case we
might end up building the classifier from scratch, starting with mathematical formulations
and implementing ground up.
 Problems that call for deep learning oriented solutions often require custom architectures.
 A lego-piece based approach where a library offers basic building blocks would be more
suited.With in-built support to run on GPUs, such libraries are well suited for custom designs
 WithTensorflow’s building blocks, you can quickly build arbitrarily complex architectures

6. TENSORFLOW COMPUTE GRAPH
 Operations are the nodes of the graph
 Tensors constitute the edges
h1 h2 h3 h4
y1 y2 y3
WS
𝑃 𝑦 ℎ = 𝑆𝑜𝑓𝑡𝑚𝑎𝑥 𝑑𝑜𝑡_𝑝𝑟𝑜𝑑
𝑑𝑜𝑡_𝑝𝑟𝑜𝑑 = 𝑊 𝑆
ℎ + 𝑏
matmul
WS h
Add
Softmax
b

7. DEVELOPING WITHTENSORFLOW: HIGH
LEVEL STEPS
Build a Graph
Initialize a
Session
Run the
Session
 Perform any pre-processing steps to set up the dataset for
the classifier.
 Set up the variables (such as the Weights and biases) and
placeholders (such as the input vector) that are fed at runtime
 Define the required computations. E.g. you may define the
dot_product computation using the matmul method of
tensorflow feeding it the weight matrix variable and the input
vector (that may be defined as a placeholder).
Note:Till this point nothing is executed byTensorflow
 Start a session instance and initialize all variables
 Run the session

8. SOME METHODSTO GET STARTED
 tf.constant
 tf.placeholder
 tf.variable
 tf.matmul( )
 tf.softmax( )
 tf.reduce_sum( )
 tf.train.GradientDescentOptimizer( )
 tf.initialize_all_variables( )
 tf.Session( )
 tf.run( )

9. EXAMPLE 1

10. EXAMPLE 2

11. SOME OBSERVATIONS
 The major benefits of using tensorflow to develop deep learning algorithms include:
 A good set of lego pieces that includes methods that support different types of output layers, activation
functions, convolution, pooling and so on.This greatly helps when you are researching with custom
architectures.
 Support for GPU:This helps in realizing better performance
 Strong type checking: tensorflow implements tighter type checks compares to numpy. For instance
when you try to mix float32 and float64 types, it throws an error. As tensorflow performs its own
compilation, given the computation graph, it’s possibly imposing tighter compile time checks.
 Support from third party – as the product is well invested by Google, we might see a large number of
third party libraries, code that aid our development process
 Tensorflow needs some initial effort to get over the learning curve. Initially, debugging
might also be a challenge.