Neural networks and google tensor flow

2. Outline  Intro to Neural Networks  Inspiration  Basic explanation  Practical applications  Google TensorFlow  What is it?  Basic Usage  Examples

3. Neural Network Inspiration  Real Neurons  Dendrites receive an input  Based on input, axons output something to next neuron

4. Artificial Neural Networks  Set of nodes connected by directional lines representing weights  Nodes represent mathematical operations  Weights learned by training

5. Linear Regression Example  No hidden layers  Inputs * Weights = Output  Weights selected that minimize the error  Great at modeling linear relationships

6. Adding Hidden Layers  Add hidden layer(s)  Input x Weights1 = Hidden Layer  Hidden Layer * Weights2 = Output  Weights selected to minimize error  Can model more complex relationships

7. Learning weights  Back Propagation  Errors are back propagated through the model  Determines the errors at each neuron in the network  Gradient Descent  Optimization method  Determine how to change the weights  Takes a step down gradient of the function  Iterative process

9. Other Architectures  Recurrent Neural Networks  Output from first inputs fed back into network  Used to predict output on future examples  Text processing/prediction

10. Other Architectures  Convolutional Neural Networks  Image processing/classification

11. Practical Applications  Pattern Recognition  Image and text processing  Time series prediction  Stock market and weather forecasting  Anomaly Detection  Bank fraud  Signal Detection  Noise filtering

12. Google TensorFlow  Open source machine learning library  Released November 9, 2015

13. Features  Can be used on desktop, mobile, servers  Linus or Mac OS X  GPU support on Linux  Written in C++ with Python interface  Excellent step by step tutorials and documentation  Auto-differentiation  Includes Tensorboard for graph visualization  Active improvement and growth  Google cloud (March 2016)  Distributed computing support (April 2016)

14. Basics  Data flow graph with nodes and edges  Nodes: mathematical operations  Edges: input/output relationship between nodes  Edges carry tensors Tensors flow through the graph

15. Building a model “TensorFlow programs are usually structured into a construction phase, that assembles a graph, and an execution phase that uses a session to execute ops in the graph.” - TensorFlow docs  Define computation graph  Inputs, operations, outputs  Symbolic representation of model  Run session  Execute graph  Fetch output

16. Simplest Example 6.0 7.0 mul 42

17. Simple Example d e add f

18. More Complex Examples  MNIST data set  70,000 Handwritten digits  28x28 pixels  Used for benchmarking machine learning algorithms  input_data.py  mnist.train = 55,000  mnist.validation = 5,000  mnist.test = 10,000

19. Softmax / Logistic Regression

20. accuracy = .9213

21. Neural Network Implementation

22.

23. accuracy = .9657

24.

25. Additional Resources  TensorFlow Tutorials  Udacity Deep Learning Course  Awesome TensorFlow  TensorFlow Examples  WildML  TF Learn (Scikit Flow)  Keras  Standford CS224d Lecture 7  TensorBoard  TensorFlow Playground

Editor's Notes

sequential processing of inputs Unreasonable effectiveness of recurrent neural networks
2012 Image Net only 1 CovNet – winner error 16.4% vs 26.2% future Image Net challenges almost all entries used CovNets -in some ways similar to our own cortex
train has 55,000 images so 550 batches
55,000 examples in train 55,000 / 100 = 550 550 * 15 = 8250 training cycles

Neural networks and google tensor flow

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Neural networks and google tensor flow

Similar to Neural networks and google tensor flow (20)

Recently uploaded

Recently uploaded (20)

Neural networks and google tensor flow

Editor's Notes