Martin Musiol - Style-transfer with TensorFlow - Codemotion Milan 2018

•

0 likes•368 views

Google’s TensorFlow drew considerable attention in recent months as it enabled rapid Machine Learning development. In response, Google developed TensorFlow Lite - a solution for deploying intelligent applications on mobile and embedded devices. We will show how to use TensorFlow Lite to develop an app which transforms the style of pictures using a deep neural network. Style-transferring apps can transform summer into winter, horses into zebras, or a Monet into a Van Gogh. In this talk, we see rapid development of intelligent mobile device products which may or may not have a lasting impact.

Technology

NOVEMBER 2018
Style-transfer
with TensorFlow
Transforming the style of images with
TensorFlow (lite) for mobile devices.
Martin Musiol

What is Generative AI?
Generative vs. Discriminative
11

12
WHAT IS GENERATIVE AI?
This bird is red
with white and
has a very short
beak.

Where are we?
AI - Artiﬁcial Intelligence
ML - Machine Learning
DL - Deep Learning
Gen. AI - Generative AI
Style-transfer
A
I
ML
DL
Gen. AI
15

Self-Learning
Algorithms
Non-Self-
Learning
Algorithm
s
ML
DL Self-Learning Algorithms
16

19
NEURAL STYLE-TRANSFER
The Three Ingredients
1. Autoencoder for image reconstruction

20
NEURAL STYLE-TRANSFER
1. Autoencoder for image reconstruction
2. Compressed representation
The Three Ingredients

21
NEURAL STYLE-TRANSFER
1. Autoencoder for image reconstruction
2. Compressed representation
3. Transforming features
The Three Ingredients

22
NEURAL STYLE-TRANSFER
1) Autoencoder for image reconstruction

23
NEURAL STYLE-TRANSFER
2) Compressed representation

24
NEURAL STYLE-TRANSFER
2) Compressed representation

25
NEURAL STYLE-TRANSFER
3) Transforming features

26
NEURAL STYLE-TRANSFER
3) Transforming features

27
NEURAL STYLE-TRANSFER
3) Transforming features
Content
Style

28
NEURAL STYLE-TRANSFER
3) Transforming features
Content
Style
Feature
Transforms

29
NEURAL STYLE-TRANSFER
3) Transforming features
Content
Style
Feature
Transforms
1) Whitening
Whitening: keep only shapes

Coloring: keep the colors

Stylization: how much shape from
content and how much shape from
style picture.
EXTRACT SHAPE

30
NEURAL STYLE-TRANSFER
3) Transforming features
Content
Style
Feature
Transforms
1) Whitening
2) Coloring
Whitening: keep only shapes

Coloring: keep the colors

Stylization: how much shape from
content and how much shape from
style picture.
EXTRACT COLORS

31
NEURAL STYLE-TRANSFER
3) Transforming features
Content
Style
Feature
Transforms
1) Whitening
2) Coloring
3) Stylization
Whitening: keep only shapes

Coloring: keep the colors

Stylization: how much shape from
content and how much shape from
style picture.
CONTENT SHAPE/ STYLE SHAPE

TENSORFLOW LITE
• Lower latency, no server calls
Why on-device ML?

TENSORFLOW LITE
• Lower latency, no server calls
• Works offline
Why on-device ML?

TENSORFLOW LITE
• Lower latency, no server calls
• Works offline
• Data stays on-device
Why on-device ML?

TENSORFLOW LITE
• Lower latency, no server calls
• Works offline
• Data stays on-device
• Power efficient
Why on-device ML?

TENSORFLOW LITE
• Lower latency, no server calls
• Works offline
• Data stays on-device
• Power efficient
• All sensor data accessible on-device
Why on-device ML?

• Core operations tuned for mobile platforms
Highlights
TENSORFLOW LITE

• Core operations tuned for mobile platforms
• FlatBuffers-based model file format
Highlights
TENSORFLOW LITE
The primary beneﬁt of FlatBuﬀers
comes from the fact that they can be
memory-mapped, and used directly
from disk without being loaded and
parsed. This gives much faster
startup times, and gives the
operating system the option of
loading and unloading the required
pages from the model ﬁle, instead of
killing the app when it is low on
memory.
SERIALIZATION LIBRARY FOR :
• JAVA
• C/C++
• PYTHON
• ETC.

• Core operations tuned for mobile platforms
• FlatBuffers-based model file format
• On-device interpreter with kernels optimised
for fast execution on mobile
Highlights
TENSORFLOW LITE

• Java API for convenience
Highlights
TENSORFLOW LITE

• Java API for convenience
• C++ API: loads model and invokes interpreter
Highlights
TENSORFLOW LITE

• Java API for convenience
• C++ API: loads model and invokes interpreter
• Interpreter executes model using set of
kernels
Highlights
TENSORFLOW LITE

The new graph converter in TF Lite is called
TensorFlow Lite Optimizing Converter aka
TOCO
How to transfer TF to
TF Lite?
TENSORFLOW LITE
How to call it in demo shortly!

FROG IS A COMPANY OF ARICENT © 2017 FROG DESIGN INC.
Thank you.
Martin Musiol

Similar to Martin Musiol - Style-transfer with TensorFlow - Codemotion Milan 2018

Short The growing amount of data captured by sensors and the real time constraints imply that not only big data analytics but also Machine Learning (ML) inference shall be executed at the edge. The multiple options for neural network acceleration in Arm-based platforms provide an unprecedented opportunity for new intelligent devices. It also raises the risk of fragmentation and duplication of efforts when multiple frameworks shall support multiple accelerators. Andrea Gallo, Linaro VP of Segment Groups, will summarise the existing NN frameworks, accelerator solutions, and will describe the efforts underway in the Arm ecosystem. Abstract The dramatically growing amount of data captured by sensors and the ever more stringent requirements for latency and real time constraints are paving the way for edge computing, and this implies that not only big data analytics but also Machine Learning (ML) inference shall be executed at the edge. The multiple options for neural network acceleration in recent Arm-based platforms provides an unprecedented opportunity for new intelligent devices with ML inference. It also raises the risk of fragmentation and duplication of efforts when multiple frameworks shall support multiple accelerators. Andrea Gallo, Linaro VP of Segment Groups, will summarise the existing NN frameworks, model description formats, accelerator solutions, low cost development boards and will describe the efforts underway to identify the best technologies to improve the consolidation and enable the competitive innovative advantage from all vendors. Audience The session will be useful for executives to engineers. Executives will gain a deeper understanding of the issues and opportunities. Engineers at NN acceleration IP design houses will take away ideas for how to collaborate in the open source community on their area of expertise, how to evaluate the performance and accelerate multiple NN frameworks without modifying them for each new IP, whether it be targeting edge computing gateways, smart devices or simple microcontrollers. Benefits to the Ecosystem The AI deep learning neural network ecosystem is starting just now and it has similar implications with open source as GPU and video accelerators had in the early days with user space drivers, binary blobs, proprietary APIs and all possible ways to protect their IPs. The session will outline a proposal for a collaborative ecosystem effort to create a common framework to manage multiple NN accelerators while at the same time avoiding to modify deep learning frameworks with multiple forks.

Deep Learning Neural Network Acceleration at the Edge - Andrea Gallo

Linaro

What drives Innovation? Innovations And Technological Solutions for the Distr...

Stefano Fago

Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...

Codemotion

Deep learning with TensorFlow

Ndjido Ardo BAR

This issue’s feature article, Tuning Autonomous Driving Using Intel® System Studio, illustrates how the tools in Intel System Studio give embedded systems and connected device developers an integrated development environment to build, debug, and tune performance and power usage. Continuing the theme of tuning edge applications, Building Fast Data Compression Code for Cloud and Edge Applications shows how to use the Intel® Integrated Performance Primitives to speed data compression.

Parallel universe-issue-29

DESMOND YUEN

Machine Learning on Your Hand - Introduction to Tensorflow Lite Preview

Modulabs

Persian MNIST in 5 Minutes

Shahriar Yazdipour

Develop a fundamental overview of Google TensorFlow, one of the most widely adopted technologies for advanced deep learning and neural network applications. Understand the core concepts of artificial intelligence, deep learning and machine learning and the applications of TensorFlow in these areas. The deck also introduces the Spotle.ai masterclass in Advanced Deep Learning With Tensorflow and Keras.

Introduction To TensorFlow

Spotle.ai

This is the extended presentation about byteLAKE's and Lenovo's Artificial Intelligence solutions for Manufacturing. Topics covered: AI strategy for manufacturing, Edge AI, Federated Learning and Machine Vision. It's the first publication in the upcoming series: AI for Manufacturing. Highlights: AI-assisted quality monitoring automation, AI-assisted production line monitoring and issues detection, AI-assisted measurements, Intelligent Cameras and many more. Reach out to us to learn more: welcome@byteLAKE.com. Presented during the world's first Federated Learning conference (Jun'20). Recording: https://youtu.be/IMqRIi45dDA Related articles: - Revolution in factories: Industry 4.0. https://medium.com/@marcrojek/revolution-in-factories-industry-4-0-conference-made-in-wroclaw-2020-translation-ae96e5e14d55 - Cognitive Automation helps where RPAs fall short. https://medium.com/@marcrojek/cognitive-automation-helps-where-rpas-fall-short-a1c5a01a66f8 - Machine Vision, how AI brings value to industries. https://medium.com/@marcrojek/machine-vision-how-ai-brings-value-to-industries-e6a4f8e56f42 Learn more: - https://www.bytelake.com/en/cognitive-services/ - https://www.lenovo.com/ai - https://federatedlearningconference.com/

AI for Manufacturing (Machine Vision, Edge AI, Federated Learning)

byteLAKE

Ibm db2update2019 machine learning and db2 ai

Gustav Lundström

Intel Deep Learning SDK enables using of optimized open source deep-learning frameworks, including Caffe and TensorFlow through a step-by-step wizard or iPython interactive notebooks. It includes easy and fast installation of all depended libraries and advanced tools for easy data pre-processing and model training, optimization and deployment, providing an end-to-end solution to the problem. In addition, it supports scale-out on multiple computers for training, as well as using compression methods for deployment of the models on various platforms, addressing memory and speed constraints.

Faster deep learning solutions from training to inference - Michele Tameni - ...

Codemotion

Deep Dive Time Series Anomaly Detection in Azure with dotnet

Marco Parenzan

Introduction to TensorFlow Lite

Koan-Sin Tan

ITCamp 2011 - Adrian Stoian - System Center Configuration Manager 2012

ITCamp

Ezako is a startup specializing in time series analysis. Ezako helps its clients detect anomalies and label their time series data. It helps accelerate the labeling process and analyze vast amounts of data from a variety of sensors in real-time. The company provides anomaly insights and makes it easier for data scientists. Ezako is the creator of Upalgo, which is a time series data management tool that uses AI to automatically detect anomalies in streaming data. During this webinar, Ezako will dive into how high-frequency sensors can generate huge amounts of data which can become desynchronized. This can result in data quality issues as it can contain errors and glitches. Ezako uses machine learning, labelling and feedback loops to identify these errors. Discover how the company helps improve its clients’ data quality and reduce the number of validation mistakes.

How to Improve Data Labels and Feedback Loops Through High-Frequency Sensor A...

InfluxData

Facebook ML Infrastructure - 2018 slides

Karthik Murugesan

Deploying and managing machine learning models at scale introduces new complexities. Fortunately, there are tools that simplify this process. In this talk we walk you through an end-to-end hands on example showing how you can go from research to production without much complexity by leveraging the Seldon Core and MLflow frameworks. We will train a set of ML models, and we will showcase a simple way to deploy them to a Kubernetes cluster through sophisticated deployment methods, including canary deployments, shadow deployments and we’ll touch upon richer ML graphs such as explainer deployments.

Seamless MLOps with Seldon and MLflow

Databricks

Squeezing Deep Learning Into Mobile Phones

Anirudh Koul

open source nn frameworks on cellphones

Koan-Sin Tan

Generative AI models, such as ChatGPT and Stable Diffusion, can create new and original content like text, images, video, audio, or other data from simple prompts, as well as handle complex dialogs and reason about problems with or without images. These models are disrupting traditional technologies, from search and content creation to automation and problem solving, and are fundamentally shaping the future user interface to computing devices. Generative AI can apply broadly across industries, providing significant enhancements for utility, productivity, and entertainment. As generative AI adoption grows at record-setting speeds and computing demands increase, on-device and hybrid processing are more important than ever. Just like traditional computing evolved from mainframes to today’s mix of cloud and edge devices, AI processing will be distributed between them for AI to scale and reach its full potential. In this presentation you’ll learn about: - Why on-device AI is key - Full-stack AI optimizations to make on-device AI possible and efficient - Advanced techniques like quantization, distillation, and speculative decoding - How generative AI models can be run on device and examples of some running now - Qualcomm Technologies’ role in scaling on-device generative AI

Generative AI at the edge.pdf

Qualcomm Research

Similar to Martin Musiol - Style-transfer with TensorFlow - Codemotion Milan 2018 (20)

Deep Learning Neural Network Acceleration at the Edge - Andrea Gallo

What drives Innovation? Innovations And Technological Solutions for the Distr...

Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...

Deep learning with TensorFlow

Parallel universe-issue-29

Machine Learning on Your Hand - Introduction to Tensorflow Lite Preview

Persian MNIST in 5 Minutes

Introduction To TensorFlow

AI for Manufacturing (Machine Vision, Edge AI, Federated Learning)

Ibm db2update2019 machine learning and db2 ai

Faster deep learning solutions from training to inference - Michele Tameni - ...

Deep Dive Time Series Anomaly Detection in Azure with dotnet

Introduction to TensorFlow Lite

ITCamp 2011 - Adrian Stoian - System Center Configuration Manager 2012

How to Improve Data Labels and Feedback Loops Through High-Frequency Sensor A...

Facebook ML Infrastructure - 2018 slides

Seamless MLOps with Seldon and MLflow

Squeezing Deep Learning Into Mobile Phones

open source nn frameworks on cellphones

Generative AI at the edge.pdf

More from Codemotion

Fuzz-testing: A hacker's approach to making your code more secure | Pascal Ze...

Codemotion

Pompili - From hero to_zero: The FatalNoise neverending story

Codemotion

Pastore - Commodore 65 - La storia

Codemotion

Rivivere l'ebbrezza di progettare un vecchio computer o una consolle da bar è oggi possibile sfruttando le FPGA, ovvero logiche programmabili che consentono a chiunque di progettare il proprio hardware o di ricrearne uno del passato. In questa sessione si racconta come dal reverse engineering dell'hardware di vecchie glorie come il Commodore 64 e lo ZX Spectrum sia stato possibile farle rivivere attraverso tecnologie oggi alla portata di tutti.

Pennisi - Essere Richard Altwasser

Codemotion

There's a lot of talk about blockchain, but how does the technology behind it actually work? For developers, getting some hands-on experience is the fastest way to get familiair with new technologies. So let's build a blockchain, then! In this session, we're going to build one in plain old Java, and have it working in 40 minutes. We'll cover key concepts of a blockchain: transactions, blocks, mining, proof-of-work, and reaching consensus in the blockchain network. After this session, you'll have a better understanding of core aspects of blockchain technology.

Michel Schudel - Let's build a blockchain... in 40 minutes! - Codemotion Amst...

Codemotion

When was the last time you were truly lost? Thanks to the maps and location technology in our phones, a whole generation has now grown up in a world where getting lost is truly a thing of the past. Location technology goes far beyond maps in the palm of our hand, however. In this talk, we will explore how a ridesharing app works. How do we discover our destination?How do we find the closest driver? How do we display this information on a map? How do we find the best route?To answer these questions,we will be learning about a variety of location APIs, including Maps, Positioning, Geocoding etc.

Richard Süselbeck - Building your own ride share app - Codemotion Amsterdam 2019

Codemotion

Eward Driehuis, SecureLink's research chief, will guide you through the bumpy ride we call the cyber threat landscape. As the industry has over a decade of experience of dealing with increasingly sophisticated attacks, you might be surprised to hear more attacks slip through the cracks than ever. From analyzing 20.000 of them in 2018, backed by a quarter of a million security events and over ten trillion data points, Eward will outline why this happens, how attacks are changing, and why it doesn't matter how neatly or securely you code.

Eward Driehuis - What we learned from 20.000 attacks - Codemotion Amsterdam 2019

Codemotion

IoT revolution is ended. Thanks to hardware improvement, building an intelligent ecosystem is easier than never before for both startups and large-scale enterprises. The real challenge is now to connect, process, store and analyze data: in the cloud, but also, at the edge. We’ll give a quick look on frameworks that aggregate dispersed devices data into a single global optimized system allowing to improve operational efficiency, to predict maintenance, to track asset in real-time, to secure cloud-connected devices and much more.

Francesco Baldassarri - Deliver Data at Scale - Codemotion Amsterdam 2019 -

Codemotion

What if Virtual Reality glasses could transform your environment into a three-dimensional work of art in realtime in the style of a painting from Van Gogh? One of the many interesting developments in the field of Deep Learning is the so called "Style Transfer". It describes a possibility to create a patchwork (or pastiche) from two images. While one of these images defines the the artistic style of the result picture, the other one is used for extracting the image content. A team from TNG Technology Consulting managed to build an AI showcase using OpenCV and Tensorflow to realize such goggles.

Martin Förtsch, Thomas Endres - Stereoscopic Style Transfer AI - Codemotion A...

Codemotion

Blockchain (and Cryptocurrency) is an evolution of 20-year old research from scientists like Chaum, Lamport, and Castro & Liskov. Due to the current hype, it's hard to distinguish beneficial aspects of the technology from a desire for a "silver bullet" for device security, verifiable logistics, or "saving democracy". The problem: blockchain introduces new security challenges - and blind adoption without understanding reduces overall security. In this talk, Melanie Rieback and Klaus Kursawe explain the pitfalls and limits of blockchain, so you can avoid making your applications LESS secure.

Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...

Codemotion

Networking is a core part of computing in the digital world we inhabit. But, how well do you know how it works? Do you understand all the moving parts of the OSI stack inside your computer, and how the network is actually put together? How can this ever work? This guided safari of layers, standards, protocols, and happenstance will bring us close to the copper wire, and up through the layers of CDMA/CD, ARP, routing and HTTP. We will make a few excursions through patchworks that still work forty years later, and cleverly designed mechanisms that show that simplicity is the only way to last.

Angelo van der Sijpt - How well do you know your network stack? - Codemotion ...

Codemotion

Performance tests are not only an important instrument for understanding a system and its runtime environment. It is also essential in order to check stability and scalability – non-functional requirements that might be decisive for success. But won't my cloud hosting service scale for me as long as I can afford it? Yes, but… It only operates and scales resources. It won't automatically make your system fast, stable and scalable. This talk shows how such and comparable questions can be clarified with performance tests and how DevOps teams benefit from regular test practise.

Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...

Codemotion

Sascha will demonstrate the opportunities and challenges of Conversational AI learned from the practice. Both Technology and User Experience will be covered introducing a process finding micro-moments, writing happy paths, gathering intents, designing the conversational flow, and finally publishing on almost all channels including Voice Services and Chatbots. Valuable for enterprises, developers, and designers. All live on stage in just minutes and with almost no code.

Sascha Wolter - Conversational AI Demystified - Codemotion Amsterdam 2019

Codemotion

A key challenge we face at Pacmed is quickly calibrating and deploying our tools for clinical decision support in different hospitals, where data formats may vary greatly. Using Intensive Care Units as a case study, I’ll delve into our scalable Python pipeline, which leverages Pandas’ split-apply-combine approach to perform complex feature engineering and automatic quality checks on large time-varying data, e.g. vital signs. I’ll show how we use the resulting flexible and interpretable dataframes to quickly (re)train our models to predict mortality, discharge, and medical complications.

Michele Tonutti - Scaling is caring - Codemotion Amsterdam 2019

Codemotion

Coolblue is a proud Dutch company, with a large internal development department; one that truly takes CI/CD to heart. Empowerment through automation is at the heart of these development teams, and with more than 1000 deployments a day, we think it's working out quite well. In this session, Pat Hermens (a Development Managers) will step you through what enables us to move so quickly, which tools we use, and most importantly, the mindset that is required to enable development teams to deliver at such a rapid pace.

Pat Hermens - From 100 to 1,000+ deployments a day - Codemotion Amsterdam 2019

Codemotion

Quantum computers can use all of the possible pathways generated by quantum decisions to solve problems that will forever remain intractable to classical compute power. As the mega players vie for quantum supremacy and Rigetti announces its $1M "quantum advantage" prize, we live in exciting times. IBM-Q and Microsoft Q# are two ways you can learn to program quantum computers so that you're ready when the quantum revolution comes. I'll demonstrate some quantum solutions to problems that will forever be out of reach of classical, including organic chemistry and large number factorisation.

James Birnie - Using Many Worlds of Compute Power with Quantum - Codemotion A...

Codemotion

Chinese food exploded across America in the early 20th century, rapidly adapting to local tastes while also spreading like wildfire. How was it able to spread so fast? The GY6 is a family of scooter engines that has achieved near total ubiquity in Europe. It is reliable and cheap to manufacture, and it's made in factories across China. How are these factories able to remain afloat? Chinese-American food and the GY6 are both riveting studies in product-market fit, and both are the product of a distributed open source-like development model. What lessons can we learn for open source software?

Don Goodman-Wilson - Chinese food, motor scooters, and open source developmen...

Codemotion

Pieter Omvlee - The story behind Sketch - Codemotion Amsterdam 2019

Codemotion

Would you fly in a plane designed by a craftsman or would you prefer your aircraft to be designed by engineers? We are learning that science and empiricism works in software development, maybe now is the time to redefine what “Software Engineering” really means. Software isn't bridge-building, it is not car or aircraft development either, but then neither is Chemical Engineering. Engineering is different in different disciplines. Maybe it is time for us to begin thinking about retrieving the term "Software Engineering" maybe it is time to define what our "Engineering" discipline should be.

Dave Farley - Taking Back “Software Engineering” - Codemotion Amsterdam 2019

Codemotion

What is the job of a CTO and how does it change as a startup grows in size and scale? As a CTO, where should you spend your focus? As an engineer aspiring to be a CTO, what skills should you pursue? In this inspiring and personal talk, I describe my journey from early Red Hat engineer to CTO at Bloomon. I will share my view on what it means to be a CTO, and ultimately answer the question: Should the CTO be coding?

Joshua Hoffman - Should the CTO be Coding? - Codemotion Amsterdam 2019

Codemotion

More from Codemotion (20)