Traditional machine learning requires volumes of labelled data that can be time consuming and expensive to produce,” “Machine teaching leverages the human capability to decompose and explain concepts to train machine learning models direction (teaching the correct answer is not by showing the data for it, but by using a person to show the answer). Project Bonsai is a low code platform for intelligent solutions but with a different perspective on data it allows a completely new approach to tasks, especially when the physical world is involved. Under the hood it combines machine teaching, calibration and optimization to create intelligent control systems using simulations. The teaching curriculum is performed using a new language concept - “Inkling” and training a model is easy and interactive.

1. DATA SATURDAY #10
Sofia, Oct 09th
Another PoV on Data
Autonomous Systems and
Project Bonsai
Ivelin Andreev

2. Speaker Bio
• Software Architect @
o 19+ years professional experience
• Microsoft Azure MVP
• External Expert Eurostars-Eureka
• External Expert InnoFund Denmark, RIF Cyprus
• Business Interests
o Web Development, SOA, Integration
o IoT, Machine Learning, Computer Intelligence
o Security & Performance Optimization
• Contact
ivelin.andreev@icb.bg
www.linkedin.com/in/ivelin
www.slideshare.net/ivoandreev

3. Thanks to our Sponsors

4. Agenda
Agenda
• Evolutionary and Genetic Algorithms
• Autonomous vs Automated System
• Bonsai Project
• Bonsai Platform Components
• Basic Concepts and Technical Overview
• Demo

5. Optimization and Control

6. Constrained Optimization
Def: Maximize/Minimize a cost function within constraints (i.e. time, money, fuel)
Evolutionary Algorithms
• Heuristics-based problem solving through mimicking the behavior of living
creatures to avoid exhaustive solution application
Pros
• Solve global optimization problems
• Scalable to higher dimensional problems
• Flexible and adaptable to many problem types
• Multiple “best” solutions
Cons
• Vulnerable to bad initial configuration
• Premature convergence to a local extremum may not reach even close to global extremum

7. Genetic Algorithms
Def: Тype of EA that uses crossover and mutations to search the solution space.
Key Principle: Mating healthier parents shall lead to healthier offspring.
• Population – set of current solutions
• Individual – a solution
• Fitness – criterion to compare generations/solutions (f(x) = 5x^2)
• Variation – modification of population to generate new solutions
• Chromosome – a set of features describing the individual
• Gene – Each part of the chromosome, encoded by numbers

8. Simple/Sample Case
Background
• Thousands of heating devices
• Current state, Usage history
• Power price/hour 1 day ahead
Objective
• Cost optimization of heating
Limitations
• Heater has maximum energy @80°C
• Charge has to be >= daily consumption
• Computation scalability
Solution
• Time series forecasting
• Genetic algorithms

9. Autonomous Systems?
Def: Intelligent systems that operate in highly dynamic PHYSICAL environments
by sense, plan and act based on changing environment variables
• To apply Real World…
o Millions of simulations
o Assessment in real world required
o Simulation needs to resemble physical
world as close as possible.
o A bad reward function = The effect
Reacting starts from gathering
information about the
environment.
Interpreted sensor information
and translated to actionable
information based on reward.
Once the action plan is in
progress, evaluate proximity of
the objective.

10. • Autonomy is moving ML from bits to atoms
• Levels of Autonomy
L0 Humans
In complete control
L1 Assistance with
subtasks
L2 Occasional,
human specifies intent
L3 Limited,
human fallback
L4 Full control,
human supervises
L5 Full autonomy,
human is absent
Autonomous, not Automated
• Automated Systems
o Execute a (complex) “script”
o Apply long term process changes (i.e. react to wear)
o Apply short-term process changes (clean, diagnose)
o No handling of uncoded decisions
• Autonomous Systems
o Aim at objectives w/o human intervention
o Monitor and sense the environment
o Suits dynamic processes and changing conditions
• Disadvantages
o Human trust and explainability (Consumer acceptance)
o What if the computer breaks (Technology and
infrastructure)
o Security and Responsibility (Policy and legislation)

11. • Industrial robotics and manufacturing
• Motion control
• Smart buildings
• Process control and automation
• Bonsai emerges as a general-purpose, deep reinforcement learning platform
• Microsoft acquire Bonsai in move to build ‘brains’ for autonomous systems
• Released for public preview in May 2020
• The simplest and richest toolchain for AS
o Machine teaching abstracting
o Automatic model generation
o Host APIs for simulator integration and inference
Some History
https://blogs.microsoft.com/blog/2018/06/20/microsoft-to-acquire-bonsai-in-move-to-build-brains-for-autonomous-systems/

12. Project Bonsai in the Spotlight
• Low code end-end platform that speeds AI-powered automation development
• Integrate realistic simulation
• Train adaptable brains
• Export and consume brains
• Cost?
o Completely free in preview
• Azure Resources
o Container Registry (brains and simulators)
o Container Instance (running simulations)
o Storage (uploaded ZIPped simulators)
o Monitor (logs of training and assessment)

13. Accelerators and Setting up
• Solution accelerators designed for specific industrial optimization problems
• Pricing
o Project Bonsai (Free for now)
o Simulator software (AnyLogic, MATLAB)
o Azure Services (VM D2_v4, public IP, 128GiB SSD, ACR) ~90EUR/month
• Supported Use Cases
o Chemical reactor optimization (Simulink - Continuous Stirred Tank Reactor)
o Building energy management (Simulink – House Thermal Model)
o Factory logistics (AnyLogic – Factory Floor)
• Templates did not work ☺, but after report fix is in progress

14. Autonomous Systems
Use Cases

15. Horizontal Drilling

16. Windfarm Energy Output Optimization

17. Warehouse Optimization

18. Bonsai
Components

19. Simulator as Data Source
• Simulator – replicate real world for realistic training environment
• Supported Simulations
o AnyLogic, MATLAB, VP Link or API (REST, Python)
• Minimum requirements
o Initial state (often random)
o Iterates over time
o Responds to external actions
• Good simulation requirements
o Good fidelity to match real world
o Visualization or data output for training assessment
o Well defined environment state (parameters)
o Discrete actions for the brain to affect state
o Clear criteria for failure and success state

20. Simulators
• Option 1: Off the shelf software
o Pros: Thousands of ready code and samples; Highly graphical
o Cons: Steep learning curve; Resource consuming; Coding still required
o (Small Business, Academia, Free Trial)
o (Enterprise, Academic, Free Matlab Online Trial)
• Option 2: Custom simulators
o API Client SDK in Python, C#, Java, TypeScript
o Build as Docker image
o Push to AZ Container Registry (managed = scalability)
o Consume from Bonsai portal using “Add Simulator”
o Pros: Complete control; no upfront costs
o Cons: A Developer approach; Hard to code some existing simulations
o Sample: https://github.com/microsoft/cartpole-py
AI Agent
Action
Simulator
State
Reward

21. Bonsai Components – Training Engine
• Architect
o Proposes the configuration of learning algorithms and topologies that have the best chance at learning
o 3 deep learning reinforcement learning algorithms with heuristics params
• Instructor
o Configures the learner in real time to follow the training plan of the architect
• Learner
o Training: train and evaluate response
o Inference: Initiate the trained system and respond to request
• Predictor
o A trained brain stored in prediction mode
o HTTP API endpoint

22. Bonsai Components - Brain
• AI models that can optimize the systems they have been trained for
• Supports multiple versions
• Versions reflect changes in training curriculum
• Any version is exportable outside the platform
• Goal Objectives
o Encapsulate intention-to-reward function and simplify the work of operator
o Avoid – avoid a region
o Reach - go to target ASAP
o Drive – go to target ASAP and stay near
o Maximize – push target as high as possible
o Minimize – push target as low as possible

23. Teaching Process

24. Iteration: Concept Graph View
Agent Action
The brain output on every iteration
Observable State
Passed to brain on every iteration
Simulator State
Output by simulator on every iteration
Concept
What the brain is trying to learn
Concept Goals
What the brain has to achieve
State Transformation
Implicit conversion of sim state
Simulator
Simulator used for concept training

25. Inkling Language
• Orig: Fictious language spoken by Nintendo Splatoon human-squid inklings
• Now: Proprietary language defines what and how you want to teach the AI
• Statically typed language
o Compiler detects and reports errors
• Starts with Version
• Data Types
o Primitives: Number (integer, float), String with ranges and enumerations
o Collections: Arrays, Structures, User defined types
inkling "2.0“
# Comments like that
type Coordinate {X: number, Y: number}
number<0 .. 6 step 2> # represents {0,2,4,6}

26. Concept – What to learn
• Concept
o Component of a brain that must be learned, programmed or imported
o Sample: Ensure hot water is available in a boiler while heating in low load hours
• Imported Concept
o Compatible models trained on other platforms (ONNX, TensorFlow)
• Programmed Concept
o Defined using the programmed keyword followed by a function definition or reference.
• Learned Concept
o Specify a curriculum that defines how the AI should be taught
concept ConceptName(Input1, Input2): OutputType {
# Concept definition
}
function doJob() {…}
concept ConceptName (Input): OutputType {
programmed doJob }

27. Curriculum – How to learn
• Defines how the training engine should train the AI within concept
o Rule: 1 curriculum per concept, each concept with curriculum
• Defines lessons for staged teaching plan and control training episodes run
• Training stops when:
o Stopped manually
o When training is no longer improving
o Limits reached (i.e. training time)
• Source
o Data source for teaching the concept (Simulator)
• Parameters
Parameter Default Description
EpisodeIterationLimit 1000 Iterations per training episode
TotalIterationLimit 50,000,000 Total iterations for the concept
NoProgressIterationLimit 250’000 Iterations allowed w/o improvement
LessonRewardThreshold None Minimum reward value for success
LessonSuccessThreshold 0.9 Min success rate to complete lesson
LessonAssessmentWindow 30 Number of episodes per assessment

28. Goal Keyword
• High level specification of what we want AI to learn
o The Bonsai engine determines appropriate reward functions
• Objectives
o Avoid
o Reach
o Drive
o Maximize
o Minimize
• Goal Ranges
using Goal # namespace
Curriculum {
…
goal (s:SimState){
avoid Fall: Math.Abs(state.angle) in Goal.RangeAbove(MaxAngle)
drive SmallAngle: Math.Abs(state.angle) in Goal.Range(0, MaxAngle/6)
}}
Parameter Description
Goal.Range(X, Y) Values between X and Y, inclusive
Goal.RangeAbove(X) Values greater than or equal to X
Goal.RangeBelow(X) Values less than or equal to X
Goal.Sphere(X, R) Values within line with radius R
Goal.Sphere([X, Y], R) Values within 2D circle with radius R
Goal.Sphere([X, Y, Z], R) Values within 3D sphere with radius R

29. Simulator Keyword
• Virtual environment to simulate the behaviour of a real-world physical env.
• Simulator has observable state information - the state at a given moment
o Typically sensor measurements
• Note: Inkling does not contain simulators code but defines interaction with
the simulator to train a concept.
• Simulator statement
o Describes the interface to the simulator program
o ActionType – action input type
o ConfigType – configuration input type
o StateType – state output type
• Packages (optional)
o references a simulator package registered for use in the cloud
simulator MySimulator(action: ActionType, config: ConfigType): StateType {}

30. Takeaways
o MS Innovation Tech Hub (News, concepts, recent developments)
• https://innovation.microsoft.com/en-us/autonomous-systems
o Bonsai Portal
• https://preview.bons.ai/
o Community Forum
• https://techcommunity.microsoft.com/t5/project-bonsai/ct-p/ProjectBonsai
o Accelerators
• https://docs.microsoft.com/en-us/autonomous-systems/bonsai-accelerators/what-are-bonsai-accelerators
o Fixed Accelerator Sample
• https://techcommunity.microsoft.com/t5/project-bonsai/accelerators-failed-to-deploy-image-is-missing/m-p/2460816
o End-end Walkthrough – Project Bonsai & AnyLogic
• https://www.youtube.com/watch?v=-iTSZaG3lg8
• https://microsoft.github.io/moab/tutorials/2-robustness/index.html
o Simulator Sample & Bonsai SDK
• https://github.com/microsoft/cartpole-py
• https://github.com/microsoft/microsoft-bonsai-api

31. Upcoming Events
jsTalks (Bulgaria), 2021
November 19-20
http://jstalks.net/
Azure MVP Unplugged, 5th Edition
October 21
Global AI Back Together Bulgaria 2021
October 19

32. Thanks to our Sponsors