A background story on AI. On what is possible, being worked on, and what are some considerations.

1. Bright Night
@hansbos
hans.bos@microsoft.com

3. https://azure.microsoft.com/en-us/services/cognitive-services/emotion/

4. X
X
X O
X O
X
X
O
X
O
X O
X
X O X
X O
X
X O
X
X O
X
X O
X
X O
O X
X O O
X
X O
X O
X O
X
O
X O
X
O
X O
X
O
X O O
X X
X O O
X X
X O O
X
X
X O O
X
X
X O O
X
X

5. http://image-net.org/index
ImageNet is an ongoing research effort to
provide researchers around the world an
easily accessible image database.
The ImageNet project is inspired by a
growing sentiment in the image and vision
research field – the need for more data. Ever
since the birth of the digital era and the
availability of web-scale data exchanges,
researchers in these fields have been working
hard to design more and more sophisticated
algorithms to index, retrieve, organize and
annotate multimedia data.

6. Input Output

7. Input Output
w11
w21
w31
Weights
Activation
Outputs

8. Feedforward of information

9. 3
Error back propagation
w11
w21
w31

10. We train neural networks with depth of over 150 layers. We propose a "deep residual learning"
framework that eases the optimization and convergence of extremely deep networks. Our "deep
residual nets" enjoy accuracy gains when the networks are substantially deeper than those used
previously. Such accuracy gains are not witnessed for many common networks when going deeper.
028%
026%
016%
012%
007% 007%
004%
2010 2011 2012 AlexNet 2013 2014 VGG 2014 GoogleNet 2015 MS ResNet
Human parity (5,1%)
http://image-net.org/challenges/LSVRC/2015/results | https://blogs.microsoft.com/ai/microsoft-researchers-win-imagenet-computer-vision-challenge/
shallow 8 layers 19 layers 22 layers 152 layers

11. Available
https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/
FEATURE NAME: VALUE
Description { "tags": [ "outdoor", "person", "building", "holding", "woman", "sidewalk",
"street", "young", "bicycle", "lady", "table", "walking", "red", "city", "girl",
"standing", "carrying", "man", "little", "wearing", "riding", "park", "phone",
"board", "kite", "hydrant" ], "captions": [ { "text": "Kelly Macdonald is
walking down the street", "confidence": 0.964099 } ] }
Tags [ { "name": "outdoor", "confidence": 0.9928818 }, { "name": "person",
"confidence": 0.9861265 }, { "name": "ground", "confidence": 0.9531361 }, {
"name": "sidewalk", "confidence": 0.7103443 }, { "name": "way",
"confidence": 0.576331437 } ]
Image format "Png"
Image dimensions 922 x 1382
Clip art type 0
Line drawing type 0
Black and white false
Adult content false
Adult score 0.0148117449
Racy false
Racy score 0.0157827977
Categories [ { "name": "people_", "score": 0.921875 } ]
Faces [ { "age": 28, "gender": "Female", "faceRectangle": { "top": 219, "left": 700,
"width": 115, "height": 115 } } ]
Dominant color background "White"
Dominant color foreground "White"
Accent Color #BF0C22

12. Available
https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/

14. https://azure.microsoft.com/en-us/services/cognitive-services/emotion/

15. https://www.nextrembrandt.com/

16. Microsoft Research project
https://blogs.microsoft.com/ai/drawing-ai/
The pictures are created by the
computer, pixel by pixel, from
scratch. This birds may not exist in
the real world — they are just an
aspect of our computer’s
imagination of birds.
The drawing bot closes a research
circle around the intersection of
computer vision and natural
language processing

17. Available
https://azure.microsoft.com/en-us/services/cognitive-services/language-understanding-intelligent-service/
A machine learning-based service to build
natural language understanding into apps,
bots, and IoT devices. Quickly create
enterprise-ready, custom models that
continuously improve.
Intents
An intent represents actions the user wants to
perform.
Utterances
An utterance is text input from the user that your app
needs to understand.
Entities
An entity represents detailed information that is
relevant in the utterance.
Book me a flight to Amsterdam
{
“query”: “Book me a flight to Amsterdam”,
“topScoringIntent”:
{
“intent”: “BookFlight”,
“score”: 0.9887482
},
“intents”: [
{
“intent”: “BookFlight”,
“score”: 0.9887482
},
{
…

18. Available
Break down language barriers with your friends, family and
colleagues.
Our online translator can help you communicate more clearly.
Our voice translator currently works in 8 languages,
Arabic, Arabic (Levantine), Chinese (Mandarin), English,
French, German, Hindi, Italian, Japanese, Portuguese (Brazil),
Russian, Spanish.
and our text translator is available in more than 50 languages for
instant messaging.
Skype Translator uses machine learning. So the more you use it,
the better it gets.
https://www.skype.com/en/features/skype-translator/

21. Microsoft Research project
The situated interaction research effort aims to enable
computers to reason more deeply about their
surroundings, and engage in fluid interaction with humans
in physically situated settings. When people interact with
each other, they engage in a rich, highly coordinated,
mixed-initiative process, regulated through both verbal
and non-verbal channels. In contrast, while their
perceptual abilities are improving, computers are still
unaware of their physical surroundings and of the
“physics” of human interaction. Current human-computer
interaction models assume there is a single user, engaging
with full attention on a single computer system.
Computers do not yet understand engagement, attention,
proximity, interruptability, turn-taking, group dynamics,
social expectations, human memory and goals, and so on.
Our research aims to address these challenges and create
the basis for a new generation of situated systems that are
capable of fluid interactions and collaborations with
people.
conversation
intention
signal
channel
why: goals and intentions
sense and reason about
beliefs, intentions, goals,
and long-term plans
what: situation and activity
sense and reason about
relevant events and
activities of self and others
who: physical awareness
Identify, track, and
characterize relevant actors,
objects, states, and
relationships
https://www.microsoft.com/en-us/research/project/situated-interaction/

22. Microsoft Research project
https://www.microsoft.com/en-us/research/project/safe-autonomous-flight-everywhere-safe/
Microsoft open-sourced a research project called AirSim,
a high-fidelity system for testing the safety of artificial
intelligence systems. AirSim provides realistic
environments, vehicle dynamics and sensing for research
into how autonomous vehicles that use AI that can
operate safely in the open world.
AirSim has been developed as a plugin for Unreal Engine,
a popular tool for game development. This means that
the car simulation is decoupled from the environment it
runs in. You can create an environment for your specific
needs, such as a city or rural road, or choose from a
variety of environments available online, and then simply
drop in the AirSim plugin to test your self-driving
algorithms in that environment. AirSim extensibility also
allows researchers and developers to incorporate new
sensors, vehicles or even use different physics engines.

24. SAE International is a global association of more than 128,000 engineers and related technical experts in the aerospace, automotive and commercial-vehicle industries.
Available here: https://www.sae.org/standards/content/j3016_201401/
Level 0: No Automation
Level 1: Driver Assistance
Level 2: Partial Automation
Level 3: Conditional Automation
Level 4: High Automation
Level 5: Full Automation
The full time performance by an automated driving system of all aspects of the dynamic driving task
under all roadway and environmental conditions that can be managed by a human driver
The driving mode-specific performance of an automated driving system of all aspects of the dynamic
driving task, even if a human driver does not respond appropriately to a request to intervene.
The driving mode-specific performance by an automated driving system of all aspects of the dynamic
driving task with the expectation that the human driver will respond appropriately to a request to
intervene.
The driving mode-specific execution by one or more driver assistance systems of both steering and
acceleration/deceleration using information about the driving environment and with the expectation
that the human driver perform all remaining aspects of the dynamic driving task.
The driving mode-specific execution by a driver assistance system of either steering or
acceleration/deceleration using information about the driving environment and with the expectation
that the human driver perform all remaining aspects of the dynamic driving task.
The full time performance by an automated driving system of all aspects of the dynamic driving task
under all roadway and environmental conditions that can be managed by a human driver
Humandrivermonitorsthedriving
environment
Automateddrivingsystem
monitorsthedrivingenvironment

26. Build in transparency
Use intelligible machines to educate
users about how technology
recognizes and analyzes information.
Encourage algorithmic
accountability
Create new technologies with the
expected and the unexpected in mind.
Design for Privacy
Adopt protection that secures personal
and organizational information in ways
that earn trust.
Maintain dignity
Prioritize preserving cultural
commitments and empowering
diversity.
Guard against bias
Ensure proper, representative research
so that the wrong heuristics aren’t used
to discriminate
Augment human capabilities
Design technology that assists
humanity and respects human
autonomy.
More information: http://microsoft.com/ai
At Microsoft, we believe that ethics and design go hand in hand. AI technology should not only be transparent, secure, inclusive, and respectful, but also maintain the highest degree of
privacy protection. These principles guide the design of our own products and services and are what we aspire to be debated and adopted more broadly across industries and society.

28. AI and its role in society
https://news.microsoft.com/futurecomputed/

29. microsoft.com/ai

30. microsoft.com/ai
Studio is a powerfully simple
browser-based, visual drag-and-
drop authoring environment
where no coding is necessary.
Go from idea to deployment in a
matter of clicks.

31. Risk Analytics
Fraud Prevention
Customer Next Best Action
Predictive Maintenance
Operations Automation
Fleets management
Personalization
Dynamic pricing
Advanced retail planning

35. Bright Night
@hansbos
hans.bos@microsoft.com