6. Rise of the Cognitive Assistant
Assistants are being
embedded
everywhere; are
increasing ability to
solve problems.
As computing
advances, so will
Virtual Assistant
ability to help in
task completion.
Virtual Assistants
ability to be “smart”,
and to understand
intent, tone, and
context.
Market demand for
meaningful, stateful
and goal oriented
conversations.
11. Language Understanding Intelligence Service (LUIS)
A Microsoft Cognitive Service that provides developers with an easy way to create language
models to allow applications to understand user commands.
Create your own
LU model
Train by providing examples
Deploy to an HTTP endpoint
and activate on any device
Maintain model with ease
12. LUIS
Language understanding in human-computer interaction is:
Technically
challenging
It’s exceedingly difficult to enable a
computer to understand what a
person wants and to find the pieces
of information that are relevant to
their intent.
Costly
to implement
Building and maintaining machine
learning systems requires a large
investment of time, money and
engineering resources
Often
domain specific
In the past, building your own
machine learned models often
required assistance of a team of
data scientists that would customize
the models to the specific domain.
15. Quick Start – Pre Built Entities
Improved number, unit, money and
date/times
“From May 1st to May 5th”
“I want to fly from SEA to AMS next
Tuesday and leave after 5 days”
Age Dimension Email
Encyclopedia Geography Money
Number Ordinal Percentage
Phone
number (US)
Temperature URL
19. Many different types of Bots
Procedural Bots
• Prescribed menu
• Little agility for end
users
Contextual Bots
• Less rigid
• Users have intents
and issue utterances
• Bot interprets
meaning and acts
Event driven Bots
• Users subscribe to
events
• Are notified when
event fires
Robotic Process
Automation (RPA)
• Automation
achieved through
GUI integration
• Macro driven
20. Dialog
[Serializable]
public class EchoDialog : IDialog<object>
{
public async Task StartAsync(IDialogContext context)
{
context.Wait(MessageReceivedAsync);
}
public async Task MessageReceivedAsync(IDialogContext context,
IAwaitable<IMessageActivity> argument)
{
var message = await argument;
await context.PostAsync("You said: " + message.Text);
context.Wait(MessageReceivedAsync);
}
}
21. FormFlow
public string NameOfVessel;
[Numeric(1000000, 9999999)]
[Prompt("Please enter your {&}")]
public int OfficialVesselNumber;
[Pattern(@"^(?:+d{1,3}|0d{1,3}|00d{1,2})?(?:s?(d+))?(?:[-/s.]|d)+$")]
public string PhoneNumber;
[Numeric(2, 20)]
[Prompt("Please enter the {&}")]
public int NumberOfPeopleOnBoard;
public Cargo? PreviousCargo;
public static IForm<ShipRegistrationDialog> BuildForm()
{
return new FormBuilder<ShipRegistrationDialog>().Message("Welcome to Rotterdam Tank
Terminals").OnCompletion(StartStoreRegistration).Build();
}
28. Bot Creator Personas
Azure Logic Apps
BOT Framework
Pro Integrator
Sandra
• Works in IT as a developer
• Codes in Visual Studio
• Builds custom solutions
• Azure Portal is her Happy Place
• Loves building APIs
• Azure Functions FTW!
Stuart
• Works in HR as an HRIS Analyst
• Has some technical training
• Excel and SharePoint are his JAM!
• Is under constant pressure to do more
• Doesn’t have Azure Portal access
• Administrates SaaS solutions like
Workday, SuccessFactors, Taleo
Citizen Integrator
Microsoft Flow
Bizzy (H3 Solutions)
Ad-hoc Integrator
Sam
• In IT, Service Desk Supervisor
• Can script in PowerShell
• Looking to reduce costs in
providing IT Service
Management
• Administrates ServiceNow
• Persona may include IT Pro/BA
Microsoft Flow
Azure Logic Apps
31. Building a Bot - Process
• Define LUIS Intents and Entities
https://www.luis.ai/
32. Building a Bot - Process
• Define LUIS Intents and Entities
• Download Bot Framework Templates and
Emulator
33. Building a Bot - Process
• Define LUIS Intents and Entities
• Download Bot Framework Templates and
Emulator
• Build Controllers, Models and Prompts
34. Building a Bot - Process
• Define LUIS Intents and Entities
• Download Bot Framework Templates and
Emulator
• Build Controllers, Models and Prompts
• Build your Logic Apps
35. Building a Bot - Process
• Define LUIS Intents and Entities
• Download Bot Framework Templates
and Emulator
• Build Controllers, Models and Prompts
• Build your Logic Apps
• Protect your Logic Apps with API
Management
• Apply additional Policies
36. Building a Bot - Process
• Define LUIS Intents and Entities
• Download Bot Framework Templates
and Emulator
• Build Controllers, Models and Prompts
• Build your Logic Apps
• Protect your Logic Apps with API
Management
• Test locally with Emulator
37. Building a Bot - Process
• Define LUIS Intents and Entities
• Download Bot Framework Templates and
Emulator
• Build Controllers, Models and Prompts
• Build your Logic Apps
• Protect your API with API Management
• Test locally with Emulator
• Publish to Azure App Service
38. Building a Bot - Process
• Define LUIS Intents and Entities
• Download Bot Framework Templates and
Emulator
• Build Controllers, Models and Prompts
• Build your Logic Apps
• Protect your Logic Apps with API
Management
• Test locally with Emulator
• Publish to Azure App Service
• Register and Connect Bot
43. Grow up to Logic Apps
• “No Cliffs” across Flow and Logic Apps
• Flow is good for simple integrations and
empowering users to do integrations
without going through development
teams in IT
• However, sometimes IT / devs need to
take over when the Flow gets too
advanced
• Flows can be converted to a Logic App
44. Microsoft Flow
https://flow.microsoft.com – Do you have Office 365?...You then have Flow.
Azure Logic Apps
https://azure.microsoft.com/en-us/services/logic-apps/
Microsoft Research Cognitive Services YouTube Channel
http://tinyurl.com/hob5zjp
Bizzy – The Enterprise Bot
https://getbizzy.io
Resources
Interested in all things integration – which of course includes MS Flow
Most of these slides are taken from Kent’s presentation at Integrate 2017 USA
Some slides are from Eldert
BOT builder:
Full SDK for .NET, Node.js
Emulator
Sample Bots
Bot Framework portal:
register,connect and manage bot in an easy and convenient way
Includes diagnostic tools and web chat control for embedding on a web page
A user may be typing simple requests based on natural language. For example, a user may type "I want a pepperoni pizza" or "Are there any vegetarian restaurants within 3 miles from my house open now?". Natural language understanding APIs such as LUIS.ai are a great fit for scenarios like this. Using the APIs, your bot can extract the key components of the user's text to identify the user's intent.
Provide important navigation clues
Where is the menu?
Where to go for help?
What is the privacy policy?
Users will not always travel in a linear path
The “stubborn bot”
The “clueless bot”
The “mysterious bot”
When implementing natural language understanding capabilities in your bot, set realistic expectations for the level of detail that users are likely to provide in their input.
44
In this fourth lab, we will be receiving the orders from the business customer’s topic in a new Logic App, and check the total amount of the invoice. In case the customer placed a large order (over $50000), we will create a task for one of our sales employees to contact the customer to verify the order. In case the order is correct, the invoice will be emailed to the customer. The Logic App will then call a function, in which we will check a storage table to determine how much discount the customer will be given (based on the total order amount), and finally will place a file on blob storage, which will be used by an employee to refund the customer.