8. But first!
Let’s generate some demo data!
• Tweet something using #SS2020
• Use a language you like (Latin script)
9.
10. Why do I like the
Power Platform
tools
– including AI
Builder?
Evolved to become a Consultant at a
Microsoft Partner
Landed by accident in the
IT-department of an end customer
Started out as business user
14. Step by Step Guidance in the UI
ADD YOUR
DATA
TRAIN THE
MODEL
CHECK THE
PERFORMANCE
PUBLISH & USE
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27. Prediction - GA
Determine the likelihood of specific business outcomes – Boolean (Yes/No – True/False)
Input: Data in 1 CDS entity
Output: prediction written back into prediction field in CDS
Use automatic schedule in CDS and in Power Automate
Specifics: Uses CDS Entity
Only 2 outcome possibilities (Yet!)
Data requirements: Minimum of 50 records in total and 10 records historical outcome for each class.
Best result with at least 1000 records.
30. Object detection - Preview
Build an AI model that recognizes and counts visual objects.
Output: Name and count of detected objects
Use in Power Apps Canvas & Power Automate
Specifics: JPG, png or BMP – max 6 MB
Data requirements: minimum of 15 images per object – preferably 50 or more.
31.
32. Text classification- Preview
Build an application that reviews, tags, and classifies large volumes of text to track sentiment,
improve customer experiences, and mine valuable insights.
Output: List of tags with certainty score in separate entity
Auto generated on schedule in CDS & used in Power Automate
Specifics: Uses CDS Entity
Languages: English, French, German, Dutch, Italian, Spanish, Portuguese
Data requirements: Text and tags in text fields in same entity – All tags in one field using a
delimeter
Max 5000 characters per text item - Minimum of 50 text items per tag.
33.
34. Form processing - Preview
Read, extract, and process data from scans, emails, PDFs, and images.
Output: Extracted key-value pairs and table data.
Use in Power Apps Canvas & Power Automate
Specifics: JPG, png or PDF – max 4 MB – Latin alphabet
High quality images – between 530 x 100 and 4200 x 4200 pixels
Data requirements: 5 – 10 sample documents with the same layout.
Must contain keys above or to the left of the value.
38. Recent updates
Manually enter objects
for Object detection
New fields recognized
with Business Card
Reader
Updates to AI Builder in
the Power Apps canvas
studio
AI Builder flow
templates
https://powerapps.microsoft.com/en-us/blog/category/ai-builder/
39. On the roadmap
New AI Models
• Prebuilt Text classification
model
• Anomaly detection
• Image moderation
• Receipt scanning
Usability
• AI Builder connector
• Guided learning Form
processor
• Entity extraction model
• New Power Apps controls
• New prebuilt Flow templates
• ‘Try’ feature for the keyword
extraction model.
AND MORE…
43. Update custom model
Create new version
• From published version
• Or
• From last trained version
Change settings or training
data
Publish
or
• Only for:
• Prediction model
• Text classification model
Retrain
model
46. Roles & security CDS security roles
ENVIRONMENT MAKERS
CREATE AI MODELS
CDS USERS
USE MODELS
SYTEM ADMINS & CUSTOMIZERS
ACCESS ALL MODELS
47. AI Builder License
AI Builder Capacity Add-on
$ 500 / subscription
1 million service credits
Tenant level
Credit consumption
Training models
Using models
Credit calculation ???? – Calculator not available yet
Credit allocation
Allocate capacity to environments in
Power Platform admin center
Business card reader
Pull contact information from business cards, store this information in your database, and import it into your CRM system.
Sentiment analysis (preview)
Detect positive, negative, neutral and mixed sentiment in social media, customer reviews or any text data you want to analyze.
Key phrase extraction (preview)
Extract the main points and key phrases in text documents.
Text recognition (preview)
Extract embedded printed and handwritten text from documents and images into machine-readable character streams
Language detection (preview)
Use this model to identify the predominant language of a text document.
PredictionUse your database and this AI model to determine the likelihood of specific business outcomes.
Determine the likelihood of specific business outcomes – Boolean (Yes/No – True/False)
Input: Data in 1 CDS entity Output: prediction written back into prediction field in CDS
Use automatic schedule in CDS and in Power Automate
Specifics: Uses CDS EntityOnly 2 outcome possibilities (Yet!)
Data requirements: Minimum of 50 records in total and 10 records historical outcome for each class.
Best result with at least 1000 records.
Data quality – no empty fields / faulty data
Data quantity – enough data with options that reflects the expected outcome
Unwanted bias – unselect irrelevant or misleading fields
Accuracy score between 50 – 100%
BUT! You are the one to determine if the accuracy makes sense!
Object Detection (preview)
Build an AI model that recognizes and counts visual objects
Build an AI model that recognizes and counts visual objects.
Output: Name and count of detected objects
Use in Power Apps Canvas & Power Automate
Specifics: JPG, png or BMP – max 6 MB
Data requirements: minimum of 15 images per object – preferably 50 or more.
Text Classification (preview)
Build an application that reviews, tags, and classifies large volumes of text to track sentiment, improve customer experiences, and mine valuable insights.
Build an application that reviews, tags, and classifies large volumes of text to track sentiment, improve customer experiences, and mine valuable insights.
Output: List of tags with certainty score in separate entity
Auto generated on schedule in CDS & used in Power Automate
Specifics: Uses CDS EntityLanguages: English, French, German, Italian, Spanish, Portuguese
Data requirements: Text and tags in text fields in same entity – All tags in one field using a delimeterMax 5000 characters per text item - Minimum of 50 text items per tag.
Data quality
No high rate of missing values
Check for typos
Check for inconsistent capitalizations
Check for inconsistent or incorrect labels
Unwanted bias
Remove duplicate or irrelevant fields
Remove fields with high correlation to the outcome
At least 100 records per tag
Balanced use of tags in your data
Training data similar to actual text
Form Processing (preview)Read, extract, and process data from scans, emails, PDFs, and images.
Read, extract, and process data from scans, emails, PDFs, and images.
Output: Extracted key-value pairs and table data.
Use in Power Apps Canvas & Power Automate
Specifics: JPG, png or PDF – max 4 MB – Latin alphabetHigh quality images – between 530 x 100 and 4200 x 4200 pixels
Data requirements: 5 – 10 sample documents with the same layout.
Must contain keys above or to the left of the value.
Not supported yet
Complex nested tables
Check boxes or radio buttons
PDF documents > 50 pages
Fillable PDF’s
Manually enter objects for Object Detection
You can now enter object names manually to train your object detection models. This mode of entry is shown by default, but you can still use your object names in Common Database Service.
New fields recognized with Business Card Reader
New fields are now available in Business Card Reader. You can extract the Post Office Box and the State fields from addresses detected on business cards in Power Apps and Power Automate.
Updates to AI Builder in the Power Apps canvas studio
Styling and control reset
You can now style all AI Builder components in your canvas apps using the color and border pickers, as well as configure the text properties easily in the property panel. The components also implement the Reset action, which allows you to clean up images and output properties on demand.
Interact with detected text in Text Recognizer
The Text Recognizer component in Power Apps now surfaces a new output property called Selected. You can interact with the component by clicking on detected text to select them. We have also renamed the output property containing the list of text to Results. You can also find more information about the text detected in the image.
Confidence scores and bounding boxes in Form Processor
The confidence scores of selected form fields are now available in Form Processor. In the Results output property, each field and table element has a corresponding key, value, confidence score, and bounding box. You can continue to leverage the fields and tables without detailed information by using the Fields and Tables output properties.
Confidence scores and bounding boxes in Object Detector
You can now see detailed information about all objects detected in your images. For each object detected in an image, you can see the object’s label, confidence score, and bounding box information. This detailed information can be found in the Results property. You can also find the count of objects for each label with the GroupedResults property.
Introducing AI Builder flow templates
To make it easier to use your AI models in Power Automate, we have added new AI Builder flow templates that you can start using today!
AI Builder flow templates
Click here to view them all.
TemplateScenarioCreate contact in CDS from a business card using AI Builder on button clickBusiness Card ReaderUpdate contacts in CDS from OneDrive(Business) records using AI BuilderBusiness Card ReaderClick a button to read and save information from documents using AI BuilderForm ProcessingClick a button to and count objects in images using AI BuilderObject DetectionSend a notification Send a notification with the sentiment of a manager’s email using AI BuilderSentiment AnalysisExtract text from images using AI Builder Text Recognition on button clickText RecognitionMake Office 365 email image attachments searchable using AI BuilderText Recognition
What does this mean?
You can use these templates to create flows using either your own AI model or a prebuilt AI model in just one click. No more reading through lengthy instructions to create your flow step-by-step. Start with a template and customize it to suit your business needs. You can even discover new ways to use AI Builder in Power Automate. For example, updating contacts in Common Data Service with Business Card Reader allows you to use the info in your Dynamics applications and elsewhere in the Power Platform seamlessly.
Related entities and filtering support in Prediction
Prediction now includes support for related entities and filtering. See this blog post for more information.