Collect data from virtually anywhere. Process structured and unstructured data from sources, like legacy systems and local system files, then store it in Snowflake. In this session we will look at how to process PDF documents and load information into Snowflake using UiPath Document Understanding and how to get data from Snowflake to use in in UiPath automations
This session is especially for data analysts and other Snowflake users interested in learning more about how UiPath can be applied to common data challenges. We’ll discuss several use cases including data gathering, preparation, and using UiPath to extend integration between Snowflake and 3rd party applications.
📕 During the meetup we will cover:
Typical use cases for RPA and Snowflake
How the native integration for UiPath and Snowflake works
How to build an automation using the UiPath / Snowflake integration, including a live example and demo
👨💻 Speakers:
Mo Roy, Senior Partner Engineer, Technology Alliances @UiPath
Driving Behavioral Change for Information Management through Data-Driven Gree...
Innovate with the data you have with UiPath and Snowflake.pdf
1. Innovate with the data you have with
UiPath and Snowflake
Tech Integrations Series
2. 2
1. Meetup Introduction with our Community Manager
2. High Level Overview of UiPath + Snowflake partnership
3. Overview of UiPath + Snowflake technical integration and use cases
4. Live Walkthrough:
1. Configuring pre-requisites in Snowflake
2. Setting up Integration Service Connection (Automation Cloud)
3. Setting up ODBC Driver and Activity (On-Prem)
5. Live demo!
6. Q&A
Agenda
3. Lynette York
Regional Community Manager
APJ
@UiPath
Meet today’s team:
Diana Gray
Community Marketing Manager
AMER
@UiPath
Cristina Vidu
Global Manager
Marketing Community
@UiPath
4. Meet today’s speakers:
Mo Roy
Senior Partner Engineer
Product Technology Alliances
@UiPath
Lynette York
Director,
Product Technology Alliances
@UiPath
5. About today’s meeting:
• The meetup will last 1 hr 30 mins
• You are encouraged to network and share your LinkedIn in the
chat box
• We apologize for keeping you on mute till Q&A time
• Please use the chat to place questions too till Q&A
• There will be some fun too: Menti quiz coming up & UiPath prize
6. 6
UiPath + Snowflake
High-level partnership overview
• UiPath is a Powered By Snowflake partner
- UiPath Insights pulls data from robot logs stored in
Snowflake which allows for advanced data model
creation and live querying.
• UiPath is a Snowflake Technology partner
- UiPath Connector For Snowflake allows users to
quickly create UiPath Robots that can act on
information in Snowflake.
“The partnership between UiPath represents customers’
needs for both data and business processes that should
be fast, frictionless, and automated. As demand for
automation on data increases, partnerships with
organizations, such as UiPath, help Snowflake serve
our joint customers globally and support our mission of
mobilizing the world’s data.”
Tarik Dwiek
Head of Technology Alliances at Snowflake.
7. 7
Geography:
Department:
Industry:
Products:
Business Intelligence
Global
Customer Highlight – Avison Young
Comm Real Estate
Enterprise Automation
Challenge
Solution
Product Preview for Enterprise Automation Platform
Avison Young’s Business Intelligence team wanted a way to create an
advanced, 360-view of each client. To do so meant a team of data scientists
working tirelessly to routinely sort through data sets for “cleanup” before they
could import them into the analytics dashboard. More than half of the data
scientist’s time was spent on this “janitorial” duty.
With the automation and low-code available on UiPath Enterprise
Automation Platform, Avison Young reduced the amount of manual “fixing” of
data sets to an automated process. More data could import more often,
leaving the team to explore new data sources, analyze clients deeper, and
find new ways to correlate to new platforms and product client-specific
output.
Processes Automated
Reducing manual data tasks and menial work frees their talent to
do what they do best. The scalability of the UiPath capabilities
means they can continue to level up their tech stack.
“I wanted a way to make sure our data scientists are spending
their time doing data exploration, thinking about algorithms,
about correlations into other products we use.”
Dave Sawdey
Global Leader of Portfolio Analytics and Business Intelligence
Avison Young
8. WHITEPAPER
The modern data pipeline:
Fast, frictionless,
and automated
In collaboration with: AWS, Tableau and Snowflake
Release Date: 1 November 2022
10. 10
UiPath + Snowflake
Automate the data-driven enterprise with UiPath and Snowflake.
Utilize Snowflake data in RPA
workflows
Collect data from hard-to-
access sources
Automate data quality and
enrichment activities
Simplify data ingestion and
maintenance
Use extracted data from Snowflake
in multi-system, complex, end-to-
end processes.
Enrich your Snowflake data and
ensure data quality with AI driven
data validation and human-in-the-
loop capabilities to manage
exceptions.
Collect data from structured and
unstructured data sources, like
legacy systems and local system
files, then store it in Snowflake.
Activities function as a drag-and-
drop experience that do not require
writing complex code, meaning any
user can now implement automation
as part of data maintenance,
continuous testing and
administration activities.
Automating data operations just got easier. With UiPath UI and API automation capabilities, robots can collect data from virtually anywhere,
validate and enrich data then store it in Snowflake. Use Snowflake data in any of your business processes that incorporate ML models and
other business systems without having to write complex code.
11. 11
Automate the data-driven enterprise with UiPath and Snowflake.
UiPath + Snowflake
• UiPath Database Activities allows user to perform
create, read, update and delete operations on Snowflake.
• UiPath Insights pulls data from robot logs stored in
Snowflake which allows for advanced data model
creation and live querying.
• UiPath Connector For Snowflake allows users to
quickly create UiPath Robots that can act on information
in Snowflake.
Key Integrations
Record
Images / PDF / Email E.g. XML / JSON /
Get Records
API
Endpoint
Create
Record
API
Endpoint
Data sources
Robots
Orchestrator
Integration
Service
Test Manager
Track and
measure, automation
performance
Insights
Action Center /
Apps / Assistant
AI Center
Document
Understanding
ML Models
Manage data
testing automations
DU Process
Templates
Trigger data
automations,
validation and
human-in-the-loop
Snowflake
Connector
Interact with
Snowflake data
API Automation
E.g. SAP / Salesforce
UI Browser Automation
Process
unstructured Information
Process semi-structured
Information
Custom Browser App
Database and standard activities
<XML>
12. 12
Integrations
UiPath users can use the Snowflake Connector on Integration Service, and/or classic Database
Activity pack based on deployment
Automation Cloud Users On-Prem Users
• Use the Snowflake Integration
Service connector to query data
and create records
• Note – event-based triggers are
not yet available for the Snowflake
Connector
• Use our classic Database
activities to read, write, and
update Snowflake data using their
official ODBC driver
13. 13
Simplify data
automation with
UiPath RPA.
Connect to a
database using a
standard connection
string
Connect
Close a connection to
a database
Disconnect
Execute a non query
statement on a
database
Execute
Non-Query
Execute a query on a
database that returns
the query result in a
DataTable
Execute Query
Insert a compatible
DataTable variable in
an existing Table
Insert
Connect to a
database and feature
a Sequence which
can perform multiple
transactions with the
database.
Start
Transaction
15. 15
Data Sources
Automate the end-to-end data pipeline
Data Sources Data Ingestion
Data Storage + Data Modeling
Data
Aggregation
and
Enrichment
Data Testing,
Cleansing, and
Validation
Data
Exports
Reporting and Analytics
Trigger downstream processes
Enrich data between analyses
Apply a combination of API-driven, UI-driven, and database-specific integrations combined with intelligent
document processing, orchestration, and human in the loop capabilities - at scale
Images / PDF
API of business system
e.g. SAP / Salesforce
Application UI
XML / JSON
<XML>
RDBMS
Flat file / XLS / CSV
ETL - UI and API-based
data
Bulk Upload
Manual Upload
Data Migration
Batch Processing
Data Masking
Data Warehouse
Data Vault ODS 3NF
Star
Schema
Data consumption
Data consumption
Machine Learning
Operationalize ML models -
UiPath AI Center or 3rd party
Data consumption
Business Systems
Trigger business processes
Data Consumption
Orchestrate review and labelling
16. 16
Integration pattern – create Snowflake records
from documents, legacy data sources
Robot
processes all
documents in
folder
Extract data with
Document
Understanding
Pull additional data
from a web app,
other legacy data
source
Data added to
Snowflake
Data structured and
standardized
17. 17
Integration pattern – synchronize legacy data sources
with Snowflake
Robot processes
CSV export of
Supplier Names
Data entered in
legacy web app
Queue items
created in
Orchestrator with
identifying
parameters
Unattended
robots process
queue items
Data added to
Snowflake
Robots can fully automate synchronization so that users can focus on the modern data source (Snowflake)
Synchronization can occur during modernization/migration; once complete sun-setting/decommissioning can be
automated with zero disruption to users
18. 18
Integration pattern – Use Snowflake data to in
downstream actions in systems without APIs
Robot processes
CSV export of
Supplier Names
Data entered in
legacy web app
Queue items
created in
Orchestrator with
identifying
parameters
Unattended
robots process
queue items
Different ways to trigger
additional processes:
• Polling for changes
(unattended or
background trigger
attended)
• Scheduled executions
(ex. nightly for new
items)
Data added to
Snowflake
New Snowflake data
incorporated into
SAP Ariba Task
generation process
19. 19
Live walkthrough
• Configuring pre-requisites in Snowflake
• Set up warehouse, database, schema
• Setting up Integration Service Connection (Automation Cloud)
• Create connection, authenticate, map to db and schema, add Studio Package
• Setting up ODBC Driver and Activity (On-Prem)
• Download Driver, set up locally, add Studio Package
22. 22
Business Benefits
• Reduce manual effort to perform
data and database checks
• Eliminate human error
• Reduce risk by scaling validation
to cover all critical data elements
Implementation Time
• One month to develop and test
the unattended UiPath robot
• Native integrations supported
for hundreds of systems
Technologies
• Snowflake
• Tableau
• Amazon S3
• Excel, CSV, TXT and JSON
1 2 3 5
4
Robot check input
files for data
accuracy
Check staging
data source for
completeness,
correctness, and
data integrity
Check Tableau
data sources for
completeness,
correctness, and
data integrity
Check data
warehouse for
completeness,
correctness, and
data integrity
A user starts data
validation
automation from
Test Manager.
Use Case - Snowflake to Tableau data validation
Excel, CSV,
JSON, TXT
23. 23
Automate Database Testing
Preparation Unit / Field Tests
Testing / Migration
Phase
Data Reconciliation
Reporting & Analytics
Testing
Monitoring
Types of
Automated Tests
• Metadata Checks
• Formatting
Validation
• Row by Row
comparison of data
• Comparison of whole
datasets
• Validating selector
data using UI
Automation
• Visual checks
• Verifying graphical
data
• Track incremental data
volume
(# of Rows)
• Data distribution over
time
• Taking action to
minimize risk upon
failed tests
• Data Completeness:
Row Count, Min/Max
• Data Correctness:
Unique, Not Null, Data
Range
• Data Integrity:
Relational, Domain
RFC
BAPI
SOAP
REST
OData
HTTP
JSON
XML
24. 24
Use Case - UiPath Test Suite Based Automated Solution
1 2 3
Cloud Staging Test Database
Phase 2
o Test robots run dynamic and standard SQL Queries
o The test robot obtains the results and mark as
expected results.
o Test robot compare actual and expected results.
o Test robot raises defects for deviations.
o Test robot reruns queries for defect fixes and
regression tests.
Cloud Production Database
Phase 3
o Production robots run dynamic and standard SQL
queries.
o The test robots obtains the results and mark as
expected results.
o Production robots compare actual and expected
results.
o Production robots raise defect for deviations.
o Production robots reruns queries for defect fixes +
regression tests.
On premise Test Database
Phase 1
o Get the Scope Definition
• No of Tables
• Table Names
• Table Definition
• Transformation rules
o Table names stored in an external data store (Excel
file)
o Dynamic SQL queries with parameterized table
names for each type of validation (row count, col
count, constraints, duplicate check, not null)
o Standard SQL Queries for transformation logic
validation
o The test robot runs the dynamic and standard SQL
queries.
o The robot obtains the results and baseline them as
expected results then stores results in external data
store (Excel).
25. 25
Example 1 – Table Structure Validation
SELECT COLUMN_NAME, DATA_TYPE ,
CHARACTER_MAXIMUM_LENGTH, IS NULLABLE FROM
INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME =
“ABC_Employee_Table”
SELECT COLUMN_NAME, DATA_TYPE ,
CHARACTER_MAXIMUM_LENGTH, IS NULLABLE FROM
INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME =
“ABC_Patient_Table”
SELECT COLUMN_NAME, DATA_TYPE ,
CHARACTER_MAXIMUM_LENGTH, IS NULLABLE FROM
INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME =
“ABC_Department_Table”
.
.
SELECT COLUMN_NAME, DATA_TYPE ,
CHARACTER_MAXIMUM_LENGTH, IS NULLABLE FROM
INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME =
“+TABLE_NAME”
EXPECTED RESULTS
Manual export to multiple xls files
Automated export to one xls file
with multiple sheets, sheet name
same as table name
# MASTER_TABLE_NAME Schema Check
(Y/N)
1 ABC_Employee_Table Y
2 ABC_Patient_Table N
. ABC_................... N
n ABC_Department_Table Y
EXPECTED RESULTS
Repeat Step like Step 1 , Step 2, Step n in the Target Environment
n
Manual export to multiple xls files
ACTUAL RESULTS
Compare Actual and Expected Results & File defects manually
EXPECTED RESULTS
Repeat Step 1 in the Target Environment
ACTUAL RESULTS
Automated export to one xls file
with multiple sheets, sheet name
same as table name
Compare Actual and Expected Results & File defects
2
1
MANUAL TESTING AUTOMATED TESTING
ACTUAL RESULTS
26. 26
Example 2 – Duplicate Record Check
SELECT Emp_No FROM ABC_Employee_Table GROUP BY
Emp_No, Dept_Number HAVING COUNT(*) > 1
SELECT Patient_No FROM ABC_Patient_Table GROUP BY
Patient_No HAVING COUNT(*) > 1
SELECT Department_No FROM ABC_Deparment_Table GROUP BY
Department_No HAVING COUNT(*) > 1
.
. SELECT “+COLUMN_NAME”, FROM “+TABLE_NAME”
GROUP BY “+COLUMN_NAME” HAVING COUNT (*) > 1
ACTUAL RESULTS
Manual export to multiple xls files
Automated export to one xls file
with multiple sheets, sheet name
same as table name
# DUPCHECK_TABLE_NAME Col Name Dup
Check
(Y/N)
1 ABC_Employee_Table Emp_No Y
2 ABC_Patient_Table Patient_No Y
. ABC_................... N
n ABC_Department_Table Department_No Y
ACTUAL RESULTS
Raise Defect based on business rules defined
n
ACTUAL RESULTS
Raise Defect based on business rules defined
2
1
MANUAL TESTING AUTOMATED TESTING
28. 28
Resources and Helpful Links
Snowflake and UiPath Overview page:
https://www.uipath.com/solutions/technology/snowflake-automation
UiPath Integration Service Documentation:
https://docs.uipath.com/integrations/docs/uipath-snowflake-snowflake
UiPath Database Activities Documentation:
https://docs.uipath.com/activities/docs/about-the-database-activities-pack
UiPath Postman Connector Guide:
https://postman.uipath.rocks/
29. 29
Vibrant ecosystem of more than 1,7 million professionals and citizen developers
learning, getting support, and succeeding together in their automation careers.
• Start with the free Community Edition to get trained and certified
• Then upgrade to the Enterprise version of the product
Academy
• Get crowdsourced support and share product feedback on UiPath Forum
• Check the product documentation
• Join the Insider Preview for early testing
Forum
Community Events
• Access the latest articles and video tutorial content created by community members and
UiPath engineers in our Community Blog
• Contribute as an author.
UiPath Community MVPs • Get recognized as a Most Valuable Professional (MVP), Automation Champion or one of
the Forum Leaders, based on the contribution to others’ growth
Join the UiPath Community
• Connect with like-minded people and share best practices with the UiPath Community
• Solve challenges in engaging hackathon competitions
• Join meetups and conferences
Blog and Tutorials
Automation Cloud
• Learn the skills of the future on UiPath Academy or through our Academic Alliance
• Earn globally recognized credentials with UiPath Certifications
30. 30
Date/Time Topic Status
Oct 13th,
16PM GMT
Intelligently automate core business apps with
UiPath and OutSystems
Recording available
Oct 27th,
16PM GMT
Innovate with the data you have with UiPath and
Snowflake
Happening now
Nov 3rd,
16PM GMT
Establish a 360-view of your data with UiPath
and Tableau
Register now
Nov 17th,
16PM GMT
Operationalize AI and ML with the combined
power of UiPath and Amazon SageMaker
Register now
Dec 1st,
16PM GMT
Extend the HR and Finance experience with
UiPath and Workday
Register now
Dec 15th,
16PM GMT
Modernize and transform user experience with
UiPath and AWS in Public Sector
Register now
32. 32
UiPath Community and UiPath Academic Alliance invite you to immerse yourself in this
first-time virtual event, where you’ll have the opportunity to engage with and be inspired by
a passionate community of certified educators and learning professionals.
Automation Education Summit
Register and join us in shaping the
future of education with automation:
https://bit.ly/UiPathEduSummit