SlideShare ist ein Scribd-Unternehmen logo
1 von 20
如何快速把你的 Data Warehouse 變成 API
VulcanSQL
Eason Kuo
Github: kokokuo
LinkedIn: Eason Kuo
VulcanSQL
Motivation
現今多數 Data Analyst / Data Engineer 的困擾與挑戰
Data Warehouse
Data Lake
Dev Engineer
Sales / Marketing
Data Analyst /
Data Engineer
Data Transform
Database / Files
.
.
.
Authorization
Inconsistency
Data Analyst /
Data Engineer
Sales
Engineer
Need “products“ but given only 3 columns
Need “products“ but given only 5 columns
Same Data Source, Different Columns
Authorization Inconsistency – Case 1
Same Data Source, Same Columns, Different results
Authorization Inconsistency – Case 2
Data Analyst /
Data Engineer
Sales
Sales Manager
Need “sold_products“ but only given per sales sold counts
Need “sold_products “ but given all sales per sold counts
Different Data Sources….
Authorization Inconsistency – Case 3
Data Analyst /
Data Engineer
2C Business
Development
Need “orders“ from BigQuery
Need “orders“ from Postgres
2B Business
Development
Before
Data Warehouse
Data Lake
Dev Engineer
Sales / Marketing
Data Analyst /
Data Engineer
Data Transform
Database / Files
.
.
.
Authorization
Inconsistency
Before
Data Warehouse
Data Lake
Dev Engineer
Sales / Marketing
Data Analyst /
Data Engineer
Data Transform
Database / Files
.
.
.
Authorization
Inconsistency
Build API and
Write Document
by Hand
Use Data Pipeline or
Create Web system
Create multiple Tables /
Use Windows ACL to
Control
Fast. Secure. Self-Serve
VulcanSQL
Github
After
Data Warehouse
Data Lake
Dev Engineer
Sales / Marketing
Data Analyst /
Data Engineer
Data Transform
Database / Files
.
.
.
Web API & API Doc
SELECT
--- masking address if
query user is not admin
{% if context.user ==
'ADMIN' %}
{% "address" %}
{% elif %}
{%
"masking(address)" %}
{% endif %},
orderId,
amount
FROM orders
--- limit the data to the
store user belongs to.
WHERE store =
{{ context.user.store }}
Access Control
Authorization
Catalog Page
Connect App / Download
Only Use VulcanSQL Part
What is VulcanSQL ?
Build API instantly with just SQL.
“Instant API on Data Warehouse / Data Lake”
Provide API best practices
(dynamic parameters , validation, error response, etc. )
Access control & authorization in SQL.
Self-serve documentation and catalog.
Connect to your Apps or Download Files
What is VulcanSQL ?
Build API instantly with just SQL.
“Instant API on Data Warehouse / Data Lake”
Provide API best practices
(dynamic parameters , validation, error response, etc. )
Access control & authorization in SQL.
Self-serve documentation and catalog.
Connect to your Apps or Download Files
API Schema
SQL
What is VulcanSQL ?
Build API instantly with just SQL.
“Instant API on Data Warehouse / Data Lake”
Provide API best practices
(dynamic parameters , validation, error response, etc. )
Access control & authorization in SQL.
Self-serve documentation and catalog.
Connect to your Apps or Download Files
SQL
Profiles
What is VulcanSQL ?
Build API instantly with just SQL.
Provide API best practices
(dynamic parameters , validation, error response, etc. )
Access control & authorization in SQL.
Self-serve documentation and catalog.
Connect to your Apps or Download Files
“Instant API on Data Warehouse / Data Lake”
Catalog Page ( Coming Next Release )
API Document
What is VulcanSQL ?
Build API instantly with just SQL.
Provide API best practices
(dynamic parameters , validation, error response, etc. )
Access control & authorization in SQL.
Self-serve documentation and catalog.
Connect to your Apps or Download Files
“Instant API on Data Warehouse / Data Lake”
Connect to Apps ( Coming Next Release )
How VulcanSQL works?
2. Write SQL & Schema
1. Initialize 3. Start Web API
API Doc
Catalog Page
VulcanSQL Official site ready ! ( vulcansql.com )
Contributors for VulcanSQL Now
Next is your
Contribution
Fast. Secure. Self-Serve
VulcanSQL
Get started to try VulcanSQL
Join our developer community
Join the discord group
Chat with VulcanSQL Team
Star VulcanSQL
Read Documentation
Play Demo Code
vulcansql.com
Contribute together
Get started to try VulcanSQL
Join our developer community
Join the discord group
Read Documentation
Play Demo Code
vulcansql.com
FAQ
Thank you
Contribute together
Chat with VulcanSQL Team
Star VulcanSQL

Weitere ähnliche Inhalte

Was ist angesagt?

[系列活動] 智慧製造與生產線上的資料科學 (製造資料科學:從預測性思維到處方性決策)
[系列活動] 智慧製造與生產線上的資料科學 (製造資料科學:從預測性思維到處方性決策)[系列活動] 智慧製造與生產線上的資料科學 (製造資料科學:從預測性思維到處方性決策)
[系列活動] 智慧製造與生產線上的資料科學 (製造資料科學:從預測性思維到處方性決策)
台灣資料科學年會
 
現場改善與管理寶典
現場改善與管理寶典現場改善與管理寶典
現場改善與管理寶典
營松 林
 

Was ist angesagt? (20)

【Intern Event:從《通膨削減法案》看美國電動車產業的發展 】
【Intern Event:從《通膨削減法案》看美國電動車產業的發展 】【Intern Event:從《通膨削減法案》看美國電動車產業的發展 】
【Intern Event:從《通膨削減法案》看美國電動車產業的發展 】
 
先駆者に学ぶ MLOpsの実際
先駆者に学ぶ MLOpsの実際先駆者に学ぶ MLOpsの実際
先駆者に学ぶ MLOpsの実際
 
ISO13485 醫療器材品質管理系統改版重點 part II
ISO13485 醫療器材品質管理系統改版重點 part IIISO13485 醫療器材品質管理系統改版重點 part II
ISO13485 醫療器材品質管理系統改版重點 part II
 
[系列活動] 智慧製造與生產線上的資料科學 (製造資料科學:從預測性思維到處方性決策)
[系列活動] 智慧製造與生產線上的資料科學 (製造資料科學:從預測性思維到處方性決策)[系列活動] 智慧製造與生產線上的資料科學 (製造資料科學:從預測性思維到處方性決策)
[系列活動] 智慧製造與生產線上的資料科學 (製造資料科學:從預測性思維到處方性決策)
 
アジャイル開発と品質保証の密なる関係 #quesqa
アジャイル開発と品質保証の密なる関係 #quesqaアジャイル開発と品質保証の密なる関係 #quesqa
アジャイル開発と品質保証の密なる関係 #quesqa
 
電子商務個案研究報告 Lativ米格國際
電子商務個案研究報告 Lativ米格國際電子商務個案研究報告 Lativ米格國際
電子商務個案研究報告 Lativ米格國際
 
Elastic APM:ログとメトリックから全体像を描き出す
Elastic APM:ログとメトリックから全体像を描き出すElastic APM:ログとメトリックから全体像を描き出す
Elastic APM:ログとメトリックから全体像を描き出す
 
現場改善與管理寶典
現場改善與管理寶典現場改善與管理寶典
現場改善與管理寶典
 
如何將實證融入教學中
如何將實證融入教學中如何將實證融入教學中
如何將實證融入教學中
 
ייעוץ שיווקי לבניית אסטרטגייה ותוכנית שיווקית
ייעוץ שיווקי לבניית אסטרטגייה ותוכנית שיווקיתייעוץ שיווקי לבניית אסטרטגייה ותוכנית שיווקית
ייעוץ שיווקי לבניית אסטרטגייה ותוכנית שיווקית
 
也許你需要的是多一點瀑布 - 敏捷八不
也許你需要的是多一點瀑布 - 敏捷八不也許你需要的是多一點瀑布 - 敏捷八不
也許你需要的是多一點瀑布 - 敏捷八不
 
楽天のデータサイエンティスト@SAS Forum 2019
楽天のデータサイエンティスト@SAS Forum 2019楽天のデータサイエンティスト@SAS Forum 2019
楽天のデータサイエンティスト@SAS Forum 2019
 
現場料品管理 material control
現場料品管理 material control現場料品管理 material control
現場料品管理 material control
 
ICCV19読み会 "Learning Single Camera Depth Estimation using Dual-Pixels"
ICCV19読み会 "Learning Single Camera Depth Estimation using Dual-Pixels"ICCV19読み会 "Learning Single Camera Depth Estimation using Dual-Pixels"
ICCV19読み会 "Learning Single Camera Depth Estimation using Dual-Pixels"
 
003 How to write 8D Report
003 How to write 8D Report003 How to write 8D Report
003 How to write 8D Report
 
製造業智慧化發展策略
 製造業智慧化發展策略 製造業智慧化發展策略
製造業智慧化發展策略
 
【卒業論文】B2Bオークションにおけるユーザ別 入札行動予測に関する研究
【卒業論文】B2Bオークションにおけるユーザ別 入札行動予測に関する研究【卒業論文】B2Bオークションにおけるユーザ別 入札行動予測に関する研究
【卒業論文】B2Bオークションにおけるユーザ別 入札行動予測に関する研究
 
數位轉型的困難與挑戰
數位轉型的困難與挑戰數位轉型的困難與挑戰
數位轉型的困難與挑戰
 
ここがスゴい(変だ)よ!Git lab!
ここがスゴい(変だ)よ!Git lab!ここがスゴい(変だ)よ!Git lab!
ここがスゴい(変だ)よ!Git lab!
 
大企業アジャイルの勘所 #devlovex #devlovexd
大企業アジャイルの勘所 #devlovex #devlovexd大企業アジャイルの勘所 #devlovex #devlovexd
大企業アジャイルの勘所 #devlovex #devlovexd
 

Ähnlich wie VulcanSQL- 如何快速把你的 Data Warehouse 變成 API

民间秘方
民间秘方民间秘方
民间秘方
dynasty
 
数据库性能诊断的七种武器
数据库性能诊断的七种武器数据库性能诊断的七种武器
数据库性能诊断的七种武器
Leyi (Kamus) Zhang
 
淘宝数据库架构演进历程
淘宝数据库架构演进历程淘宝数据库架构演进历程
淘宝数据库架构演进历程
Jian Peng
 
Struts+Spring+Hibernate整合教程
Struts+Spring+Hibernate整合教程Struts+Spring+Hibernate整合教程
Struts+Spring+Hibernate整合教程
yiditushe
 
Struts+Spring+Hibernate整合教程
Struts+Spring+Hibernate整合教程Struts+Spring+Hibernate整合教程
Struts+Spring+Hibernate整合教程
appollo0312
 
API Survey #2 - Firebase realtime database
API Survey #2 - Firebase realtime databaseAPI Survey #2 - Firebase realtime database
API Survey #2 - Firebase realtime database
Szuping Wang
 

Ähnlich wie VulcanSQL- 如何快速把你的 Data Warehouse 變成 API (20)

民间秘方
民间秘方民间秘方
民间秘方
 
数据库性能诊断的七种武器
数据库性能诊断的七种武器数据库性能诊断的七种武器
数据库性能诊断的七种武器
 
Delta Lake Architecture: Delta Lake + Apache Spark Structured Streaming
Delta Lake Architecture: Delta Lake + Apache Spark Structured StreamingDelta Lake Architecture: Delta Lake + Apache Spark Structured Streaming
Delta Lake Architecture: Delta Lake + Apache Spark Structured Streaming
 
選擇正確的Solution 來建置現代化的雲端資料倉儲
選擇正確的Solution 來建置現代化的雲端資料倉儲選擇正確的Solution 來建置現代化的雲端資料倉儲
選擇正確的Solution 來建置現代化的雲端資料倉儲
 
JIRA Live DEMO 2020 v17
JIRA Live DEMO 2020 v17JIRA Live DEMO 2020 v17
JIRA Live DEMO 2020 v17
 
Oracle db 12c 加速企业转型之十大功能
Oracle db 12c 加速企业转型之十大功能Oracle db 12c 加速企业转型之十大功能
Oracle db 12c 加速企业转型之十大功能
 
淘宝数据库架构演进历程
淘宝数据库架构演进历程淘宝数据库架构演进历程
淘宝数据库架构演进历程
 
淘宝数据库架构演进历程
淘宝数据库架构演进历程淘宝数据库架构演进历程
淘宝数据库架构演进历程
 
Analytics in a Day.pptx
Analytics in a Day.pptxAnalytics in a Day.pptx
Analytics in a Day.pptx
 
Oracle sql developer支持第三方数据库
Oracle  sql developer支持第三方数据库Oracle  sql developer支持第三方数据库
Oracle sql developer支持第三方数据库
 
Azure Data Lake 簡介
Azure Data Lake 簡介Azure Data Lake 簡介
Azure Data Lake 簡介
 
Jira live demo 2021 v23
Jira live demo 2021 v23Jira live demo 2021 v23
Jira live demo 2021 v23
 
Struts+Spring+Hibernate整合教程
Struts+Spring+Hibernate整合教程Struts+Spring+Hibernate整合教程
Struts+Spring+Hibernate整合教程
 
Struts+Spring+Hibernate整合教程
Struts+Spring+Hibernate整合教程Struts+Spring+Hibernate整合教程
Struts+Spring+Hibernate整合教程
 
(宇宏)Talend工具說明
(宇宏)Talend工具說明(宇宏)Talend工具說明
(宇宏)Talend工具說明
 
客戶常見問題分享與解決
客戶常見問題分享與解決客戶常見問題分享與解決
客戶常見問題分享與解決
 
深入淺出 AWS 大數據工具
深入淺出 AWS 大數據工具深入淺出 AWS 大數據工具
深入淺出 AWS 大數據工具
 
初探 Data API Builder:在幾分鐘內將資料庫轉換成 REST 和 GraphQL 不再是夢想
初探 Data API Builder:在幾分鐘內將資料庫轉換成 REST 和 GraphQL 不再是夢想初探 Data API Builder:在幾分鐘內將資料庫轉換成 REST 和 GraphQL 不再是夢想
初探 Data API Builder:在幾分鐘內將資料庫轉換成 REST 和 GraphQL 不再是夢想
 
API Survey #2 - Firebase realtime database
API Survey #2 - Firebase realtime databaseAPI Survey #2 - Firebase realtime database
API Survey #2 - Firebase realtime database
 
在Windows azure平台上進行資料庫處理及架構設計
在Windows azure平台上進行資料庫處理及架構設計在Windows azure平台上進行資料庫處理及架構設計
在Windows azure平台上進行資料庫處理及架構設計
 

VulcanSQL- 如何快速把你的 Data Warehouse 變成 API

  • 1. 如何快速把你的 Data Warehouse 變成 API VulcanSQL Eason Kuo Github: kokokuo LinkedIn: Eason Kuo
  • 3. 現今多數 Data Analyst / Data Engineer 的困擾與挑戰 Data Warehouse Data Lake Dev Engineer Sales / Marketing Data Analyst / Data Engineer Data Transform Database / Files . . . Authorization Inconsistency
  • 4. Data Analyst / Data Engineer Sales Engineer Need “products“ but given only 3 columns Need “products“ but given only 5 columns Same Data Source, Different Columns Authorization Inconsistency – Case 1
  • 5. Same Data Source, Same Columns, Different results Authorization Inconsistency – Case 2 Data Analyst / Data Engineer Sales Sales Manager Need “sold_products“ but only given per sales sold counts Need “sold_products “ but given all sales per sold counts
  • 6. Different Data Sources…. Authorization Inconsistency – Case 3 Data Analyst / Data Engineer 2C Business Development Need “orders“ from BigQuery Need “orders“ from Postgres 2B Business Development
  • 7. Before Data Warehouse Data Lake Dev Engineer Sales / Marketing Data Analyst / Data Engineer Data Transform Database / Files . . . Authorization Inconsistency
  • 8. Before Data Warehouse Data Lake Dev Engineer Sales / Marketing Data Analyst / Data Engineer Data Transform Database / Files . . . Authorization Inconsistency Build API and Write Document by Hand Use Data Pipeline or Create Web system Create multiple Tables / Use Windows ACL to Control
  • 10. After Data Warehouse Data Lake Dev Engineer Sales / Marketing Data Analyst / Data Engineer Data Transform Database / Files . . . Web API & API Doc SELECT --- masking address if query user is not admin {% if context.user == 'ADMIN' %} {% "address" %} {% elif %} {% "masking(address)" %} {% endif %}, orderId, amount FROM orders --- limit the data to the store user belongs to. WHERE store = {{ context.user.store }} Access Control Authorization Catalog Page Connect App / Download Only Use VulcanSQL Part
  • 11. What is VulcanSQL ? Build API instantly with just SQL. “Instant API on Data Warehouse / Data Lake” Provide API best practices (dynamic parameters , validation, error response, etc. ) Access control & authorization in SQL. Self-serve documentation and catalog. Connect to your Apps or Download Files
  • 12. What is VulcanSQL ? Build API instantly with just SQL. “Instant API on Data Warehouse / Data Lake” Provide API best practices (dynamic parameters , validation, error response, etc. ) Access control & authorization in SQL. Self-serve documentation and catalog. Connect to your Apps or Download Files API Schema SQL
  • 13. What is VulcanSQL ? Build API instantly with just SQL. “Instant API on Data Warehouse / Data Lake” Provide API best practices (dynamic parameters , validation, error response, etc. ) Access control & authorization in SQL. Self-serve documentation and catalog. Connect to your Apps or Download Files SQL Profiles
  • 14. What is VulcanSQL ? Build API instantly with just SQL. Provide API best practices (dynamic parameters , validation, error response, etc. ) Access control & authorization in SQL. Self-serve documentation and catalog. Connect to your Apps or Download Files “Instant API on Data Warehouse / Data Lake” Catalog Page ( Coming Next Release ) API Document
  • 15. What is VulcanSQL ? Build API instantly with just SQL. Provide API best practices (dynamic parameters , validation, error response, etc. ) Access control & authorization in SQL. Self-serve documentation and catalog. Connect to your Apps or Download Files “Instant API on Data Warehouse / Data Lake” Connect to Apps ( Coming Next Release )
  • 16. How VulcanSQL works? 2. Write SQL & Schema 1. Initialize 3. Start Web API API Doc Catalog Page
  • 17. VulcanSQL Official site ready ! ( vulcansql.com )
  • 18. Contributors for VulcanSQL Now Next is your Contribution
  • 19. Fast. Secure. Self-Serve VulcanSQL Get started to try VulcanSQL Join our developer community Join the discord group Chat with VulcanSQL Team Star VulcanSQL Read Documentation Play Demo Code vulcansql.com Contribute together
  • 20. Get started to try VulcanSQL Join our developer community Join the discord group Read Documentation Play Demo Code vulcansql.com FAQ Thank you Contribute together Chat with VulcanSQL Team Star VulcanSQL