Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Spark on sql server?

477 Aufrufe

Veröffentlicht am

2018/12/22 meetup app Odaka@3
https://meetupapp.connpass.com/event/112659/

Veröffentlicht in: Technologie
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (Unlimited) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download Full EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ACCESS WEBSITE for All Ebooks ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (Unlimited) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... Download Full EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ACCESS WEBSITE for All Ebooks ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... Download EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... Download doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Gehören Sie zu den Ersten, denen das gefällt!

Spark on sql server?

  1. 1. Spark on SQL Server? meetup app Osaka@3 2018/12/22 @shinsukeoda
  2. 2. SQL Server 2019 最近 CTP 2.2 出てます https://docs.microsoft.com/en-us/sql/sql- server/sql-server-ver15-release- notes?view=sqlallproducts-allversions
  3. 3. 目玉機能は何?
  4. 4. 多くの人は これを推すはず
  5. 5. Big Data Clusters
  6. 6. Big Data Clusters k8s で動かすクラスタ (SQL Server / Spark / HDFS) データの仮想化 Mongo, HDFS, Oracle とかにアクセス Data lake 一つの場所にデータを集約(Shared HDFS) スケールアウトデータマート Compute Plane / Data Plane
  7. 7. https://docs.microsoft.com/en-us/sql/big-data-cluster/big-data-cluster-overview?view=sqlallproducts-allversions
  8. 8. https://docs.microsoft.com/en-us/sql/big-data-cluster/concept-storage- pool?view=sqlallproducts-allversions
  9. 9. Big Data Clusters Spark? Apache Spark http://spark.apache.org/ Spark ジョブ から HDFS/SQL Server の データにアクセス
  10. 10. Spark on SQL Server?
  11. 11. Spark on SQL Server?
  12. 12. Big Data Clusters 知りたい人は… SEの雑記 https://blog.engine er-memo.com/
  13. 13. Java language extension in SQL Server 2019 external_script で Java が呼べるように なった! external_script = 外部スクリプト 2016 で R、2017 で Python 2019 で Java https://docs.microsoft.com/en- us/sql/advanced-analytics/java/extension- java?view=sqlallproducts-allversions
  14. 14. http://sparkjava.com/
  15. 15. 残念なお知らせ 実行は出来たけど別プロセスからアクセ ス出来ない 同じプロセス内からならアクセス可
  16. 16. public static void runSqlServer() { // Spark 設定 get("/hello", (request, response) -> "<h1>Hello Spark!! on SQL Server?</h1>"); // HTTP でアクセス! try { String res = run("http://localhost:4567/hello"); System.out.println(res); } catch (IOException e) { System.out.println(e); } } static OkHttpClient client = new OkHttpClient(); static String run(String url) throws IOException { Request request = new Request.Builder() .url(url) .build(); try (Response response = client.newCall(request).execute()) { return response.body().string(); } }
  17. 17. デモ
  18. 18. https://docs.microsoft.com/en-us/sql/advanced-analytics/concepts/extensibility- framework?view=sqlallproducts-allversions
  19. 19. 実行プロセスは? R RTerm.exe Python Python.exe Java Java.exe ExtHost.exe SQL External Runtime Host
  20. 20. ExtHost.exe インストールパス /MSSQL15.MSSQLSERVER¥MSSQL¥Bi nn¥ExtHost.exe 情報無くて動きよー分からん。
  21. 21. Get-NetTCPConnection と Task Manager の照合
  22. 22. 試した結果 localhost:4567 で ExtHost.exe が待ち 受けてるのが確認出来る。 でも外部プロセスからアクセス不可 Invoke-WebRequest –Uri http://localhost:4567/hello で蹴ると Firewall に阻まれてる感じのレスポンス の遅さ Firewalll 無効にしてもダメ。。
  23. 23. まとめ 惜しい感じで Spark on ~ じゃない。 今のところ SQL Server で Web アプリ は動かせるけど、アクセス出来ないから 使えない。 ちなみに 外部に HTTP アクセスは出来る ので、どっかの API 叩くとかは出来そう データの仮想化!!!!

×