SlideShare ist ein Scribd-Unternehmen logo
1 von 9
Introduction of Distributed Key-Value Storage “ okuyama” Kobe Digital Labo, Inc.              http://www.kdl.co.jp/
BigTable Dynamo Tokyo Ty r ant kumofs okuyama What is okuyama? Distributed Key-Value Storage
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],What is okuyama? -Features
・ 100%Java -Communication Part, Control Part, Data Storage  ・ doesn’t depend on OS -Java Virtual Machine environment ・ Verified on WindowsXP / CentOS5 series -Developed and verified on Windows -Load tested on CentOS ,[object Object]
・ You can choose the way of preservation  to data node Main Data Node 1. Preserve all data to memory - Non-perpetuity type 2. Only preserve the data operation  record to files - Perpetuity type 3. Preserve data themselves to files - Perpetuity type 2. Multiple data preservation form
・  The scale out is possible without the system hung in both mastering nodes and the data nodes. ・  All the data shifts at the scale out etc. are done by the automatic operation.  Main Data Node Slave Data Node Main Data Node Slave Data Node Main Data Node Slave Data Node Main Data Node Slave Data Node Main Data Node Slave Data Node Main Data Node Slave Data Node Add Node Data Shift Add 3. Performance gain by scale out
・ Data Flow Main Master Node Main Data Node Slave Data Node Main Data Node Slave Data Node Main Data Node Slave Data Node Main Data Node Slave Data Node Client Slave Master Node ① Input Data 4. Composition where SPOF doesn't exist
5. Function of managing collectively
・  It is not only a relation of Key-Value! >You can add Tags set (Key=“okuyama”, Tag={“oss”, ”kvs”}, Value=“Ditributed KVS”);  set (Key=“httpd”, Tag={“oss”, ”webserver”}, Value=“Typical WebSV”);  getTagKeys(“oss”);   >Result  {“okuyama”, ”httpd”} You can get all keys, resistered in same tag Data can be grouped! 6.Unique function

Weitere ähnliche Inhalte

Was ist angesagt?

Myrocks in the wild wild west! FOSDEM 2020
Myrocks in the wild wild west! FOSDEM 2020Myrocks in the wild wild west! FOSDEM 2020
Myrocks in the wild wild west! FOSDEM 2020Alkin Tezuysal
 
Shared Hosting Hepsia.Co plan and feature
Shared Hosting Hepsia.Co plan and featureShared Hosting Hepsia.Co plan and feature
Shared Hosting Hepsia.Co plan and featureAgus R
 
Mysql 8 vs Mariadb 10.4 Highload++ 2019
Mysql 8 vs Mariadb 10.4 Highload++ 2019Mysql 8 vs Mariadb 10.4 Highload++ 2019
Mysql 8 vs Mariadb 10.4 Highload++ 2019Alkin Tezuysal
 
Mysql 8 vs Mariadb 10.4 Webinar 2020 Feb
Mysql 8 vs Mariadb 10.4 Webinar 2020 FebMysql 8 vs Mariadb 10.4 Webinar 2020 Feb
Mysql 8 vs Mariadb 10.4 Webinar 2020 FebAlkin Tezuysal
 
Cassandra Lunch #92: Securing Apache Cassandra - Managing Roles and Permissions
Cassandra Lunch #92: Securing Apache Cassandra - Managing Roles and PermissionsCassandra Lunch #92: Securing Apache Cassandra - Managing Roles and Permissions
Cassandra Lunch #92: Securing Apache Cassandra - Managing Roles and PermissionsAnant Corporation
 
Ovh webmail | Ovh Server
Ovh webmail | Ovh ServerOvh webmail | Ovh Server
Ovh webmail | Ovh Servernewfasthost
 
RelStorage Plone Zope RDB Storage Backend
RelStorage Plone Zope RDB Storage BackendRelStorage Plone Zope RDB Storage Backend
RelStorage Plone Zope RDB Storage BackendJens Klein
 
Configuring MongoDB HA Replica Set on AWS EC2
Configuring MongoDB HA Replica Set on AWS EC2Configuring MongoDB HA Replica Set on AWS EC2
Configuring MongoDB HA Replica Set on AWS EC2ShepHertz
 
Scaling WordPress On A Small Budget
Scaling WordPress On A Small BudgetScaling WordPress On A Small Budget
Scaling WordPress On A Small BudgetBrecht Ryckaert
 
How to backup Oracle Database to Dropbox, Windows Azure, Amazon S3, and local...
How to backup Oracle Database to Dropbox, Windows Azure, Amazon S3, and local...How to backup Oracle Database to Dropbox, Windows Azure, Amazon S3, and local...
How to backup Oracle Database to Dropbox, Windows Azure, Amazon S3, and local...Adeline Wong
 
Scaling MongoDB in the cloud with Microsoft Azure
Scaling MongoDB in the cloud with Microsoft AzureScaling MongoDB in the cloud with Microsoft Azure
Scaling MongoDB in the cloud with Microsoft AzureIvan Fioravanti
 
Oracle golden gate comparison
Oracle golden gate comparisonOracle golden gate comparison
Oracle golden gate comparisonDoddi Priyambodo
 
Optimizing Your WordPress Site
Optimizing Your WordPress SiteOptimizing Your WordPress Site
Optimizing Your WordPress Siteozzyr
 
Introduction to Elastic Beanstalk
Introduction to Elastic BeanstalkIntroduction to Elastic Beanstalk
Introduction to Elastic BeanstalkWolfgang Schell
 
Semide dicated catalog server
Semide dicated catalog serverSemide dicated catalog server
Semide dicated catalog serverAgus R
 
Mysql 2007 Tech At Digg V3
Mysql 2007 Tech At Digg V3Mysql 2007 Tech At Digg V3
Mysql 2007 Tech At Digg V3epee
 
Insight on MongoDB Change Stream - Abhishek.D, Mydbops Team
Insight on MongoDB Change Stream - Abhishek.D, Mydbops TeamInsight on MongoDB Change Stream - Abhishek.D, Mydbops Team
Insight on MongoDB Change Stream - Abhishek.D, Mydbops TeamMydbops
 
Continuous Deployment with Cassandra
Continuous Deployment with CassandraContinuous Deployment with Cassandra
Continuous Deployment with CassandraMichael Kjellman
 
MongoDB at community engine
MongoDB at community engineMongoDB at community engine
MongoDB at community enginemathraq
 

Was ist angesagt? (20)

Myrocks in the wild wild west! FOSDEM 2020
Myrocks in the wild wild west! FOSDEM 2020Myrocks in the wild wild west! FOSDEM 2020
Myrocks in the wild wild west! FOSDEM 2020
 
Shared Hosting Hepsia.Co plan and feature
Shared Hosting Hepsia.Co plan and featureShared Hosting Hepsia.Co plan and feature
Shared Hosting Hepsia.Co plan and feature
 
Mysql 8 vs Mariadb 10.4 Highload++ 2019
Mysql 8 vs Mariadb 10.4 Highload++ 2019Mysql 8 vs Mariadb 10.4 Highload++ 2019
Mysql 8 vs Mariadb 10.4 Highload++ 2019
 
Mysql 8 vs Mariadb 10.4 Webinar 2020 Feb
Mysql 8 vs Mariadb 10.4 Webinar 2020 FebMysql 8 vs Mariadb 10.4 Webinar 2020 Feb
Mysql 8 vs Mariadb 10.4 Webinar 2020 Feb
 
Cassandra Lunch #92: Securing Apache Cassandra - Managing Roles and Permissions
Cassandra Lunch #92: Securing Apache Cassandra - Managing Roles and PermissionsCassandra Lunch #92: Securing Apache Cassandra - Managing Roles and Permissions
Cassandra Lunch #92: Securing Apache Cassandra - Managing Roles and Permissions
 
Ovh webmail | Ovh Server
Ovh webmail | Ovh ServerOvh webmail | Ovh Server
Ovh webmail | Ovh Server
 
RelStorage Plone Zope RDB Storage Backend
RelStorage Plone Zope RDB Storage BackendRelStorage Plone Zope RDB Storage Backend
RelStorage Plone Zope RDB Storage Backend
 
Configuring MongoDB HA Replica Set on AWS EC2
Configuring MongoDB HA Replica Set on AWS EC2Configuring MongoDB HA Replica Set on AWS EC2
Configuring MongoDB HA Replica Set on AWS EC2
 
Scaling WordPress On A Small Budget
Scaling WordPress On A Small BudgetScaling WordPress On A Small Budget
Scaling WordPress On A Small Budget
 
How to backup Oracle Database to Dropbox, Windows Azure, Amazon S3, and local...
How to backup Oracle Database to Dropbox, Windows Azure, Amazon S3, and local...How to backup Oracle Database to Dropbox, Windows Azure, Amazon S3, and local...
How to backup Oracle Database to Dropbox, Windows Azure, Amazon S3, and local...
 
Amis puppet building blocks demo for Oracle Database and Weblogic cluster
Amis puppet building blocks demo for Oracle Database and Weblogic clusterAmis puppet building blocks demo for Oracle Database and Weblogic cluster
Amis puppet building blocks demo for Oracle Database and Weblogic cluster
 
Scaling MongoDB in the cloud with Microsoft Azure
Scaling MongoDB in the cloud with Microsoft AzureScaling MongoDB in the cloud with Microsoft Azure
Scaling MongoDB in the cloud with Microsoft Azure
 
Oracle golden gate comparison
Oracle golden gate comparisonOracle golden gate comparison
Oracle golden gate comparison
 
Optimizing Your WordPress Site
Optimizing Your WordPress SiteOptimizing Your WordPress Site
Optimizing Your WordPress Site
 
Introduction to Elastic Beanstalk
Introduction to Elastic BeanstalkIntroduction to Elastic Beanstalk
Introduction to Elastic Beanstalk
 
Semide dicated catalog server
Semide dicated catalog serverSemide dicated catalog server
Semide dicated catalog server
 
Mysql 2007 Tech At Digg V3
Mysql 2007 Tech At Digg V3Mysql 2007 Tech At Digg V3
Mysql 2007 Tech At Digg V3
 
Insight on MongoDB Change Stream - Abhishek.D, Mydbops Team
Insight on MongoDB Change Stream - Abhishek.D, Mydbops TeamInsight on MongoDB Change Stream - Abhishek.D, Mydbops Team
Insight on MongoDB Change Stream - Abhishek.D, Mydbops Team
 
Continuous Deployment with Cassandra
Continuous Deployment with CassandraContinuous Deployment with Cassandra
Continuous Deployment with Cassandra
 
MongoDB at community engine
MongoDB at community engineMongoDB at community engine
MongoDB at community engine
 

Ähnlich wie Okuyama Summary

2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 Geir Høydalsvik
 
Caching and tuning fun for high scalability @ FrOSCon 2011
Caching and tuning fun for high scalability @ FrOSCon 2011Caching and tuning fun for high scalability @ FrOSCon 2011
Caching and tuning fun for high scalability @ FrOSCon 2011Wim Godden
 
Exploring Scalability, Performance And Deployment
Exploring Scalability, Performance And DeploymentExploring Scalability, Performance And Deployment
Exploring Scalability, Performance And Deploymentrsnarayanan
 
Azure + DataStax Enterprise (DSE) Powers Office365 Per User Store
Azure + DataStax Enterprise (DSE) Powers Office365 Per User StoreAzure + DataStax Enterprise (DSE) Powers Office365 Per User Store
Azure + DataStax Enterprise (DSE) Powers Office365 Per User StoreDataStax Academy
 
SSIS Monitoring Deep Dive
SSIS Monitoring Deep DiveSSIS Monitoring Deep Dive
SSIS Monitoring Deep DiveDavide Mauri
 
Caching and tuning fun for high scalability
Caching and tuning fun for high scalabilityCaching and tuning fun for high scalability
Caching and tuning fun for high scalabilityWim Godden
 
Spark in the Maritime Domain
Spark in the Maritime DomainSpark in the Maritime Domain
Spark in the Maritime DomainDemi Ben-Ari
 
Linuxfest Northwest Proper Care and Feeding Of a MySQL for Busy Linux Admins
Linuxfest Northwest Proper Care and Feeding Of a MySQL for Busy Linux AdminsLinuxfest Northwest Proper Care and Feeding Of a MySQL for Busy Linux Admins
Linuxfest Northwest Proper Care and Feeding Of a MySQL for Busy Linux AdminsDave Stokes
 
Caching and tuning fun for high scalability
Caching and tuning fun for high scalabilityCaching and tuning fun for high scalability
Caching and tuning fun for high scalabilityWim Godden
 
Proper Care and Feeding of a MySQL Database for Busy Linux Administrators
Proper Care and Feeding of a MySQL Database for Busy Linux AdministratorsProper Care and Feeding of a MySQL Database for Busy Linux Administrators
Proper Care and Feeding of a MySQL Database for Busy Linux AdministratorsDave Stokes
 
MySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMark Leith
 
MySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMark Leith
 
VMworld 2013: Virtualizing Databases: Doing IT Right
VMworld 2013: Virtualizing Databases: Doing IT Right VMworld 2013: Virtualizing Databases: Doing IT Right
VMworld 2013: Virtualizing Databases: Doing IT Right VMworld
 
EM12C High Availability without SLB and RAC
EM12C High Availability without SLB and RACEM12C High Availability without SLB and RAC
EM12C High Availability without SLB and RACSecure-24
 
DAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI Convergence
DAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI ConvergenceDAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI Convergence
DAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI Convergenceinside-BigData.com
 
The Proper Care and Feeding of a MySQL Database for Busy Linux Admins -- SCaL...
The Proper Care and Feeding of a MySQL Database for Busy Linux Admins -- SCaL...The Proper Care and Feeding of a MySQL Database for Busy Linux Admins -- SCaL...
The Proper Care and Feeding of a MySQL Database for Busy Linux Admins -- SCaL...Dave Stokes
 
MySQL Utilities -- PyTexas 2015
MySQL Utilities -- PyTexas 2015MySQL Utilities -- PyTexas 2015
MySQL Utilities -- PyTexas 2015Dave Stokes
 
TechDay - Toronto 2016 - Hyperconvergence and OpenNebula
TechDay - Toronto 2016 - Hyperconvergence and OpenNebulaTechDay - Toronto 2016 - Hyperconvergence and OpenNebula
TechDay - Toronto 2016 - Hyperconvergence and OpenNebulaOpenNebula Project
 
01 demystifying mysq-lfororacledbaanddeveloperv1
01 demystifying mysq-lfororacledbaanddeveloperv101 demystifying mysq-lfororacledbaanddeveloperv1
01 demystifying mysq-lfororacledbaanddeveloperv1Ivan Ma
 
One daytalk hbraun_oct2011
One daytalk hbraun_oct2011One daytalk hbraun_oct2011
One daytalk hbraun_oct2011hbraun
 

Ähnlich wie Okuyama Summary (20)

2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
 
Caching and tuning fun for high scalability @ FrOSCon 2011
Caching and tuning fun for high scalability @ FrOSCon 2011Caching and tuning fun for high scalability @ FrOSCon 2011
Caching and tuning fun for high scalability @ FrOSCon 2011
 
Exploring Scalability, Performance And Deployment
Exploring Scalability, Performance And DeploymentExploring Scalability, Performance And Deployment
Exploring Scalability, Performance And Deployment
 
Azure + DataStax Enterprise (DSE) Powers Office365 Per User Store
Azure + DataStax Enterprise (DSE) Powers Office365 Per User StoreAzure + DataStax Enterprise (DSE) Powers Office365 Per User Store
Azure + DataStax Enterprise (DSE) Powers Office365 Per User Store
 
SSIS Monitoring Deep Dive
SSIS Monitoring Deep DiveSSIS Monitoring Deep Dive
SSIS Monitoring Deep Dive
 
Caching and tuning fun for high scalability
Caching and tuning fun for high scalabilityCaching and tuning fun for high scalability
Caching and tuning fun for high scalability
 
Spark in the Maritime Domain
Spark in the Maritime DomainSpark in the Maritime Domain
Spark in the Maritime Domain
 
Linuxfest Northwest Proper Care and Feeding Of a MySQL for Busy Linux Admins
Linuxfest Northwest Proper Care and Feeding Of a MySQL for Busy Linux AdminsLinuxfest Northwest Proper Care and Feeding Of a MySQL for Busy Linux Admins
Linuxfest Northwest Proper Care and Feeding Of a MySQL for Busy Linux Admins
 
Caching and tuning fun for high scalability
Caching and tuning fun for high scalabilityCaching and tuning fun for high scalability
Caching and tuning fun for high scalability
 
Proper Care and Feeding of a MySQL Database for Busy Linux Administrators
Proper Care and Feeding of a MySQL Database for Busy Linux AdministratorsProper Care and Feeding of a MySQL Database for Busy Linux Administrators
Proper Care and Feeding of a MySQL Database for Busy Linux Administrators
 
MySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMySQL Monitoring Mechanisms
MySQL Monitoring Mechanisms
 
MySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMySQL Monitoring Mechanisms
MySQL Monitoring Mechanisms
 
VMworld 2013: Virtualizing Databases: Doing IT Right
VMworld 2013: Virtualizing Databases: Doing IT Right VMworld 2013: Virtualizing Databases: Doing IT Right
VMworld 2013: Virtualizing Databases: Doing IT Right
 
EM12C High Availability without SLB and RAC
EM12C High Availability without SLB and RACEM12C High Availability without SLB and RAC
EM12C High Availability without SLB and RAC
 
DAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI Convergence
DAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI ConvergenceDAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI Convergence
DAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI Convergence
 
The Proper Care and Feeding of a MySQL Database for Busy Linux Admins -- SCaL...
The Proper Care and Feeding of a MySQL Database for Busy Linux Admins -- SCaL...The Proper Care and Feeding of a MySQL Database for Busy Linux Admins -- SCaL...
The Proper Care and Feeding of a MySQL Database for Busy Linux Admins -- SCaL...
 
MySQL Utilities -- PyTexas 2015
MySQL Utilities -- PyTexas 2015MySQL Utilities -- PyTexas 2015
MySQL Utilities -- PyTexas 2015
 
TechDay - Toronto 2016 - Hyperconvergence and OpenNebula
TechDay - Toronto 2016 - Hyperconvergence and OpenNebulaTechDay - Toronto 2016 - Hyperconvergence and OpenNebula
TechDay - Toronto 2016 - Hyperconvergence and OpenNebula
 
01 demystifying mysq-lfororacledbaanddeveloperv1
01 demystifying mysq-lfororacledbaanddeveloperv101 demystifying mysq-lfororacledbaanddeveloperv1
01 demystifying mysq-lfororacledbaanddeveloperv1
 
One daytalk hbraun_oct2011
One daytalk hbraun_oct2011One daytalk hbraun_oct2011
One daytalk hbraun_oct2011
 

Okuyama Summary

  • 1. Introduction of Distributed Key-Value Storage “ okuyama” Kobe Digital Labo, Inc.              http://www.kdl.co.jp/
  • 2. BigTable Dynamo Tokyo Ty r ant kumofs okuyama What is okuyama? Distributed Key-Value Storage
  • 3.
  • 4.
  • 5. ・ You can choose the way of preservation to data node Main Data Node 1. Preserve all data to memory - Non-perpetuity type 2. Only preserve the data operation record to files - Perpetuity type 3. Preserve data themselves to files - Perpetuity type 2. Multiple data preservation form
  • 6. ・ The scale out is possible without the system hung in both mastering nodes and the data nodes. ・ All the data shifts at the scale out etc. are done by the automatic operation. Main Data Node Slave Data Node Main Data Node Slave Data Node Main Data Node Slave Data Node Main Data Node Slave Data Node Main Data Node Slave Data Node Main Data Node Slave Data Node Add Node Data Shift Add 3. Performance gain by scale out
  • 7. ・ Data Flow Main Master Node Main Data Node Slave Data Node Main Data Node Slave Data Node Main Data Node Slave Data Node Main Data Node Slave Data Node Client Slave Master Node ① Input Data 4. Composition where SPOF doesn't exist
  • 8. 5. Function of managing collectively
  • 9. ・ It is not only a relation of Key-Value! >You can add Tags set (Key=“okuyama”, Tag={“oss”, ”kvs”}, Value=“Ditributed KVS”); set (Key=“httpd”, Tag={“oss”, ”webserver”}, Value=“Typical WebSV”); getTagKeys(“oss”);   >Result {“okuyama”, ”httpd”} You can get all keys, resistered in same tag Data can be grouped! 6.Unique function

Hinweis der Redaktion

  1. ・ Java で実装された分散キーバリューストア ・データ保存方式の選択が可能 ・スケールアウトによる性能向上 ・ SPOF の存在しない構成 ・一括管理機能 ・ユニークな機能
  2. ・ 100%Java > 通信部分、制御部分、データ保存部分 ・ OS 非依存 >JavaVirtualMachine が動く環境なら動く ・ WindowsXP 系と CentOS5 系で動作検証 > 開発・検証は Windows で負荷テストは CentOS
  3. 1. 全てのデータをメモリに保存 > 非永続型 2 . データ操作履歴のみファイルに保存 > 永続型 3. データ本体をファイルに保存 > 永続型