SlideShare ist ein Scribd-Unternehmen logo
1 von 54
Downloaden Sie, um offline zu lesen
Intravert
                             Experimental
                      Transport for Apache
                                Cassandra


Friday, April 5, 13
Hi Folks!
                      I’m Nate
                      @zznate



Friday, April 5, 13
API Management
                      API Analytics
                      API Tools

Friday, April 5, 13
Vert.x + Cassandra = ?




Friday, April 5, 13
But first:
                      Architectural stuff




Friday, April 5, 13
Collapsed for brevity.
                      see slides 6-31 from this presentation:
                      http://www.slideshare.net/zznate/
                      apachecon-cassandra-transport




Friday, April 5, 13
So there are options,
                      why create something
                      new?



Friday, April 5, 13
Common client
                      patterns emerged




Friday, April 5, 13
union operations
                      aggregations
                      filtering
                      joins (round-trip required!)
                      redundant data on insertion


Friday, April 5, 13
These client patterns
                      are wasteful.




Friday, April 5, 13
Lots of CPU and memory
                      burned (server and client)

                      Lots of data on the wire
                      won’t be used (ser/deSer
                      cost)

Friday, April 5, 13
One more thing about
                      CQL...




Friday, April 5, 13
“If [they] think that CQL is the
                      answer to usability then I just
                      won. We at least know where
                      our problems are.”
                      - 10gen exec.


Friday, April 5, 13
Transports in sum




Friday, April 5, 13
Thrift:
                      I can has distributed
                      computing degree



Friday, April 5, 13
CQL:
                      MOAR TOOLz!!!




Friday, April 5, 13
IMO: In 2013, binary
                      protocols with
                      idiomatic clients
                      misses the market.


Friday, April 5, 13
I love writing apps in
                      Java (sometimes
                      python), but...



Friday, April 5, 13
The most popular
                      application framework
                      these days:



Friday, April 5, 13
The most popular
                      application framework
                      these days:
                      HTML5


Friday, April 5, 13
and thus...
                      Intravert



Friday, April 5, 13
Any new transport had
                      to be HTTP based




Friday, April 5, 13
Easy to use
                      Easy to secure
                      Easy to compress
                      You can test with a
                      browser!

Friday, April 5, 13
The payload had to be
                      JSON based




Friday, April 5, 13
Quick Vert.x summary




Friday, April 5, 13
- Event-driven IO (Netty)
              - Message bus
              - Runtime load/unload of
              modules
              - Polyglot support via
              invokeDynamic in JDK7
                               http://vertx.io
Friday, April 5, 13
Initial use case:
              Server side indexing




Friday, April 5, 13
Easy to construct most
              common operations




Friday, April 5, 13
Simple REST semantics
              POST /myks/mycf/key1
              {“uname”:”zznate”}

              GET /myks/mycf/key1/uname
              {”zznate”}
Friday, April 5, 13
Command-driven JSON
              payload




Friday, April 5, 13
Extensible and transparent
                      Ordered operations
                      Not tied to any predefined
                      notions



Friday, April 5, 13
Benefits of doing this
                      with Vert.x




Friday, April 5, 13
Message bus flexibility
                      Configure which modules
                      to load (read-only nodes!)
                      Hot deploy of fixes and new
                      features


Friday, April 5, 13
Cassandra is best-of-
                      breed for a lot of things.
                      So we used it as a
                      framework.


Friday, April 5, 13
Example Intravert
                      Operations




Friday, April 5, 13
Slice:
              {"id":6,
              "type":"slice",
              "op":{
              	
  	
  	
  "end":"9",
              	
  	
  	
  "rowkey":"5",
              	
  	
  	
  "size":4,
              	
  	
  	
  "start":"1"}
              }

Friday, April 5, 13
Set:
              {"id":2,
              "type":"SET",
              "op":{	
  	
  	
  
              	
  	
  	
  "rowkey":"key1",
              	
  	
  	
  "uname":"zznate",
              	
  	
  	
  "email":"zznate.m@gmail.com",
              	
  	
  	
  "age":37}
              }

Friday, April 5, 13
Easy to use Composites




Friday, April 5, 13
{"id":1,
              "type":"SET",
              "op":{	
  	
  	
  
              	
  	
  	
  "rowkey":"key1",
              	
  	
  	
  "COMP_NAME":[
              	
  	
  	
  	
  	
  "37",	
  "zznate"
              	
  	
  	
  ],
              	
  	
  	
  "value":"key1",
              	
  	
  	
  }
              }
Friday, April 5, 13
One or more commands
              per payload




Friday, April 5, 13
Friday, April 5, 13
Flexible batching
              - batch reads and writes
              - different keyspaces in
              same op
              - change ks/cf mid-op
              - streaming (soon)

Friday, April 5, 13
Server-side Filtering




Friday, April 5, 13
{
                "type": "CREATEFILTER",
                "op": {
                  "name": "stouts",
                  "spec": "javascript",
                  "value": "function(row) { if (row['value'] == 'Breakfast Stout') return row;
                           else return null; }"
                }
           },
           {
                "type": "FILTERMODE",
                "op": {
                  "name": "stouts",
                  "on": true
                }
           },
           {
                "type": "SLICE",
                "op": {
                  "rowkey": "beers",
                  "start": "Allagash",
                  "end": "Sierra Nevada",
                  "size": 9
                }
           }

Friday, April 5, 13
{
                      "exception":null,
                      "exceptionId":null,
                      "opsRes": {
                        "0":"OK",
                        "1":"OK",
                        "2":"OK",
                        "3":"OK",
                        "4":"OK",
                        "5":"OK",
                        "6":[{
                           "name":"Founders",
                           "value":"Breakfast Stout"
                        }]
              }}

Friday, April 5, 13
GetRef:
              Results of one operation as
              input from another




Friday, April 5, 13
{
           "type": "GET",
           "op": {
             "rowkey": "getref_test", "name": "col1"
           }
      },
      {
           "type": "SET",
           "op": {
             "rowkey": "getref_test", "name": "col3",
             "value": {
               "type": "GETREF",
               "op": {
                  "resultref": 4, "wanted": "value"
               }
             }
           }
      },
      {
           "type": "GET",
           "op": {
             "rowkey": "getref_test",
             "name": "col3"
           }
      }

Friday, April 5, 13
“Triggers”
                      - Perform complex multi-step
                      operations server side
                      - Examine and change request
                      objects
                      - Interface directly with IntraVert
                      - Interface directly with Cassandra
                      (StorageProxy, CFMetaData, etc)

Friday, April 5, 13
Multi-Processors:
                      Shape results with a union
                      of operations




Friday, April 5, 13
Prepared Request
                      Statement:
                      Requests saved server side
                      - send only the bind
                      variables


Friday, April 5, 13
CQL support built-in.




Friday, April 5, 13
What’s missing?




Friday, April 5, 13
Security (Shiro integration
                      coming soon)
                      Distribution semantics
                      Cassandra-REST
                      HBASE-like scanners


Friday, April 5, 13
github.com/zznate/
                      intravert-ug




Friday, April 5, 13

Weitere ähnliche Inhalte

Andere mochten auch

2016-JAN-28 -- High Performance Production Databases on Ceph
2016-JAN-28 -- High Performance Production Databases on Ceph2016-JAN-28 -- High Performance Production Databases on Ceph
2016-JAN-28 -- High Performance Production Databases on CephCeph Community
 
ESA and the Cloud
ESA and the CloudESA and the Cloud
ESA and the CloudNetcetera
 
Cassandra in Industry
Cassandra in IndustryCassandra in Industry
Cassandra in Industryniallmilton
 
Advanced Apache Cassandra Operations with JMX
Advanced Apache Cassandra Operations with JMXAdvanced Apache Cassandra Operations with JMX
Advanced Apache Cassandra Operations with JMXzznate
 
Best Practices with Ceph as Distributed, Intelligent, Unified Cloud Storage -...
Best Practices with Ceph as Distributed, Intelligent, Unified Cloud Storage -...Best Practices with Ceph as Distributed, Intelligent, Unified Cloud Storage -...
Best Practices with Ceph as Distributed, Intelligent, Unified Cloud Storage -...Ceph Community
 
Ceph Day SF 2015 - SysAdmin's Toolbox: Tools for Running Ceph in Production
Ceph Day SF 2015 - SysAdmin's Toolbox: Tools for Running Ceph in Production Ceph Day SF 2015 - SysAdmin's Toolbox: Tools for Running Ceph in Production
Ceph Day SF 2015 - SysAdmin's Toolbox: Tools for Running Ceph in Production Ceph Community
 
Docker 101 @KACST Saudi HPC 2016
Docker 101  @KACST Saudi HPC 2016Docker 101  @KACST Saudi HPC 2016
Docker 101 @KACST Saudi HPC 2016Walid Shaari
 
Escape From Hadoop: Spark One Liners for C* Ops
Escape From Hadoop: Spark One Liners for C* OpsEscape From Hadoop: Spark One Liners for C* Ops
Escape From Hadoop: Spark One Liners for C* OpsRussell Spitzer
 
GC Tuning Confessions Of A Performance Engineer - Improved :)
GC Tuning Confessions Of A Performance Engineer - Improved :)GC Tuning Confessions Of A Performance Engineer - Improved :)
GC Tuning Confessions Of A Performance Engineer - Improved :)Monica Beckwith
 
It's not you, it's me: Ending a 15 year relationship with RRD
It's not you, it's me: Ending a 15 year relationship with RRDIt's not you, it's me: Ending a 15 year relationship with RRD
It's not you, it's me: Ending a 15 year relationship with RRDEric Evans
 
A Storage Story #ChefConf2013
A Storage Story #ChefConf2013A Storage Story #ChefConf2013
A Storage Story #ChefConf2013Kyle Bader
 
Epic Move: An AWS Exodus
Epic Move: An AWS ExodusEpic Move: An AWS Exodus
Epic Move: An AWS ExodusDanny Guinther
 
Optimizing Docker Images
Optimizing Docker ImagesOptimizing Docker Images
Optimizing Docker ImagesBrian DeHamer
 
A Year with Cinder and Ceph at TWC
A Year with Cinder and Ceph at TWCA Year with Cinder and Ceph at TWC
A Year with Cinder and Ceph at TWCbryanstillwell
 
Raffi Krikorian, Twitter Timelines at Scale
Raffi Krikorian, Twitter Timelines at ScaleRaffi Krikorian, Twitter Timelines at Scale
Raffi Krikorian, Twitter Timelines at ScaleMariano Amartino
 
Cassandra Workshop - Cassandra from scratch in one day
Cassandra Workshop - Cassandra from scratch in one dayCassandra Workshop - Cassandra from scratch in one day
Cassandra Workshop - Cassandra from scratch in one dayCarlos Alonso Pérez
 
Techtalks: taking docker to production
Techtalks: taking docker to productionTechtalks: taking docker to production
Techtalks: taking docker to productionmuayyad alsadi
 

Andere mochten auch (20)

2016-JAN-28 -- High Performance Production Databases on Ceph
2016-JAN-28 -- High Performance Production Databases on Ceph2016-JAN-28 -- High Performance Production Databases on Ceph
2016-JAN-28 -- High Performance Production Databases on Ceph
 
ESA and the Cloud
ESA and the CloudESA and the Cloud
ESA and the Cloud
 
Cassandra in Industry
Cassandra in IndustryCassandra in Industry
Cassandra in Industry
 
Advanced Apache Cassandra Operations with JMX
Advanced Apache Cassandra Operations with JMXAdvanced Apache Cassandra Operations with JMX
Advanced Apache Cassandra Operations with JMX
 
Best Practices with Ceph as Distributed, Intelligent, Unified Cloud Storage -...
Best Practices with Ceph as Distributed, Intelligent, Unified Cloud Storage -...Best Practices with Ceph as Distributed, Intelligent, Unified Cloud Storage -...
Best Practices with Ceph as Distributed, Intelligent, Unified Cloud Storage -...
 
Ceph Day SF 2015 - SysAdmin's Toolbox: Tools for Running Ceph in Production
Ceph Day SF 2015 - SysAdmin's Toolbox: Tools for Running Ceph in Production Ceph Day SF 2015 - SysAdmin's Toolbox: Tools for Running Ceph in Production
Ceph Day SF 2015 - SysAdmin's Toolbox: Tools for Running Ceph in Production
 
Docker 101 @KACST Saudi HPC 2016
Docker 101  @KACST Saudi HPC 2016Docker 101  @KACST Saudi HPC 2016
Docker 101 @KACST Saudi HPC 2016
 
Escape From Hadoop: Spark One Liners for C* Ops
Escape From Hadoop: Spark One Liners for C* OpsEscape From Hadoop: Spark One Liners for C* Ops
Escape From Hadoop: Spark One Liners for C* Ops
 
GC Tuning Confessions Of A Performance Engineer - Improved :)
GC Tuning Confessions Of A Performance Engineer - Improved :)GC Tuning Confessions Of A Performance Engineer - Improved :)
GC Tuning Confessions Of A Performance Engineer - Improved :)
 
OpenVZ Linux Containers
OpenVZ Linux ContainersOpenVZ Linux Containers
OpenVZ Linux Containers
 
It's not you, it's me: Ending a 15 year relationship with RRD
It's not you, it's me: Ending a 15 year relationship with RRDIt's not you, it's me: Ending a 15 year relationship with RRD
It's not you, it's me: Ending a 15 year relationship with RRD
 
A Storage Story #ChefConf2013
A Storage Story #ChefConf2013A Storage Story #ChefConf2013
A Storage Story #ChefConf2013
 
Cassandra at scale
Cassandra at scaleCassandra at scale
Cassandra at scale
 
Epic Move: An AWS Exodus
Epic Move: An AWS ExodusEpic Move: An AWS Exodus
Epic Move: An AWS Exodus
 
Optimizing Docker Images
Optimizing Docker ImagesOptimizing Docker Images
Optimizing Docker Images
 
A Year with Cinder and Ceph at TWC
A Year with Cinder and Ceph at TWCA Year with Cinder and Ceph at TWC
A Year with Cinder and Ceph at TWC
 
Raffi Krikorian, Twitter Timelines at Scale
Raffi Krikorian, Twitter Timelines at ScaleRaffi Krikorian, Twitter Timelines at Scale
Raffi Krikorian, Twitter Timelines at Scale
 
Graphite cluster setup blueprint
Graphite cluster setup blueprintGraphite cluster setup blueprint
Graphite cluster setup blueprint
 
Cassandra Workshop - Cassandra from scratch in one day
Cassandra Workshop - Cassandra from scratch in one dayCassandra Workshop - Cassandra from scratch in one day
Cassandra Workshop - Cassandra from scratch in one day
 
Techtalks: taking docker to production
Techtalks: taking docker to productionTechtalks: taking docker to production
Techtalks: taking docker to production
 

Ähnlich wie Intravert Experimental Transport for Apache Cassandra

Writing testable code
Writing testable codeWriting testable code
Writing testable codeAlvaro Videla
 
CPANci: Continuous Integration for CPAN
CPANci: Continuous Integration for CPANCPANci: Continuous Integration for CPAN
CPANci: Continuous Integration for CPANMike Friedman
 
The State of Puppet
The State of PuppetThe State of Puppet
The State of PuppetPuppet
 
PHP Conference Argentina 2013 - Independizate de tu departamento IT - Habilid...
PHP Conference Argentina 2013 - Independizate de tu departamento IT - Habilid...PHP Conference Argentina 2013 - Independizate de tu departamento IT - Habilid...
PHP Conference Argentina 2013 - Independizate de tu departamento IT - Habilid...Pablo Godel
 
Microservices and functional programming
Microservices and functional programmingMicroservices and functional programming
Microservices and functional programmingMichael Neale
 
JSUG - ActionScript 3 vs Java by Christoph Pickl
JSUG - ActionScript 3 vs Java by Christoph PicklJSUG - ActionScript 3 vs Java by Christoph Pickl
JSUG - ActionScript 3 vs Java by Christoph PicklChristoph Pickl
 
JSUG - AS3 vs Java by Christoph Pickl
JSUG - AS3 vs Java by Christoph PicklJSUG - AS3 vs Java by Christoph Pickl
JSUG - AS3 vs Java by Christoph PicklChristoph Pickl
 
Hadoop meets Cloud with Multi-Tenancy
Hadoop meets Cloud with Multi-TenancyHadoop meets Cloud with Multi-Tenancy
Hadoop meets Cloud with Multi-TenancyTreasure Data, Inc.
 
Keeping your users happy with testable apps - Greg Shackles
Keeping your users happy with testable apps - Greg ShacklesKeeping your users happy with testable apps - Greg Shackles
Keeping your users happy with testable apps - Greg ShacklesXamarin
 
Puppet and AWS: Getting the best of both worlds
Puppet and AWS: Getting the best of both worldsPuppet and AWS: Getting the best of both worlds
Puppet and AWS: Getting the best of both worldsPuppet
 
The InstallShield of the 21st Century – Theo Schlossnagle
The InstallShield of the 21st Century – Theo SchlossnagleThe InstallShield of the 21st Century – Theo Schlossnagle
The InstallShield of the 21st Century – Theo SchlossnagleChef Software, Inc.
 
The Not Java That's Not Scala
The Not Java That's Not ScalaThe Not Java That's Not Scala
The Not Java That's Not ScalaJustin Lee
 
Genestack BioIT-World-2013
Genestack BioIT-World-2013Genestack BioIT-World-2013
Genestack BioIT-World-2013genestack
 
Intro to Laravel PHP Framework
Intro to Laravel PHP FrameworkIntro to Laravel PHP Framework
Intro to Laravel PHP FrameworkBill Condo
 
Practical pairing of generative programming with functional programming.
Practical pairing of generative programming with functional programming.Practical pairing of generative programming with functional programming.
Practical pairing of generative programming with functional programming.Eugene Lazutkin
 
Keeping it small - Getting to know the Slim PHP micro framework
Keeping it small - Getting to know the Slim PHP micro frameworkKeeping it small - Getting to know the Slim PHP micro framework
Keeping it small - Getting to know the Slim PHP micro frameworkJeremy Kendall
 
Ab(Using) the MetaCPAN API for Fun and Profit v2013
Ab(Using) the MetaCPAN API for Fun and Profit v2013Ab(Using) the MetaCPAN API for Fun and Profit v2013
Ab(Using) the MetaCPAN API for Fun and Profit v2013Olaf Alders
 
Redesigning UBC Library
Redesigning UBC LibraryRedesigning UBC Library
Redesigning UBC Libraryjtcchan
 

Ähnlich wie Intravert Experimental Transport for Apache Cassandra (20)

Writing testable code
Writing testable codeWriting testable code
Writing testable code
 
CPANci: Continuous Integration for CPAN
CPANci: Continuous Integration for CPANCPANci: Continuous Integration for CPAN
CPANci: Continuous Integration for CPAN
 
The State of Puppet
The State of PuppetThe State of Puppet
The State of Puppet
 
PHP Conference Argentina 2013 - Independizate de tu departamento IT - Habilid...
PHP Conference Argentina 2013 - Independizate de tu departamento IT - Habilid...PHP Conference Argentina 2013 - Independizate de tu departamento IT - Habilid...
PHP Conference Argentina 2013 - Independizate de tu departamento IT - Habilid...
 
Microservices and functional programming
Microservices and functional programmingMicroservices and functional programming
Microservices and functional programming
 
JSUG - ActionScript 3 vs Java by Christoph Pickl
JSUG - ActionScript 3 vs Java by Christoph PicklJSUG - ActionScript 3 vs Java by Christoph Pickl
JSUG - ActionScript 3 vs Java by Christoph Pickl
 
JSUG - AS3 vs Java by Christoph Pickl
JSUG - AS3 vs Java by Christoph PicklJSUG - AS3 vs Java by Christoph Pickl
JSUG - AS3 vs Java by Christoph Pickl
 
Hadoop meets Cloud with Multi-Tenancy
Hadoop meets Cloud with Multi-TenancyHadoop meets Cloud with Multi-Tenancy
Hadoop meets Cloud with Multi-Tenancy
 
Writing the docs
Writing the docsWriting the docs
Writing the docs
 
Keeping your users happy with testable apps - Greg Shackles
Keeping your users happy with testable apps - Greg ShacklesKeeping your users happy with testable apps - Greg Shackles
Keeping your users happy with testable apps - Greg Shackles
 
Puppet and AWS: Getting the best of both worlds
Puppet and AWS: Getting the best of both worldsPuppet and AWS: Getting the best of both worlds
Puppet and AWS: Getting the best of both worlds
 
The InstallShield of the 21st Century – Theo Schlossnagle
The InstallShield of the 21st Century – Theo SchlossnagleThe InstallShield of the 21st Century – Theo Schlossnagle
The InstallShield of the 21st Century – Theo Schlossnagle
 
The Not Java That's Not Scala
The Not Java That's Not ScalaThe Not Java That's Not Scala
The Not Java That's Not Scala
 
Genestack BioIT-World-2013
Genestack BioIT-World-2013Genestack BioIT-World-2013
Genestack BioIT-World-2013
 
Intro to Laravel PHP Framework
Intro to Laravel PHP FrameworkIntro to Laravel PHP Framework
Intro to Laravel PHP Framework
 
Practical pairing of generative programming with functional programming.
Practical pairing of generative programming with functional programming.Practical pairing of generative programming with functional programming.
Practical pairing of generative programming with functional programming.
 
Keeping it small - Getting to know the Slim PHP micro framework
Keeping it small - Getting to know the Slim PHP micro frameworkKeeping it small - Getting to know the Slim PHP micro framework
Keeping it small - Getting to know the Slim PHP micro framework
 
Ab(Using) the MetaCPAN API for Fun and Profit v2013
Ab(Using) the MetaCPAN API for Fun and Profit v2013Ab(Using) the MetaCPAN API for Fun and Profit v2013
Ab(Using) the MetaCPAN API for Fun and Profit v2013
 
Redesigning UBC Library
Redesigning UBC LibraryRedesigning UBC Library
Redesigning UBC Library
 
Future layouts
Future layoutsFuture layouts
Future layouts
 

Mehr von zznate

Seattle C* Meetup: Hardening cassandra for compliance or paranoia
Seattle C* Meetup: Hardening cassandra for compliance or paranoiaSeattle C* Meetup: Hardening cassandra for compliance or paranoia
Seattle C* Meetup: Hardening cassandra for compliance or paranoiazznate
 
Hardening cassandra for compliance or paranoia
Hardening cassandra for compliance or paranoiaHardening cassandra for compliance or paranoia
Hardening cassandra for compliance or paranoiazznate
 
Successful Software Development with Apache Cassandra
Successful Software Development with Apache CassandraSuccessful Software Development with Apache Cassandra
Successful Software Development with Apache Cassandrazznate
 
Stampede con 2014 cassandra in the real world
Stampede con 2014   cassandra in the real worldStampede con 2014   cassandra in the real world
Stampede con 2014 cassandra in the real worldzznate
 
An Introduction to the Vert.x framework
An Introduction to the Vert.x frameworkAn Introduction to the Vert.x framework
An Introduction to the Vert.x frameworkzznate
 
Oscon 2012 tdd_cassandra
Oscon 2012 tdd_cassandraOscon 2012 tdd_cassandra
Oscon 2012 tdd_cassandrazznate
 
Strata west 2012_java_cassandra
Strata west 2012_java_cassandraStrata west 2012_java_cassandra
Strata west 2012_java_cassandrazznate
 
Nyc summit intro_to_cassandra
Nyc summit intro_to_cassandraNyc summit intro_to_cassandra
Nyc summit intro_to_cassandrazznate
 
Meetup cassandra sfo_jdbc
Meetup cassandra sfo_jdbcMeetup cassandra sfo_jdbc
Meetup cassandra sfo_jdbczznate
 
Meetup cassandra for_java_cql
Meetup cassandra for_java_cqlMeetup cassandra for_java_cql
Meetup cassandra for_java_cqlzznate
 
Introduciton to Apache Cassandra for Java Developers (JavaOne)
Introduciton to Apache Cassandra for Java Developers (JavaOne)Introduciton to Apache Cassandra for Java Developers (JavaOne)
Introduciton to Apache Cassandra for Java Developers (JavaOne)zznate
 
Introduction to apache_cassandra_for_developers-lhg
Introduction to apache_cassandra_for_developers-lhgIntroduction to apache_cassandra_for_developers-lhg
Introduction to apache_cassandra_for_developers-lhgzznate
 
Introduction to apache_cassandra_for_develope
Introduction to apache_cassandra_for_developeIntroduction to apache_cassandra_for_develope
Introduction to apache_cassandra_for_developezznate
 
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...zznate
 

Mehr von zznate (14)

Seattle C* Meetup: Hardening cassandra for compliance or paranoia
Seattle C* Meetup: Hardening cassandra for compliance or paranoiaSeattle C* Meetup: Hardening cassandra for compliance or paranoia
Seattle C* Meetup: Hardening cassandra for compliance or paranoia
 
Hardening cassandra for compliance or paranoia
Hardening cassandra for compliance or paranoiaHardening cassandra for compliance or paranoia
Hardening cassandra for compliance or paranoia
 
Successful Software Development with Apache Cassandra
Successful Software Development with Apache CassandraSuccessful Software Development with Apache Cassandra
Successful Software Development with Apache Cassandra
 
Stampede con 2014 cassandra in the real world
Stampede con 2014   cassandra in the real worldStampede con 2014   cassandra in the real world
Stampede con 2014 cassandra in the real world
 
An Introduction to the Vert.x framework
An Introduction to the Vert.x frameworkAn Introduction to the Vert.x framework
An Introduction to the Vert.x framework
 
Oscon 2012 tdd_cassandra
Oscon 2012 tdd_cassandraOscon 2012 tdd_cassandra
Oscon 2012 tdd_cassandra
 
Strata west 2012_java_cassandra
Strata west 2012_java_cassandraStrata west 2012_java_cassandra
Strata west 2012_java_cassandra
 
Nyc summit intro_to_cassandra
Nyc summit intro_to_cassandraNyc summit intro_to_cassandra
Nyc summit intro_to_cassandra
 
Meetup cassandra sfo_jdbc
Meetup cassandra sfo_jdbcMeetup cassandra sfo_jdbc
Meetup cassandra sfo_jdbc
 
Meetup cassandra for_java_cql
Meetup cassandra for_java_cqlMeetup cassandra for_java_cql
Meetup cassandra for_java_cql
 
Introduciton to Apache Cassandra for Java Developers (JavaOne)
Introduciton to Apache Cassandra for Java Developers (JavaOne)Introduciton to Apache Cassandra for Java Developers (JavaOne)
Introduciton to Apache Cassandra for Java Developers (JavaOne)
 
Introduction to apache_cassandra_for_developers-lhg
Introduction to apache_cassandra_for_developers-lhgIntroduction to apache_cassandra_for_developers-lhg
Introduction to apache_cassandra_for_developers-lhg
 
Introduction to apache_cassandra_for_develope
Introduction to apache_cassandra_for_developeIntroduction to apache_cassandra_for_develope
Introduction to apache_cassandra_for_develope
 
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
 

Intravert Experimental Transport for Apache Cassandra

  • 1. Intravert Experimental Transport for Apache Cassandra Friday, April 5, 13
  • 2. Hi Folks! I’m Nate @zznate Friday, April 5, 13
  • 3. API Management API Analytics API Tools Friday, April 5, 13
  • 4. Vert.x + Cassandra = ? Friday, April 5, 13
  • 5. But first: Architectural stuff Friday, April 5, 13
  • 6. Collapsed for brevity. see slides 6-31 from this presentation: http://www.slideshare.net/zznate/ apachecon-cassandra-transport Friday, April 5, 13
  • 7. So there are options, why create something new? Friday, April 5, 13
  • 8. Common client patterns emerged Friday, April 5, 13
  • 9. union operations aggregations filtering joins (round-trip required!) redundant data on insertion Friday, April 5, 13
  • 10. These client patterns are wasteful. Friday, April 5, 13
  • 11. Lots of CPU and memory burned (server and client) Lots of data on the wire won’t be used (ser/deSer cost) Friday, April 5, 13
  • 12. One more thing about CQL... Friday, April 5, 13
  • 13. “If [they] think that CQL is the answer to usability then I just won. We at least know where our problems are.” - 10gen exec. Friday, April 5, 13
  • 15. Thrift: I can has distributed computing degree Friday, April 5, 13
  • 16. CQL: MOAR TOOLz!!! Friday, April 5, 13
  • 17. IMO: In 2013, binary protocols with idiomatic clients misses the market. Friday, April 5, 13
  • 18. I love writing apps in Java (sometimes python), but... Friday, April 5, 13
  • 19. The most popular application framework these days: Friday, April 5, 13
  • 20. The most popular application framework these days: HTML5 Friday, April 5, 13
  • 21. and thus... Intravert Friday, April 5, 13
  • 22. Any new transport had to be HTTP based Friday, April 5, 13
  • 23. Easy to use Easy to secure Easy to compress You can test with a browser! Friday, April 5, 13
  • 24. The payload had to be JSON based Friday, April 5, 13
  • 26. - Event-driven IO (Netty) - Message bus - Runtime load/unload of modules - Polyglot support via invokeDynamic in JDK7 http://vertx.io Friday, April 5, 13
  • 27. Initial use case: Server side indexing Friday, April 5, 13
  • 28. Easy to construct most common operations Friday, April 5, 13
  • 29. Simple REST semantics POST /myks/mycf/key1 {“uname”:”zznate”} GET /myks/mycf/key1/uname {”zznate”} Friday, April 5, 13
  • 30. Command-driven JSON payload Friday, April 5, 13
  • 31. Extensible and transparent Ordered operations Not tied to any predefined notions Friday, April 5, 13
  • 32. Benefits of doing this with Vert.x Friday, April 5, 13
  • 33. Message bus flexibility Configure which modules to load (read-only nodes!) Hot deploy of fixes and new features Friday, April 5, 13
  • 34. Cassandra is best-of- breed for a lot of things. So we used it as a framework. Friday, April 5, 13
  • 35. Example Intravert Operations Friday, April 5, 13
  • 36. Slice: {"id":6, "type":"slice", "op":{      "end":"9",      "rowkey":"5",      "size":4,      "start":"1"} } Friday, April 5, 13
  • 37. Set: {"id":2, "type":"SET", "op":{            "rowkey":"key1",      "uname":"zznate",      "email":"zznate.m@gmail.com",      "age":37} } Friday, April 5, 13
  • 38. Easy to use Composites Friday, April 5, 13
  • 39. {"id":1, "type":"SET", "op":{            "rowkey":"key1",      "COMP_NAME":[          "37",  "zznate"      ],      "value":"key1",      } } Friday, April 5, 13
  • 40. One or more commands per payload Friday, April 5, 13
  • 42. Flexible batching - batch reads and writes - different keyspaces in same op - change ks/cf mid-op - streaming (soon) Friday, April 5, 13
  • 44. { "type": "CREATEFILTER", "op": { "name": "stouts", "spec": "javascript", "value": "function(row) { if (row['value'] == 'Breakfast Stout') return row; else return null; }" } }, { "type": "FILTERMODE", "op": { "name": "stouts", "on": true } }, { "type": "SLICE", "op": { "rowkey": "beers", "start": "Allagash", "end": "Sierra Nevada", "size": 9 } } Friday, April 5, 13
  • 45. { "exception":null, "exceptionId":null, "opsRes": { "0":"OK", "1":"OK", "2":"OK", "3":"OK", "4":"OK", "5":"OK", "6":[{ "name":"Founders", "value":"Breakfast Stout" }] }} Friday, April 5, 13
  • 46. GetRef: Results of one operation as input from another Friday, April 5, 13
  • 47. { "type": "GET", "op": { "rowkey": "getref_test", "name": "col1" } }, { "type": "SET", "op": { "rowkey": "getref_test", "name": "col3", "value": { "type": "GETREF", "op": { "resultref": 4, "wanted": "value" } } } }, { "type": "GET", "op": { "rowkey": "getref_test", "name": "col3" } } Friday, April 5, 13
  • 48. “Triggers” - Perform complex multi-step operations server side - Examine and change request objects - Interface directly with IntraVert - Interface directly with Cassandra (StorageProxy, CFMetaData, etc) Friday, April 5, 13
  • 49. Multi-Processors: Shape results with a union of operations Friday, April 5, 13
  • 50. Prepared Request Statement: Requests saved server side - send only the bind variables Friday, April 5, 13
  • 53. Security (Shiro integration coming soon) Distribution semantics Cassandra-REST HBASE-like scanners Friday, April 5, 13
  • 54. github.com/zznate/ intravert-ug Friday, April 5, 13