SlideShare ist ein Scribd-Unternehmen logo
1 von 15
Timothy Perrett
Bath Camp 2010
What is Riak?
• Documented orientated database
• Written in Erlang
• Based on Dynamo[1] and CAP Theorem[2]
• Highly fault tolerant
• HTTP and ProtoBuff interface
• Write MapReduce in Erlang or JavaScript
1. http://goo.gl/r8Np
2. http://www.julianbrowne.com/article/viewer/brewers-cap-theorem
Same, Same but different
• Riak solves similar problems to MongoDB
• Semi-structured data modeled as "documents”
• Storage of non-document data in the database
• High write-availability
• Riak is intrinsically multi-node scalable
• Mongo in comparison is single system (+ sharding)
• Riak achieves availability via quorum writes
• Mongo uses performant in-place writes
• Riak uses “masterless” replication
N/R/W – Dynamo
N = Number of replicas to store
R = Number of replicas needed to read
W = Number of replicas needed to read
• These principals first appeared in an Amazon
research paper known as Dynamo
• 160bit integer key
space. Each node that
joins is assigned part
of that space for
consistent hashing
• Hashing means any
node can service any
request making the
cluster masterless and
eventually consistant
Number of replicas
• Number of replies
before Riak gives
the client a
successful reply.
• Tries to access all
nodes, but as soon
as the N/R is
satisfied a response
is given
Reads
• Same as reads; W
implies the number
of successful nodes
that must reply
before the write
is considered
consistent by
the client
Writes
Extreme example
• Given N=10, R=W=2 we
could have 8 nodes
down and the cluster
would still be fully
available to all clients
What does this all mean?
• N/R/W specified at request time, so each
client can specify its own tolerance for
outages dynamically
• Despite any outages within the cluster, the whole
cluster can still appear available based on N/R/W
• Given N=3 and R=W=2, we can have 3-2=1 node
down/unreachable/laggy in the cluster
• Stupidly high availability complete with eventual
consistency controlled by dynamic clients
Brewer’s CAP Theorem
• Consistency
• Availability
• Partition Tolerance
• You cant have all things, all the time…
• …but you can have some of each, all the time!
• Riak is about choosing your own levels of
each according to your use case
Consistency
• Start with document
version zero
• Things get redistributed
and n0 and n2 are
sitting in NYC and n1
and n3 are in London
• What if stuff changes??
Consistency
• Uh oh: inconsistency
• Both parts of the cluster
are still fully available
• NYC serves v1 whilst
London serves v0
• The network resumes
and Riak determines
the latest version by
using vector clocks
Consistency
• What if both sides of
the Atlantic changed?
• Riak is unable to
determine which is the
right document, both
are returned to the
client with an indication
of the inconsistency
• Distributed, fault-tolerant full-text searching
• Lucene syntax for queries
• No need for index sharding
• Linier scaling
• Double the number of nodes to get double the
search capacity (awesome!)
• Search via:
• Fields, wildcards, fuzzy text or token proximity
Riak Search
Questions?
basho.com/riak.html
github.com/basho/riak
twitter.com/timperrett
github.com/timperrett
blog.getintheloop.eu

Weitere ähnliche Inhalte

Was ist angesagt?

Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...
Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...
Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...Flink Forward
 
Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...
Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...
Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...Flink Forward
 
Aljoscha Krettek - Portable stateful big data processing in Apache Beam
Aljoscha Krettek - Portable stateful big data processing in Apache BeamAljoscha Krettek - Portable stateful big data processing in Apache Beam
Aljoscha Krettek - Portable stateful big data processing in Apache BeamVerverica
 
Building your own Distributed System The easy way - Cassandra Summit EU 2014
Building your own Distributed System The easy way - Cassandra Summit EU 2014Building your own Distributed System The easy way - Cassandra Summit EU 2014
Building your own Distributed System The easy way - Cassandra Summit EU 2014Kévin LOVATO
 
How to manage large amounts of data with akka streams
How to manage large amounts of data with akka streamsHow to manage large amounts of data with akka streams
How to manage large amounts of data with akka streamsIgor Mielientiev
 
ChronoLogic Tools Demo: 6/12/18
ChronoLogic Tools Demo: 6/12/18ChronoLogic Tools Demo: 6/12/18
ChronoLogic Tools Demo: 6/12/18ChronoLogic
 
Kafka At Scale in the Cloud
Kafka At Scale in the CloudKafka At Scale in the Cloud
Kafka At Scale in the Cloudconfluent
 
Thoughts on consistency models
Thoughts on consistency modelsThoughts on consistency models
Thoughts on consistency modelsrogerbodamer
 
Virtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy Farkas
Virtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy FarkasVirtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy Farkas
Virtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy FarkasFlink Forward
 
Parallel processing for splitter in mule esb
Parallel processing for splitter in mule esbParallel processing for splitter in mule esb
Parallel processing for splitter in mule esbSunil Kumar
 
Alexander Kolb – Flink. Yet another Streaming Framework?
Alexander Kolb – Flink. Yet another Streaming Framework?Alexander Kolb – Flink. Yet another Streaming Framework?
Alexander Kolb – Flink. Yet another Streaming Framework?Flink Forward
 
Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...
Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...
Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...Flink Forward
 
Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...
Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...
Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...Flink Forward
 
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...Flink Forward
 
Introduction to Structured streaming
Introduction to Structured streamingIntroduction to Structured streaming
Introduction to Structured streamingdatamantra
 
Matthias J. Sax – A Tale of Squirrels and Storms
Matthias J. Sax – A Tale of Squirrels and StormsMatthias J. Sax – A Tale of Squirrels and Storms
Matthias J. Sax – A Tale of Squirrels and StormsFlink Forward
 
Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...
Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...
Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...Flink Forward
 
Apache Software Foundation: How To Contribute, with Apache Flink as Example (...
Apache Software Foundation: How To Contribute, with Apache Flink as Example (...Apache Software Foundation: How To Contribute, with Apache Flink as Example (...
Apache Software Foundation: How To Contribute, with Apache Flink as Example (...Apache Flink Taiwan User Group
 

Was ist angesagt? (20)

Project Reactor By Example
Project Reactor By ExampleProject Reactor By Example
Project Reactor By Example
 
Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...
Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...
Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...
 
Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...
Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...
Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...
 
Aljoscha Krettek - Portable stateful big data processing in Apache Beam
Aljoscha Krettek - Portable stateful big data processing in Apache BeamAljoscha Krettek - Portable stateful big data processing in Apache Beam
Aljoscha Krettek - Portable stateful big data processing in Apache Beam
 
Building your own Distributed System The easy way - Cassandra Summit EU 2014
Building your own Distributed System The easy way - Cassandra Summit EU 2014Building your own Distributed System The easy way - Cassandra Summit EU 2014
Building your own Distributed System The easy way - Cassandra Summit EU 2014
 
How to manage large amounts of data with akka streams
How to manage large amounts of data with akka streamsHow to manage large amounts of data with akka streams
How to manage large amounts of data with akka streams
 
ChronoLogic Tools Demo: 6/12/18
ChronoLogic Tools Demo: 6/12/18ChronoLogic Tools Demo: 6/12/18
ChronoLogic Tools Demo: 6/12/18
 
Kafka At Scale in the Cloud
Kafka At Scale in the CloudKafka At Scale in the Cloud
Kafka At Scale in the Cloud
 
Thoughts on consistency models
Thoughts on consistency modelsThoughts on consistency models
Thoughts on consistency models
 
Virtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy Farkas
Virtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy FarkasVirtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy Farkas
Virtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy Farkas
 
Parallel processing for splitter in mule esb
Parallel processing for splitter in mule esbParallel processing for splitter in mule esb
Parallel processing for splitter in mule esb
 
Alexander Kolb – Flink. Yet another Streaming Framework?
Alexander Kolb – Flink. Yet another Streaming Framework?Alexander Kolb – Flink. Yet another Streaming Framework?
Alexander Kolb – Flink. Yet another Streaming Framework?
 
Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...
Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...
Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...
 
Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...
Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...
Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...
 
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
 
Introduction to Structured streaming
Introduction to Structured streamingIntroduction to Structured streaming
Introduction to Structured streaming
 
Matthias J. Sax – A Tale of Squirrels and Storms
Matthias J. Sax – A Tale of Squirrels and StormsMatthias J. Sax – A Tale of Squirrels and Storms
Matthias J. Sax – A Tale of Squirrels and Storms
 
Notes on Netty baics
Notes on Netty baicsNotes on Netty baics
Notes on Netty baics
 
Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...
Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...
Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...
 
Apache Software Foundation: How To Contribute, with Apache Flink as Example (...
Apache Software Foundation: How To Contribute, with Apache Flink as Example (...Apache Software Foundation: How To Contribute, with Apache Flink as Example (...
Apache Software Foundation: How To Contribute, with Apache Flink as Example (...
 

Andere mochten auch

Actividad1 curriculum
Actividad1 curriculumActividad1 curriculum
Actividad1 curriculumLuzfrida
 
Lectura ser digital
Lectura ser digitalLectura ser digital
Lectura ser digitalLUISFDAVILA
 
Initial presentation Tesla management project (Swinburne University)
Initial presentation Tesla management project (Swinburne University)Initial presentation Tesla management project (Swinburne University)
Initial presentation Tesla management project (Swinburne University)Anthony Campana
 
Gas licuado de petróleo GLP
Gas licuado de petróleo GLPGas licuado de petróleo GLP
Gas licuado de petróleo GLPHéctor Chire
 
Condylar fractures /certified fixed orthodontic courses by Indian dental acad...
Condylar fractures /certified fixed orthodontic courses by Indian dental acad...Condylar fractures /certified fixed orthodontic courses by Indian dental acad...
Condylar fractures /certified fixed orthodontic courses by Indian dental acad...Indian dental academy
 
Mandibular fracture 2 / fixed orthodontic courses
Mandibular fracture 2 / fixed orthodontic coursesMandibular fracture 2 / fixed orthodontic courses
Mandibular fracture 2 / fixed orthodontic coursesIndian dental academy
 
Mod morphology of deciduous dentition
Mod morphology of deciduous dentitionMod morphology of deciduous dentition
Mod morphology of deciduous dentitionJamil Kifayatullah
 
Phần 1 công ty kết cấu sx thép thái nguyên
Phần 1   công ty kết cấu sx thép thái nguyênPhần 1   công ty kết cấu sx thép thái nguyên
Phần 1 công ty kết cấu sx thép thái nguyêntranthihoaivan
 
A2 Media Studies Preproduction Development
A2 Media Studies Preproduction DevelopmentA2 Media Studies Preproduction Development
A2 Media Studies Preproduction DevelopmentGeorginaMediaStudies
 
Feuillet memento Degremont - n°1 Ultragreen
Feuillet memento Degremont - n°1 UltragreenFeuillet memento Degremont - n°1 Ultragreen
Feuillet memento Degremont - n°1 UltragreenDegrémont
 

Andere mochten auch (19)

Electronica jj
Electronica jjElectronica jj
Electronica jj
 
Actividad1 curriculum
Actividad1 curriculumActividad1 curriculum
Actividad1 curriculum
 
ville du mont-dore
ville du mont-doreville du mont-dore
ville du mont-dore
 
Que es slideshare
Que es slideshareQue es slideshare
Que es slideshare
 
CSR Events at Technika 15
CSR Events at Technika 15CSR Events at Technika 15
CSR Events at Technika 15
 
Lectura ser digital
Lectura ser digitalLectura ser digital
Lectura ser digital
 
Exposicion equipo 1
Exposicion equipo 1Exposicion equipo 1
Exposicion equipo 1
 
P
PP
P
 
Initial presentation Tesla management project (Swinburne University)
Initial presentation Tesla management project (Swinburne University)Initial presentation Tesla management project (Swinburne University)
Initial presentation Tesla management project (Swinburne University)
 
Wiring of-mandible
Wiring of-mandibleWiring of-mandible
Wiring of-mandible
 
Gas licuado de petróleo GLP
Gas licuado de petróleo GLPGas licuado de petróleo GLP
Gas licuado de petróleo GLP
 
Condylar fractures /certified fixed orthodontic courses by Indian dental acad...
Condylar fractures /certified fixed orthodontic courses by Indian dental acad...Condylar fractures /certified fixed orthodontic courses by Indian dental acad...
Condylar fractures /certified fixed orthodontic courses by Indian dental acad...
 
Favolare
FavolareFavolare
Favolare
 
Mandibular fracture 2 / fixed orthodontic courses
Mandibular fracture 2 / fixed orthodontic coursesMandibular fracture 2 / fixed orthodontic courses
Mandibular fracture 2 / fixed orthodontic courses
 
Mod morphology of deciduous dentition
Mod morphology of deciduous dentitionMod morphology of deciduous dentition
Mod morphology of deciduous dentition
 
Phần 1 công ty kết cấu sx thép thái nguyên
Phần 1   công ty kết cấu sx thép thái nguyênPhần 1   công ty kết cấu sx thép thái nguyên
Phần 1 công ty kết cấu sx thép thái nguyên
 
A2 Media Studies Preproduction Development
A2 Media Studies Preproduction DevelopmentA2 Media Studies Preproduction Development
A2 Media Studies Preproduction Development
 
Human Dentition
Human DentitionHuman Dentition
Human Dentition
 
Feuillet memento Degremont - n°1 Ultragreen
Feuillet memento Degremont - n°1 UltragreenFeuillet memento Degremont - n°1 Ultragreen
Feuillet memento Degremont - n°1 Ultragreen
 

Ähnlich wie Bathcamp 2010-riak

Scalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeScalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeAmir Ghaffari
 
HPC Controls Future
HPC Controls FutureHPC Controls Future
HPC Controls Futurercastain
 
Getting started with Riak in the Cloud
Getting started with Riak in the CloudGetting started with Riak in the Cloud
Getting started with Riak in the CloudInes Sombra
 
Running a distributed system across kubernetes clusters - Kubecon North Ameri...
Running a distributed system across kubernetes clusters - Kubecon North Ameri...Running a distributed system across kubernetes clusters - Kubecon North Ameri...
Running a distributed system across kubernetes clusters - Kubecon North Ameri...Alex Robinson
 
Multi-Datacenter Kafka - Strata San Jose 2017
Multi-Datacenter Kafka - Strata San Jose 2017Multi-Datacenter Kafka - Strata San Jose 2017
Multi-Datacenter Kafka - Strata San Jose 2017Gwen (Chen) Shapira
 
Highly available, scalable and secure data with Cassandra and DataStax Enterp...
Highly available, scalable and secure data with Cassandra and DataStax Enterp...Highly available, scalable and secure data with Cassandra and DataStax Enterp...
Highly available, scalable and secure data with Cassandra and DataStax Enterp...Johnny Miller
 
Lessons learned from scaling YARN to 40K machines in a multi tenancy environment
Lessons learned from scaling YARN to 40K machines in a multi tenancy environmentLessons learned from scaling YARN to 40K machines in a multi tenancy environment
Lessons learned from scaling YARN to 40K machines in a multi tenancy environmentDataWorks Summit
 
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...Matteo Ferroni
 
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInJay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInLinkedIn
 
Tuning kafka pipelines
Tuning kafka pipelinesTuning kafka pipelines
Tuning kafka pipelinesSumant Tambe
 
End-to-End Reactive Data Access Using R2DBC with RSocket and Proteus
End-to-End Reactive Data Access Using R2DBC with RSocket and ProteusEnd-to-End Reactive Data Access Using R2DBC with RSocket and Proteus
End-to-End Reactive Data Access Using R2DBC with RSocket and ProteusVMware Tanzu
 
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...tdc-globalcode
 
High performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User GroupHigh performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User GroupHungWei Chiu
 
Building Distributed Systems With Riak and Riak Core
Building Distributed Systems With Riak and Riak CoreBuilding Distributed Systems With Riak and Riak Core
Building Distributed Systems With Riak and Riak CoreAndy Gross
 
Incremental Export of Relational Database Contents into RDF Graphs
Incremental Export of Relational Database Contents into RDF GraphsIncremental Export of Relational Database Contents into RDF Graphs
Incremental Export of Relational Database Contents into RDF GraphsNikolaos Konstantinou
 
Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...
Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...
Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...confluent
 

Ähnlich wie Bathcamp 2010-riak (20)

Scalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeScalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and Practice
 
HPC Controls Future
HPC Controls FutureHPC Controls Future
HPC Controls Future
 
Getting started with Riak in the Cloud
Getting started with Riak in the CloudGetting started with Riak in the Cloud
Getting started with Riak in the Cloud
 
Running a distributed system across kubernetes clusters - Kubecon North Ameri...
Running a distributed system across kubernetes clusters - Kubecon North Ameri...Running a distributed system across kubernetes clusters - Kubecon North Ameri...
Running a distributed system across kubernetes clusters - Kubecon North Ameri...
 
Multi-Datacenter Kafka - Strata San Jose 2017
Multi-Datacenter Kafka - Strata San Jose 2017Multi-Datacenter Kafka - Strata San Jose 2017
Multi-Datacenter Kafka - Strata San Jose 2017
 
MySQL on Ceph
MySQL on CephMySQL on Ceph
MySQL on Ceph
 
My SQL on Ceph
My SQL on CephMy SQL on Ceph
My SQL on Ceph
 
Aurora_session.pdf
Aurora_session.pdfAurora_session.pdf
Aurora_session.pdf
 
Highly available, scalable and secure data with Cassandra and DataStax Enterp...
Highly available, scalable and secure data with Cassandra and DataStax Enterp...Highly available, scalable and secure data with Cassandra and DataStax Enterp...
Highly available, scalable and secure data with Cassandra and DataStax Enterp...
 
Lessons learned from scaling YARN to 40K machines in a multi tenancy environment
Lessons learned from scaling YARN to 40K machines in a multi tenancy environmentLessons learned from scaling YARN to 40K machines in a multi tenancy environment
Lessons learned from scaling YARN to 40K machines in a multi tenancy environment
 
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...
 
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInJay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
 
Tuning kafka pipelines
Tuning kafka pipelinesTuning kafka pipelines
Tuning kafka pipelines
 
End-to-End Reactive Data Access Using R2DBC with RSocket and Proteus
End-to-End Reactive Data Access Using R2DBC with RSocket and ProteusEnd-to-End Reactive Data Access Using R2DBC with RSocket and Proteus
End-to-End Reactive Data Access Using R2DBC with RSocket and Proteus
 
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
 
High performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User GroupHigh performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User Group
 
Building Distributed Systems With Riak and Riak Core
Building Distributed Systems With Riak and Riak CoreBuilding Distributed Systems With Riak and Riak Core
Building Distributed Systems With Riak and Riak Core
 
Incremental Export of Relational Database Contents into RDF Graphs
Incremental Export of Relational Database Contents into RDF GraphsIncremental Export of Relational Database Contents into RDF Graphs
Incremental Export of Relational Database Contents into RDF Graphs
 
Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...
Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...
Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...
 
Scalable Web Apps
Scalable Web AppsScalable Web Apps
Scalable Web Apps
 

Mehr von Timothy Perrett

Nelson: Rigorous Deployment for a Functional World
Nelson: Rigorous Deployment for a Functional WorldNelson: Rigorous Deployment for a Functional World
Nelson: Rigorous Deployment for a Functional WorldTimothy Perrett
 
Online Experimentation with Immutable Infrastructure
Online Experimentation with Immutable InfrastructureOnline Experimentation with Immutable Infrastructure
Online Experimentation with Immutable InfrastructureTimothy Perrett
 
Enterprise Algebras, Scala World 2016
Enterprise Algebras, Scala World 2016Enterprise Algebras, Scala World 2016
Enterprise Algebras, Scala World 2016Timothy Perrett
 
Large-scale Infrastructure Automation at Verizon
Large-scale Infrastructure Automation at VerizonLarge-scale Infrastructure Automation at Verizon
Large-scale Infrastructure Automation at VerizonTimothy Perrett
 
Reasonable RPC with Remotely
Reasonable RPC with RemotelyReasonable RPC with Remotely
Reasonable RPC with RemotelyTimothy Perrett
 
Building Enigma with State Monad & Lens
Building Enigma with State Monad & LensBuilding Enigma with State Monad & Lens
Building Enigma with State Monad & LensTimothy Perrett
 
Functional Programming at Verizon
Functional Programming at VerizonFunctional Programming at Verizon
Functional Programming at VerizonTimothy Perrett
 
Scalalable Language for a Scalable Web
Scalalable Language for a Scalable WebScalalable Language for a Scalable Web
Scalalable Language for a Scalable WebTimothy Perrett
 
Javazone 2011: Goal Directed Web Applications
Javazone 2011: Goal Directed Web ApplicationsJavazone 2011: Goal Directed Web Applications
Javazone 2011: Goal Directed Web ApplicationsTimothy Perrett
 
Concurrency and Parallelism with Scala
Concurrency and Parallelism with ScalaConcurrency and Parallelism with Scala
Concurrency and Parallelism with ScalaTimothy Perrett
 
Scaladays 2011: Task Driven Scala Web Applications
Scaladays 2011: Task Driven Scala Web ApplicationsScaladays 2011: Task Driven Scala Web Applications
Scaladays 2011: Task Driven Scala Web ApplicationsTimothy Perrett
 
Javazone 2010-lift-framework-public
Javazone 2010-lift-framework-publicJavazone 2010-lift-framework-public
Javazone 2010-lift-framework-publicTimothy Perrett
 
Devoxx 2009: The Lift Framework
Devoxx 2009: The Lift FrameworkDevoxx 2009: The Lift Framework
Devoxx 2009: The Lift FrameworkTimothy Perrett
 

Mehr von Timothy Perrett (15)

Nelson: Rigorous Deployment for a Functional World
Nelson: Rigorous Deployment for a Functional WorldNelson: Rigorous Deployment for a Functional World
Nelson: Rigorous Deployment for a Functional World
 
Online Experimentation with Immutable Infrastructure
Online Experimentation with Immutable InfrastructureOnline Experimentation with Immutable Infrastructure
Online Experimentation with Immutable Infrastructure
 
Enterprise Algebras, Scala World 2016
Enterprise Algebras, Scala World 2016Enterprise Algebras, Scala World 2016
Enterprise Algebras, Scala World 2016
 
Large-scale Infrastructure Automation at Verizon
Large-scale Infrastructure Automation at VerizonLarge-scale Infrastructure Automation at Verizon
Large-scale Infrastructure Automation at Verizon
 
Reasonable RPC with Remotely
Reasonable RPC with RemotelyReasonable RPC with Remotely
Reasonable RPC with Remotely
 
Building Enigma with State Monad & Lens
Building Enigma with State Monad & LensBuilding Enigma with State Monad & Lens
Building Enigma with State Monad & Lens
 
Functional Programming at Verizon
Functional Programming at VerizonFunctional Programming at Verizon
Functional Programming at Verizon
 
Scalalable Language for a Scalable Web
Scalalable Language for a Scalable WebScalalable Language for a Scalable Web
Scalalable Language for a Scalable Web
 
BRUG - Hello, Scala
BRUG - Hello, ScalaBRUG - Hello, Scala
BRUG - Hello, Scala
 
Scala Helix
Scala HelixScala Helix
Scala Helix
 
Javazone 2011: Goal Directed Web Applications
Javazone 2011: Goal Directed Web ApplicationsJavazone 2011: Goal Directed Web Applications
Javazone 2011: Goal Directed Web Applications
 
Concurrency and Parallelism with Scala
Concurrency and Parallelism with ScalaConcurrency and Parallelism with Scala
Concurrency and Parallelism with Scala
 
Scaladays 2011: Task Driven Scala Web Applications
Scaladays 2011: Task Driven Scala Web ApplicationsScaladays 2011: Task Driven Scala Web Applications
Scaladays 2011: Task Driven Scala Web Applications
 
Javazone 2010-lift-framework-public
Javazone 2010-lift-framework-publicJavazone 2010-lift-framework-public
Javazone 2010-lift-framework-public
 
Devoxx 2009: The Lift Framework
Devoxx 2009: The Lift FrameworkDevoxx 2009: The Lift Framework
Devoxx 2009: The Lift Framework
 

Kürzlich hochgeladen

"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 

Kürzlich hochgeladen (20)

"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 

Bathcamp 2010-riak

  • 2. What is Riak? • Documented orientated database • Written in Erlang • Based on Dynamo[1] and CAP Theorem[2] • Highly fault tolerant • HTTP and ProtoBuff interface • Write MapReduce in Erlang or JavaScript 1. http://goo.gl/r8Np 2. http://www.julianbrowne.com/article/viewer/brewers-cap-theorem
  • 3. Same, Same but different • Riak solves similar problems to MongoDB • Semi-structured data modeled as "documents” • Storage of non-document data in the database • High write-availability • Riak is intrinsically multi-node scalable • Mongo in comparison is single system (+ sharding) • Riak achieves availability via quorum writes • Mongo uses performant in-place writes • Riak uses “masterless” replication
  • 4. N/R/W – Dynamo N = Number of replicas to store R = Number of replicas needed to read W = Number of replicas needed to read • These principals first appeared in an Amazon research paper known as Dynamo
  • 5. • 160bit integer key space. Each node that joins is assigned part of that space for consistent hashing • Hashing means any node can service any request making the cluster masterless and eventually consistant Number of replicas
  • 6. • Number of replies before Riak gives the client a successful reply. • Tries to access all nodes, but as soon as the N/R is satisfied a response is given Reads
  • 7. • Same as reads; W implies the number of successful nodes that must reply before the write is considered consistent by the client Writes
  • 8. Extreme example • Given N=10, R=W=2 we could have 8 nodes down and the cluster would still be fully available to all clients
  • 9. What does this all mean? • N/R/W specified at request time, so each client can specify its own tolerance for outages dynamically • Despite any outages within the cluster, the whole cluster can still appear available based on N/R/W • Given N=3 and R=W=2, we can have 3-2=1 node down/unreachable/laggy in the cluster • Stupidly high availability complete with eventual consistency controlled by dynamic clients
  • 10. Brewer’s CAP Theorem • Consistency • Availability • Partition Tolerance • You cant have all things, all the time… • …but you can have some of each, all the time! • Riak is about choosing your own levels of each according to your use case
  • 11. Consistency • Start with document version zero • Things get redistributed and n0 and n2 are sitting in NYC and n1 and n3 are in London • What if stuff changes??
  • 12. Consistency • Uh oh: inconsistency • Both parts of the cluster are still fully available • NYC serves v1 whilst London serves v0 • The network resumes and Riak determines the latest version by using vector clocks
  • 13. Consistency • What if both sides of the Atlantic changed? • Riak is unable to determine which is the right document, both are returned to the client with an indication of the inconsistency
  • 14. • Distributed, fault-tolerant full-text searching • Lucene syntax for queries • No need for index sharding • Linier scaling • Double the number of nodes to get double the search capacity (awesome!) • Search via: • Fields, wildcards, fuzzy text or token proximity Riak Search