Talk about building real shared nothing web backends using crate data on Web Backend Meetup May 2014

1. Shared Nothing Web
Backends
Matthias Wahl - developer @ crate.io

2. Agenda
Shared Nothing
Crate
Shared Nothing Architectures using Crate
Cluster Workout

3. Prepare for Workout
1.) bash -c "$(curl -L try.crate.io)"
https://cdn.crate.io/downloads/releases/crate-0.38.2.tar.gz
2.) sh$ bin/crate
0.) share our “craty” WLAN
pw: select*fromcrate;

4. Prepare for Workout
3.) start up the twitter tutorial

5. Shared Nothing

6. Shared Nothing
Wikipedia:
!
a distributed computing architecture in which each
node is independent and self-sufficient, and there is no
single point of contention across the system. More
specifically, none of the nodes share memory or disk
storage.

7. Shared Nothing
Origin:
!
Micheal Stonebraker
“The Case for Shared Nothing” (1986)
!
http://pdf.aminer.org/000/255/770/
the_case_for_shared_nothing.pdf

8. Shared Nothing
Principle of most functional languages
Shared state/resources (memory) is what
makes threading hard
No SPOF
Easy concurrency

9. DB
Cache
App App App App
DB -
Replica
Cache Cache
Bad Idea

10. DB
Cache
App App App App
DB -
Replica
Cache Cache
Bad Idea

11. Crate
Austria and Berlin based Startup
founded 2013
open source since april
https://github.com/crate/crate

12. Crate
shared nothing massively scalable datastore
standing on the shoulders of giants

13. Crate

14. Crate
automatic sharding and replication
(semi-) structured models
blob support
SQL query language

15. Crate
powerful fulltext search capabilities
complex aggregations in near real-time
- no joins

16. Crate
crate-java
crate-jdbc
crate-python (+SQLAlchemy)
crate-ruby
…
CRATE Clients

17. Shared Nothing with
Crate
data and query execution is
distributed among all nodes
no master/slave - all nodes are equal
automatic sharding & replication

18. Shared Nothing with
Crate
example architecture
crate
app
crate
app
crate
app
…
Load Balancer

19. Shared Nothing with
Crate
horizontally scalable as hell
still flexible enough for complex applications
high availability for your whole stack

20. Shared Nothing with
Crate
create table blogpost (
id string primary key,
created timestamp,
text string,
creator string
)

21. Shared Nothing with
Crate
from sqlalchemy import Column, String, DateTime
from microblog.model import Base
import uuid
!
!
def genuuid():
return str(uuid.uuid4())
!
!
class BlogPost(Base):
!
__tablename__ = 'blogpost'
!
id = Column(String, default=genuuid, primary_key=True)
text = Column('text', String, nullable=False)
creator = Column('creator', String, nullable=False)
created = Column('created', DateTime, nullable=False)

22. Shared Nothing with
Crate
@rpcmethod_route(request_method="GET")
def list(self):
""" Return all blogposts
"""
query =
DBSession.query(BlogPost).order_by(BlogPost.created.desc())
blogposts = query.all()
result = []
for post in blogposts:
result.append({'id': post.id,
'created': post.created.isoformat(),
'text': post.text,
'creator': post.creator})
return {"data": {"blogposts": result}}

23. Shared Nothing with
Crate
get into detail:
!
http://lovelysystems.github.io/
lovely.microblog/

24. Cluster Workout!!!

25. CRATE
Thank you
web: https://crate.io/
github: https://github.com/crate
twitter: @cratedata
IRC: #crate
stackoverflow tag: cratedata