1. 10/19/12 Introduction to MongoDB and Ruby
INTRODUCTION TO MONGODB AND RUBY
Gary J. Murakami, Ph.D.
Lead Engineer & Ruby Evangelist
10gen (the MongoDB company)
gary.murakami@10gen.com
Tuesday, September 25, 2012
https://github.com/gjmurakami-10gen/talk-mother-lode
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 1/26

2. 10/19/12 Introduction to MongoDB and Ruby
THE CALIFORNIA GOLD RUSH OF 1849
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 2/26

3. 10/19/12 Introduction to MongoDB and Ruby
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 3/26

4. 10/19/12 Introduction to MongoDB and Ruby
MONGODB: RUBY'S MOTHER LODE
Ruby is a gem of a programming
language
dynamic, multi-paradigm
expressive, powerful, enjoyable
MongoDB is the mother lode of DBs
BSON documents, high-performance
indexing, replication, sharding
flexible, powerful, enjoyable
Ruby and MongoDB are well matched
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 4/26

5. 10/19/12 Introduction to MongoDB and Ruby
OUTFITTING BEYOND YOUR PICKAXE BOOK
MAC OS X - MONGODB QUICKSTART
$be udt
rw pae
$be isalmnob
rw ntl ogd
$mno
ogd
RUBY DRIVER SETUP - RUBY LANGUAGE CENTER
$gmisalmno
e ntl og
$gmisalbo_x
e ntl snet
Gemfile
gm'og'
e mno
gm'snet
e bo_x'
RUBYMINE
IDE includes support for Mongoid and
MongoDB
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 5/26

6. 10/19/12 Introduction to MongoDB and Ruby
LET'S DIG IN
TWITTER FRIENDS - LOAD USERS
rqie'og'
eur mno
rqie'tplet
eur htcin'
rqie'sn
eur jo'
sre_ae=AG[]| 'ayuaai
cennm RV0 | grmrkm'
finsisui="tp:/p.wte.o//red/d.sncro=1sre_ae#sre_ae"
red_d_r hts/aititrcm1finsisjo?usr-&cennm={cennm}
fin_d =JO.as(TPletgtfinsisui.oy[is]
redis SNpreHTCin.e(red_d_r)bd)'d'
a few lines of Ruby gets and parses JSON data from the web - twitter API
libraries/gems
mongo, bson, bson_ext
httpclient, json
language capability
method call chaining
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 6/26

7. 10/19/12 Introduction to MongoDB and Ruby
TWITTER FRIENDS - LOAD USERS (CONTINUED)
cneto =Mno:oncinnw
oncin og:Cneto.e
d =cneto[titr]
b oncin'wte'
cleto =d[ues]
olcin b'sr'
fin_d.ahsie10 d |d|#bs patcs
redisec_lc(0) o is et rcie
ueslou_r ="tp:/p.wte.o//sr/okpjo?sri={d.on''}
sr_okpui hts/aititrcm1ueslou.snue_d#isji(,)"
rsos =HTCin.e(sr_okpui
epne TPletgtueslou_r)
dc =JO.as(epnebd)#hg-ee ojcs-Aryo Hse
os SNprersos.oy ihlvl bet ra f ahs
dc.ah|o|dc'i' =dc'd] #ue sple _d
osec{dc o[_d] o[i'} sr upid i
cleto.netdc,:ae= tu)#n shm!-bl isr -bs patcs
olcinisr(os sf > re o cea uk net et rcie
ed
n
pt "sr:{olcincut"
us ues#cleto.on}
a small number of lines load JSON data from the web into a collection
DB connection, database, collection
docs - Ruby Array of Hashes from JSON parsing
doc primary _id set by user or driver
collection bulk insert and safe mode
collection count
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 7/26

8. 10/19/12 Introduction to MongoDB and Ruby
HITTING PAYDIRT
TWITTER TOP 10 USERS BY FOLLOWERS COUNT
ues=Mno:oncinnw'wte'[ues]
sr og:Cneto.e[titr]'sr'
uesfn(} :ot= {floescut - } :ii = 1)ec d |o|
sr.id{, sr > olwr_on: 1 , lmt > 0.ah o dc
pt dcvle_t'olwr_on' 'cennm'.on"t)
us o.ausa(floescut, sre_ae)ji("
ed
n
a simple query with sort and limit
find - cursor enumerable
expressed in Ruby code
driver serializes into BSON and Mongo Wire Protocol
no text language generation or parsing
1023
491 Dobx
rpo
202
59 MnoB
ogD
97
04 ocn
so
84
18 risof
alcn
70
32 sf3
p1
47
34 1gn
0e
25
29 khdrw
cooo
12
95 rt
i
19
25 der
mr
14
18 arnekan
aohcmn
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 8/26

9. 10/19/12 Introduction to MongoDB and Ruby
DIGGING DEEPER
TWITTER - LOAD TWEETS (SIMPLIFIED)
d =Mno:oncinnw'wte'
b og:Cneto.e[titr]
ues=d[ues]
sr b'sr'
tet =d[tet'
wes b'wes]
tet.nueidx'sri' #dtntto t seiysbil
wesesr_ne(ue.d) o oain o pcf ufed
uesfn(} {ils {d tu,sre_ae tu,snei:tu})ec d |sr
sr.id{, fed: i: re cennm: re ic_d re}.ah o ue|
titrue_ieieui= "tp:/p.wte.o//ttssue_ieiejo?sri={s
wte_srtmln_r hts/aititrcm1saue/srtmln.snue_d#u
e[i'}cut20icuerstu&otiuo_eal=re
r'd]&on=0&nld_t=recnrbtrdtistu"
titrue_ieieui+ "ic_d#ue[snei'} i ue[snei'
wte_srtmln_r = snei={sr'ic_d]" f sr'ic_d]
rsos =HTCin.e(wte_srtmln_r)
epne TPletgttitrue_ieieui
dc =JO.as(epnebd)#hg-ee ojcs
os SNprersos.oy ihlvl bet
dc.ah|o|dc'i' =dc'd] #ue sple _d
osec{dc o[_d] o[i'} sr upid i
tet.netdc,:otneo_ro = tu)#bl isr
wesisr(os cniu_nerr > re uk net
uesudt(_d ue[_d],{$e'= {snei'= dc.at'd]}
sr.pae{i: sr'i'} 'st > 'ic_d > osls[i'})
pt tet.on(qey= {ue.d = ue[i'}
us wescut:ur > 'sri' > sr'd])
ed
n
pt "wes#tet.on}
us tet:{wescut"
a small number of lines loads and records tweets
indexing - secondary indexes
retrieving a subset of fields
update - record since_id
count with query selector
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 9/26

10. 10/19/12 Introduction to MongoDB and Ruby
TWITTER TOP 10 TWEETS FOR MONGODB BY RETWEET_COUNT
tet =Mno:oncinnw'wte'[tet'
wes og:Cneto.e[titr]'wes]
sre_ae='ogD'
cennm MnoB
tet.id{ue.cennm'= sre_ae,:ot= {rtetcut - } :ii = 1)ec
wesfn('srsre_ae > cennm} sr > ewe_on: 1 , lmt > 0.ah
d |o|
o dc
pt dcvle_t'ewe_on' 'et)ji("
us o.ausa(rtetcut, tx'.on"t)
ed
n
a simple query with a query selector, sort and limit
MongoDB query language
dot notation for nested sub-document fields
12#ogD v. rlae ht:/.os6z7
7 MnoB 22 eesd tp/tc/NRcD
7 #ogD2o2ofcal rlae,w Avne AgeainFaeok MliDt Cne Dpomn
7 mnoBdt fiily eesd / dacd grgto rmwr, ut-aa etr elyet
+60nwfaue ht:/.oDSWw
0 e etrs tp/tc/MdGN
5 Anucn fe oln MnoBcass ht:/.oRobl
9 nonig re nie ogD lse. tp/tc/IA77
3 R @mkl:Itouigmnop afotn fr#ogD' qeypoie hts/tc/OSsW
0 T jioa nrdcn ogq: rned o MnoBs ur rflr tp:/.oRCz6
2 Sg u frfe oln MnoBcusssatn i Otbrht:/.oI6QN
9 in p o re nie ogD ore trig n coe tp/tc/m84M
2 100 hv sge u frfe MnoBtann snetecusswr anucdtdyht:/.o
4 ,0+ ae ind p o re ogD riig ic h ore ee none oa tp/tc
/IxHt
oNf1
2 HwDse bitabgdt pafr o asatpbde uigMnoB ht:/.o8Wy7
2 o iny ul i aa ltom n tru ugt sn ogD. tp/tc/q5cT
2 #ogD2o2 aalbefrdwla.ht:/.okGUh
1 mnoBdt, vial o onod tp/tc/chDI
1 MnoBSadn Vsaie ht:/.ooI0j aohrpouto #0eLb
9 ogD hrig iulzr tp/tc/nG8v nte rdc f 1gnas
1 MnoBJv Die 290rlae ht:/.oUvoY
6 ogD aa rvr .. eesd tp/tc/DZAV
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 10/26

11. 10/19/12 Introduction to MongoDB and Ruby
STRIKING GOLD
BSON - BINARY JSON
http://bsonspec.org/
{BO" [aeoe,50,18]
"SN: "wsm" .5 96} →
"x1x0x0x0
3000
0BO0
x4SNx0
&x0x0x0
000
00x0x8x0x0x0wsm0
x200000aeoex0
01x0333x4
x103331@
12x0x2x7x0x0
x00c000
0
x0
0"
x0
1. base types extended to match modern programming languages
2. transfer over wire - Mongo Wire Protocol
request payload includes query and documents to server
response and documents to client
3. store on server disk
4. index on server
fast - no parsing, no server-side data repackaging to disk
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 11/26

12. 10/19/12 Introduction to MongoDB and Ruby
REPLICA SETS
multiple database server nodes
one primary member, many secondary members
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 12/26

13. 10/19/12 Introduction to MongoDB and Ruby
high availability
auto-failover via election by majority
write concern
read scaling - read preferences
backups
upgrades
use (hidden) replica for secondary workload
analytics
data-processing
integration with external systems
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 13/26

14. 10/19/12 Introduction to MongoDB and Ruby
SHARDING
horizontal scaling across multiple nodes
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 14/26

15. 10/19/12 Introduction to MongoDB and Ruby
horizontal scaling across multiple nodes
automatic balancing
failover - each shard can be a replica set
shard key - partition by range
chunks up to 64MB
routing processes - mongos
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 15/26

16. 10/19/12 Introduction to MongoDB and Ruby
NUGGETS - MORE MONGODB FEATURES
mongo shell
JavaScript, native for server-side processing
highly recommended for development and administration
map/reduce
batch processing of data and aggregation operations
aggregation framework
calculate aggregated values without having to use map-reduce
pipeline, operators, expressions
GridFS
specification for storing large files in MongoDB chunks usually 256KB
Capped collections
fixed sized collections that have a very high performance auto-FIFO age-
out feature
Hadoop connector
reading MongoDB data into Hadoop as well as writing map/reduce results
back out
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 16/26

17. 10/19/12 Introduction to MongoDB and Ruby
FULL-BORE MINING WITH WEB FRAMEWORKS
Ruby on Rails, Sinatra, etc.
MONGOMAPPER ODM - JOHN NUNEMAKER
casUe
ls sr
icueMnoapr:ouet
nld ogMpe:Dcmn
ky:ae Srn
e nm, tig
ky:g, Itgr
e ae nee
mn :obe
ay hbis
ed
n
casHby
ls ob
icueMnoapr:meddouet
nld ogMpe:EbdeDcmn
ky:ae
e nm, Srn
tig
ky:tre,Tm
e satd ie
ed
n
ue =Ue.e(nm = 'rno'
sr srnw:ae > Badn)
ue.obe.ul(nm = 'rgamn' :tre = 1.er.g)
srhbisbid:ae > Pormig, satd > 0yasao
ue.ae
srsv!
Ue.hr(nm = 'rno'.is
srwee:ae > Badn)frt
simplified model different from ActiveRecord
good example for Ruby metaprogramming
small is beautiful
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 17/26

18. 10/19/12 Introduction to MongoDB and Ruby
MONGOID ODM - DURRAN JORDAN
casAts
ls rit
icueMnod:ouet
nld ogi:Dcmn
fed:ae tp:Srn
il nm, ye tig
ebd_ay:ntuet
mesmn isrmns
ed
n
casIsrmn
ls ntuet
icueMnod:ouet
nld ogi:Dcmn
fed:ae tp:Srn
il nm, ye tig
ebde_n:rit
meddi ats
ed
n
sd=Ats.hr(ae "y Vcos)btenae 1.2)frt
y ritweenm: Sd iiu".ewe(g: 8.5.is
sdisrmnscet(ae "as)
y.ntuet.raenm: Bs"
sdwt(aaae "ad" ssin "akp)sv!
y.ihdtbs: bns, eso: bcu".ae
documents
persistence
querying
ActiveModel relations (belongs_to, has_one, has_many, habtm)
nested attributes
validation
indexing
extras
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 18/26

19. 10/19/12 Introduction to MongoDB and Ruby
MOPED DRIVER - BERNERD SCHAEFER
ssin=Mpd:eso.e( "2...:71"]
eso oe:Ssinnw[ 17001207 )
ssinue"cots"
eso.s eh_et
ssinwt(ae tu)d |ae
eso.ihsf: re o sf|
sf[atss.netnm:"y Vcos)
ae:rit]isr(ae Sd iiu"
ed
n
ssin:rit]fn(ae "y Vcos)udt($uh= {isrmns {nm:"as })
eso[atss.idnm: Sd iiu".pae:ps > ntuet: ae Bs" }
a MongoDB driver for Ruby which exposes a simple, elegant, and fast API
fast, thread-safe, uses sidekiq
pure Ruby - no C extensions
metaprogramming - extends standard classes
Ruby > 1.9
used by Mongoid 3
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 19/26

20. 10/19/12 Introduction to MongoDB and Ruby
JRUBY
The Best of the JVM
High performance
Real threading - multi-core parallelism
Vast array of libraries
It's Just Ruby
Ruby 1.8.7 & 1.9.2 compatible
Both versions in single install
bson_java
Mongo Hadoop connector
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 20/26

21. 10/19/12 Introduction to MongoDB and Ruby
FOOL'S GOLD - RDBMS CASE STUDY
ADOBE PHOTOSHOP LIGHTROOM
"Lightroom 3 Catalog.lrcat"
0.5 GB
66 Sqlite3 tables
20 tables with column image INTEGER
can be loaded with 40 lines of Ruby
can be reduced to 1 image collection
THE COST OF USING RDBMS FOR PERSISTENCE
RDBMS flattened and normalized our data into rigid tables
RDBMS costs O(10**6) - O(1000000) and more with many joins
Memory O(10**9/sec) - O(1000000000)
RDBMS O(10**3/sec) - O(1000)
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 21/26

22. 10/19/12 Introduction to MongoDB and Ruby
THE MOTHER LODE
MONGODB SUMMARY
replica sets for high availability
sharded clusters for scaling
very high performance
secondary indices
BSON is fast for transfer, storage, queries
server-side processing
map-reduce, aggregation
Hadoop connector
very low cost - open source - community
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 22/26

23. 10/19/12 Introduction to MongoDB and Ruby
MONGODB CONCLUSION
enjoyable software development
10gen and Moped Ruby drivers
MongoMapper and Mongoid ODMs
no schema migrations forced
well-matched to programming languages
bigger data
Ruby and objects
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 23/26

24. 10/19/12 Introduction to MongoDB and Ruby
ANNOUNCEMENT AND QUESTIONS
MONGODB AND RUBY BLOGGING CONTEST
Objective
Write about best practices for using MongoDB with Ruby.
Post your blog posts in the comments section of
this blog post by October 10 to be entered to win.
http://blog.10gen.com/post/32158163299/mongodb-and-ruby-blogging-
contest
Google: mongodb and ruby blogging contest
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 24/26

25. 10/19/12 Introduction to MongoDB and Ruby
THANK YOU
AND MAY YOU STRIKE GOLD
{
nm:"ayJ Mrkm,P..,
ae Gr . uaai hD"
tte "edEgne adRb Eagls"
il: La nier n uy vneit,
cmay "0e (h MnoBcmay"
opn: 1gn te ogD opn),
poe "-6-3-85x05,
hn: 1862781 81"
mbl:"-0-8-61,
oie 1987762"
eal "aymrkm@0e.o"
mi: gr.uaai1gncm,
i:"juaai(I),
m gmrkm AM"
titr "GrMrkm"
wte: @ayuaai,
bo:"ryhswie.lgptcm,
lg gagotrtrboso.o"
wbie "w.oelog,
est: wwnbl.r"
lnei:"w.ikdncmpbgr-uaai13a37,
ikdn wwlnei.o/u/aymrkm//6/2"
fcbo:"aeokcmgr..uaai
aeok fcbo.o/ayjmrkm"
}
https://github.com/gjmurakami-10gen/talk-mother-lode
git://github.com/gjmurakami-10gen/talk-mother-lode.git
MongoDB
Ruby Language Center
Github mongodb/mongo-ruby-driver
file:///Users/gjm/10gen/talk‑mother‑lode/index.html?print‑pdf#/ 25/26