A presentation about the problems you'll face when dealing with the relational model and highly customizable general-purpose projects, with a look at the NoSQL word focusing on a real solution, a graph database.

1. Long story short, a true tale about an hopeless DevTeam
CMF: a pain in the F
David Funaro & Alessandro Nadalin
May, 14 2011

2. let us introduce
The Problem™

3. There comes the boss
< there's a new project

4. There comes the boss
< seems easy,
let's use Joomla!

5. Don't open that door!

6. CMS
The 80/20 dilemma

7. 80% is cool at all

8. but as
the requirements grows

9. (you always knew that)

10. the project becomes a mess

11. There comes the boss
< can I haz fotonic
menthal scanner
in the website

12. There comes the boss
< can't do boss
Joomla! is not meant...

13. There comes the boss
< STFU NOOB DO THAT
OR BLA BLA BLA

14. what if

15. There comes the boss
< seems complex,
no Joomla this time,
let's use a framework!

16. There comes the boss
< STFU NOOB WE
SHOULD GO LIVE
NEXT WEEK BLA BLA

17. Framework
The 20/80 dilemma

18. 20% is cool at all

19. the 80%
is really
funny

20. but has a cost
Time To Market

21. again, what if

22. There comes the boss
< seems complex,
no Joomla this time,
when should it go live?

23. There comes the boss
< salesguy just told the
client tomorrow

24. There comes the boss
< ok, let's use a CMF

25. unfortunately, there is no stable
CMF in the PHP ecosystem

26. so it has a cost
initial development

27. investment
=
know how (team) +
quality (framework) +
velocity (that customers pretend)
=
CMF

28. "Our" CMF

29. Target: End User/Site admin
should be able to manage a site editing pages with
different elements.
Easily, like a CMS.

30. Target: Developer
should be able to plug the CMF with NO PAIN,
using the kick-ass framework at the bottom of the CMF.

32. 2
things

33. Pages

34. Pages
main structure of a web site
organized as a tree
a way to express our contents
usually composed by a series of
containers ( that we'll refer to as
boxes )

35. Boxes

36. Boxes
The way to separate
single elements of a page
Loosely coupled HMVC
structures

37. Boxes
{Bridge}
Entities Pages
Boxes let entities be shown inside webpages

38. Boxes
We dispose of different types of
boxes:
Common
DSL

39. Common Box
Simple
Input field
Textarea
Video
Image
File
Richtext
Composite
Input field + image
input field + image + richetex
textarea + video

40. what about the DSL?

41. Domain Specific Language
DSL = ! (Website 1 Website 2 )

42. Example: Identify DSL
Event Page Event Page
Books Cars Users
Review
News Users News Contract
Widget Widget

43. Web Elements = CMF Entities
Page Cars
Books News
Review Widget
Contract
DSL

44. back to the CMF

45. CMF Stack
Framework: symfony, for example
CMF entities: Site, Pages, Menus, Users ...
DSL entities: cars, contracts (for Hertz)
Common Boxes : Textarea, image, video, ...
Dsl boxes: Cars image gallery

46. Common Boxes + DSL Boxes
=
The main tool for the developer to build a page
with the CMF

47. Example : page & box composition

48. The Page-Box Binding Storage
Data can change its structure fastly, according to the domain model.
We need a technology supporting this fast change.
Since we have N box type, to compose a page we should
look for a few tables.
We need a technology supporting this kind of lookups.

49. so,
How will we
manage the data?

50. RDBMS , of course

51. because different
data always have
the same need

52. one size fits all

53. We tend to think at the way we should store data
not how we're gonna use it

54. and we think that the
shit-it-always-happens migration
will be easy

55. also if that column is useful for one and one only row

56. among milions

57. so we know we're gonna face
a few well-known problems...

58. Data Mapping
Data Structure RDBMS
id 10
name tom
parent_id 19
we have to find a way to re-organize the data-structre to insert our
tree inside a table.
But is really different! Tree find a node in O(nlogn)

59. Data Mapping
Data Structure RDBMS
is better than before, but isn't the best way

60. Data Mapping
Data Structure RDBMS
?
... and NOW??????

61. The Alien
spend more time to normalize
your models
BUT
you always find
something that as a
different Behaviour

62. Unpredictable
Tomorrow
User Story evolve continous
Worse than the Alien

63. EAV
formerly known as:
normalization WTF?!?!?!
or:
how to do 70k joins to retrieve an entity of you model

64. Overnormalized
user_id | mail | phone | address
Large schema is broken
into smaller ones
user_id | mail_id
user_id | phone_id
user_id | address_id

65. Overnormalized
user_id | mail_id LEFT JOIN
user_id | phone_id LEFT JOIN
...
...
...
...
...
user_id | address_id LEFT JOIN

66. Overnormalized
Users List
LEFT JOIN LEFT JOIN
LEFT JOIN
LEFT JOIN
LEFT JOIN
LEFT JOIN LEFT JOIN

67. Overnormalized
LEFT JOIN LEFT JOIN
LEFT JOIN
LEFT JOINList
Users LEFT JOIN
LEFT JOIN LEFT JOIN
LEFT JOIN
LEFT JOIN LEFT JOIN
LEFT JOIN
LEFT JOIN
LEFT JOIN LEFT JOIN
LEFT JOIN
LEFT JOIN LEFT JOIN
LEFT JOIN
What if you need to show only a single primary address?

68. back to the CMF

69. Bind Page with Boxes
Foreing Key -> TextareaBox

70. Bind Page with Boxes
Foreing Key -> ?
AND NOW
AND NOW ???

71. #1 Mantain the RI

72. #2 Not Mantain the RI

73. Maintain or not Maintain
the RI; that is the
question

74. we have to choose

75. have you ever thought
"there should be
another way!"
?

76. Can someone help us?

77. lets try
NoSQL
world

78. Written in: C/C++
Main point: Blazing fast
Protocol: Telnet-like
Disk-backed in-memory database,
but since 2.0, it can swap to disk.
Master-slave replication
Simple keys and values,
but complex operations like
ZREVRANGEBYSCORE
INCR
Values can be set to expire (as in a cache)
$ redis-cli set mykey "my binary safe value" => OK
$ redis-cli get mykey => my binary safe value

79. Best used:
rapidly changing data
with a foreseeable
database size (should fit
mostly in memory)
For example:
analytics
real-time data collection
caching
MQ
log
http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

80. very fast

81. very fast
Object != key -> value

82. very fast
Object != key -> value
Not reliable
Can I haz otha nozql?

83. Written in: C++
Main point: Retains some friendly properties
of SQL
Protocol: Custom, binary (BSON)
Master/slave replication
speaks JavaScript
Built-in sharding
Performance over features
After crash, it needs to repair tables
> j = { name : 'david', surname : 'funaro' };
{ "name" : "david", "surname" : "funaro" }
> db.things.save(t);
> db.things.find();
{ "_id" : ObjectId("4c2209f9f3924d31102bd84a"), "name" : "mongo" }

84. Best used:
you need dynamic queries
you prefer to define indexes,
not map/reduce functions
you need good performance
on a big DB
For example:
all things that you would do
with MySQL or PostgreSQL,
but having predefined
columns really holds you back
http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

85. Document Oriented

86. Document Oriented
Schema Free

87. Document Oriented
Schema Free
Navigate Object
Can I haz otha nozql?

88. Written in: Erlang
Main point: DB consistency, ease of use
Protocol: HTTP, some say REST
Bi-directional replication => master-master
conflict detection ( 409 Conflict )
MVCC: write operations do not block reads
Previous versions of documents are
available
Crash-only (reliable) design
Needs compacting from time to time
Views: embedded map/reduce
jQuery library included
curl -d '{"keys":["bar"]}' -X POST http://host:port/foo/_all_docs?include_docs=true

89. Best used:
accumulating occasionally-
changing data
pre-defined queries
versioning is king
For example:
CRM/CMS systems
API
http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

90. Reliable

91. Reliable
Document Oriented

92. Reliable
Document Oriented
Schema Free

93. Reliable
Document Oriented
Schema Free
Navigate Object
Can I haz otha nozql?

94. Why do we so hardly
need a GraphDB
like OrientDB?

95. "GraphDBs don't avoid relations but they embrace them in
a way that they are not a computational problem
anymore, by making them explicit instead of implicit
through joins."
Claudio Martella
( http://blog.acaro.org/entry/somebody-is-going-to-hate-me-nosparql )

96. Easy joins
select from users where city.country.continent = 'Oceania'

97. Same link semantics,
different objects
select links from menu >> 28:1, 17:5

98. Same link semantics,
different objects
select from [28:1, 17:5]
28:1 17:5
ExternalLink ProductLink
url: http://www.google.com id: 4

99. 28:1 17:5
ExternalLink ProductLink
url: http://www.google.com id: 4
$link->render()

100. Written in: Java
Main point: Graph
Protocol: Binary or HTTP
ACID
Disk-persisted or in-memory,
Cluster
150k inserts per second
Schema-less, *-full, *-mixed
Speaks JSON
Integrated GUI
Documental
SQL commands
curl -X PUT '127.0.0.1:2480/document/demo/1:2' --data "{'@rid': '1:2'}"

101. Reliable

102. Reliable
Graph - Documental

103. Reliable
Graph - Documental
Schema-*

104. Reliable
Graph - Documental
Schema-*
Easy traversal

105. Reliable
Graph - Documental
Schema-*
Easy traversal
SQL-friendly

106. Reliable
Graph - Documental
Schema-*
Easy traversal
SQL-friendly
UberFast ( binary protocol )

107. Reliable
Graph - Documental
Schema-*
Easy traversal
SQL-friendly
UberFast ( binary protocol )
Universal ( HTTP protocol )

108. is so cool

109. but has a problem...

110. ver 1.0

111. rc1

112. but has a couple problems...

113. JAVA

114. but somebody

115. started writing the binary-protocol binding
https://github.com/AntonTerekhov/OrientDB-PHP
( beta, 28 April 2010 )

116. and others

118. wrote the HTTP one
https://github.com/odino/Orient

119. $driver = new OrientHttpClientCurl();
$orient = new OrientFoundationBinding(
$driver, '127.0.0.1', '2480', 'admin', 'admin', 'demo
');
$response = $orient->query("select from fellas where any() traverse(0,-1) ( @rid = 1:1 )");
$output = json_decode($response->getBody());
foreach ($output->result as $friend)
{
echo $friend->name;
// other fun...
}

120. you can start playing with a pure
GraphDB now

121. High-level PHP Component?

122. No

123. but we know that ;)

124. that's why in DNSEE we will build a basic
Object Graph Mapper
for OrientDB

125. ...because...

126. hey, you remember about our CMF?

127. codename: ConGoW

128. codename: ConGoW
contents gone wild
( don't ask why )

129. ConGoW
soon the real kickstart

130. ConGoW
soon the real kickstart
10 devs at the starting point

131. ConGoW
soon the real kickstart
10 devs at the starting point
OSS on GitHub

132. ConGoW
soon the real kickstart
10 devs at the starting point
OSS on GitHub
Stack
Symfony2

133. ConGoW
soon the real kickstart
10 devs at the starting point
OSS on GitHub
Stack
Symfony2
Orient

134. ConGoW
soon the real kickstart
10 devs at the starting point
OSS on GitHub
Stack
Symfony2
Orient
Doctrine2

135. "My only friend, the end"

136. "My only friend, the end"
David Funaro
@ingdavidino
davidfunaro.com

137. "My only friend, the end"
David Funaro Alessandro Nadalin
@ingdavidino @_odino_
davidfunaro.com odino.org

138. "My only friend, the end"
David Funaro Alessandro Nadalin
@ingdavidino @_odino_
davidfunaro.com odino.org

139. "My only friend, the end"
David Funaro Alessandro Nadalin
@ingdavidino @_odino_
http://davidfunaro.com http://odino.org
http://joind.in/talk/view/3008

140. Credits
http://www.flickr.com/photos/pagedooley/5313217918/sizes/l/in/photostream/
http://www.flickr.com/photos/zoetnet/5520594473/sizes/l/in/photostream/
http://www.flickr.com/photos/mhxbhd/3962410821/sizes/o/in/photostream/
http://www.flickr.com/photos/labyrinthx/1955594336/sizes/z/in/photostream/
http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
http://www.flickr.com/photos/salim/19426192/sizes/o/in/photostream/
http://www.flickr.com/photos/mybluevan/3780363712/sizes/l/in/photostream/
http://farm4.static.flickr.com/3064/3086258014_f1925639e0.jpg