De normalised london aggregation framework overview

DeNormalised London:
Aggregation Framework Overview
Chris Harris
Email : charris@10gen.com
Twitter : cj_harris5

Wednesday, 21 March 12

Terminology

RDBMS MongoDB
Table Collection
Row(s) JSON Document
Index Index
Join Embedding & Linking
Partition Shard
Partition Key Shard Key


Here is a “simple” SQL Model
mysql> select * from book;
+----+----------------------------------------------------------+
| id | title |
+----+----------------------------------------------------------+
| 1 | The Demon-Haunted World: Science as a Candle in the Dark |
| 2 | Cosmos |
| 3 | Programming in Scala |
+----+----------------------------------------------------------+
3 rows in set (0.00 sec)

mysql> select * from bookauthor;
+---------+-----------+
| book_id | author_id |
+---------+-----------+
| 1| 1|
| 2| 1|
| 3| 2|
| 3| 3|
| 3| 4|
+---------+-----------+

mysql> select * from author;
+----+-----------+------------+-------------+-------------+---------------+
| id | last_name | ﬁrst_name | middle_name | nationality | year_of_birth |
+----+-----------+------------+-------------+-------------+---------------+
| 1 | Sagan | Carl | Edward | NULL | 1934 |
| 2 | Odersky | Martin | NULL | DE | 1958 |
| 3 | Spoon | Lex | NULL | NULL | NULL |
| 4 | Venners | Bill | NULL | NULL | NULL |
+----+-----------+------------+-------------+-------------+---------------+


The Same Data in MongoDB

{
"_id" : ObjectId("4dfa6baa9c65dae09a4bbda5"),
"title" : "Programming in Scala",
"author" : [
{
"first_name" : "Martin",
"last_name" : "Odersky",
"nationality" : "DE",
"year_of_birth" : 1958
},
{
"first_name" : "Lex",
"last_name" : "Spoon"
},
{
"first_name" : "Bill",
"last_name" : "Venners"
}
]
}


What problem are we solving?
• Map/Reduce can be used for aggregation…
• Currently being used for totaling, averaging, etc
• Map/Reduce is a big hammer
• Simpler tasks should be easier
• Shouldn’t need to write JavaScript
• Avoid the overhead of JavaScript engine
• We’re seeing requests for help in handling
complex documents
• Select only matching subdocuments or arrays


How will we solve the problem?
• New aggregation framework
• Declarative framework (no JavaScript)
• Describe a chain of operations to apply
• Expression evaluation
• Return computed values
• Framework: new operations added easily
• C++ implementation


Aggregation - Pipelines
• Aggregation requests specify a pipeline
• A pipeline is a series of operations
• Members of a collection are passed
through a pipeline to produce a result
• ps -ef | grep -i mongod


Example - twitter
{
"_id" : ObjectId("4f47b268fb1c80e141e9888c"),
"user" : {
"friends_count" : 73,
"location" : "Brazil",
"screen_name" : "Bia_cunha1",
"name" : "Beatriz Helena Cunha",
"followers_count" : 102,
}
}

• Find the # of followers and # friends by location


Example - twitter
db.tweets.aggregate(
{$match:
{"user.friends_count": { $gt: 0 },
"user.followers_count": { $gt: 0 }
}
},
{$project:
{ location: "$user.location",
friends: "$user.friends_count",
followers: "$user.followers_count"
}
},
{$group:
{_id: "$location",
friends: {$sum: "$friends"},
followers: {$sum: "$followers"}
}
}
);


Example - twitter
{$match:
{"user.friends_count": { $gt: 0 }, Predicate
}
},
{$project:
friends: "$user.friends_count",
followers: "$user.followers_count"
}
},
{$group:
{_id: "$location",
}
}
);


Example - twitter
{$match:
}
},
{$project:
Parts of the
friends: "$user.friends_count", document you
followers: "$user.followers_count" want to project
}
},
{$group:
{_id: "$location",
}
}
);


Example - twitter
{$match:
}
},
{$project:
Parts of the
friends: "$user.friends_count", document you
followers: "$user.followers_count" want to project
}
},
{$group:
{_id: "$location", Function to
friends: {$sum: "$friends"}, apply to the
} result set
}
);


Example - twitter
{

"result" : [

{

"_id" : "Far Far Away",

"friends" : 344,

"followers" : 789

},
...

],

"ok" : 1
}


Demo
Demo ﬁles are at https://gist.github.com/
2036709


Projections
• $project can reshape results
• Include or exclude fields
• Computed fields
• Arithmetic expressions
• Pull fields from nested documents to the top
• Push fields from the top down into new virtual
documents


Unwinding
• $unwind can “stream” arrays
• Array values are doled out one at time in the
context of their surrounding documents
• Makes it possible to ﬁlter out elements before
returning


Grouping
• $group aggregation expressions
• Deﬁne a grouping key as the _id of the result
• Total grouped column values: $sum
• Average grouped column values: $avg
• Collect grouped column values in an array or
set: $push, $addToSet
• Other functions
• $min, $max, $ﬁrst, $last


Sorting
• $sort can sort documents
• Sort speciﬁcations are the same as today,
e.g., $sort:{ key1: 1, key2: -1, …}


Computed Expressions
• Available in $project operations
• Prefix expression language
• $add:[“$field1”, “$field2”]
• $ifNull:[“$field1”, “$field2”]
• Nesting: $add:[“$field1”, $ifNull:[“$field2”,
“$field3”]]
• Other functions….
• $divide, $mod, $multiply


Computed Expressions
• String functions
• $toUpper, $toLower, $substr
• Date ﬁeld extraction
• $year, $month, $day, $hour...
• Date arithmetic
• $ifNull
• Ternary conditional
• Return one of two values based on a
predicate


download at mongodb.org

We’re Hiring !
Chris Harris
Email : charris@10gen.com
Twitter : cj_harris5

conferences, appearances
http://www.10gen.com/events
and meetups
http://www.meetup.com/London-MongoDB-User-Group


De normalised london aggregation framework overview

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (7)

Ähnlich wie De normalised london aggregation framework overview

Ähnlich wie De normalised london aggregation framework overview (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

De normalised london aggregation framework overview