Talk about how MarkLogic Server uses a inverted index (like search engines) to optimize this document oriented NoSQL database

1. MarkLogic Developer Community NoSQL Frankfurt, 2010 Awesome document-oriented NoSQL database Beyond NoSQLwith MarkLogicThe Universal Index and

2. nunojob nuno.job@marklogic.com @dscape| nunojob.com

3. how?? Ad hoc Structure Predefined IDMS Ad hoc Predefined Queries

4. Indexes! indexes! so… filter map reduce !? well… sort of… flickr.com/ayalan

5. divide and conquer level of abstraction: ease of use database consistent-hashing-like thingy partition2 partition3 partition1 standa group of trees makes sense to have indexes in the same place

6. 1st index resolution 2nd get documents shared-nothing cluster E Host 1 E Host 3 E Host 2 AppServer Same Code- base Data D Host 4 D Host 5 D Host 6 D Host k HA&DR partition1 partition2 partition3 partitionm partition4

7. universal index Range Indexes Term Term List “accelerating” 123, 127, 129, 152, 344, 791 . . . “creation” 122, 125, 126, 129, 130, 167 . . . “content” 123, 126, 130, 142, 143, 167 . . . “application” 123, 130, 131, 135, 162, 177 . . . “agility” Document References 126, 130, 167, 212, 219, 377 . . . <article> . . . <article> / <title> . . . 126, 130, 167, … product: MarkLogic Geospatial

8. semi structured article title paragraph get tables from computer science articles that include a title with word “content” but not the word “agility” information un-ordered list metadata structure parentchild paragraph table full text footer

9. universal index in kelly speak: zippy-ing Range Indexes Term Term List “accelerating” 123, 127, 129, 152, 344, 791 . . . “creation” 122, 125, 126, 129, 130, 167 . . . “content” 123, 126, 130, 142, 143, 167 . . . “application” 123, 130, 131, 135, 162, 177 . . . “agility” Document References 126, 130, 167, 212, 219, 377 . . . <article> 122, 125, 126, 129, 130, 143, 167 <article> / <title> 122, 125, 126, 129, 130, 167 . . . 126, 130, 167, … product: MarkLogic Geospatial

10. wait a minute… Directories Exclusive, hierarchical, analogous to file system, map to URI Collections Set-based, N:N relationship Security Invisible to your app

11. universal index Range Indexes Term Term List “accelerating” 123, 127, 129, 152, 344, 791 . . . “creation” 122, 125, 126, 129, 130, 167 . . . “content” 123, 126, 130, 142, 143, 167 . . . “application” 123, 130, 131, 135, 162, 177 . . . “data base” Document References 126, 130, 167, 212, 219, 377 . . . <article> . . . <article> / <title> . . . 126, 130, 167, … product: MarkLogic Directory: /articles/ Collection: CS Role:Editor + Action:Read Geospatial

12. throughput in memory stand(s) durability: journal flickr.com/kt

13. mvcc append only database, use sys-timestamps to know which document is currently available and the marklogic time machine delete update (could also be create) create System timestamp query

14. too good to be true? try us out… free version available! developer.marklogic.com/products markmail.org pairs.demo.marklogic.com heatmap.demo.marklogic.com bit.ly/ml-demo flickr.com/nattu

15. questions? Love NoSQLdatabases? Want to change the world? We are hiring!! spkr8.com/t/4590 Feedback nuno.job@marklogic.com

16. Open-source, closed development? REST Mobile XQuery and why it’s awesome! not covered but conversations are welcome! App Server + Search + Database Scalable ACID transactions XML vs. JSON ? Merging / Compaction Relevance MVCC Reverse Indexes Alerting High Order Functions Geospatial queries Co-occurrence Meta programming Document databases