Cybersecurity Challenges with Generative AI - for Good and Bad
No SQL : Which way to go? Presented at DDDMelbourne 2015
1. NoSQL: Which way to go?
DDDMelbourne2015
HimanshuDesai,AhmedElharouny&DanielJanczak
3. About us
• Himanshu Desai (@H_Desai)
• Ahmed Elharouny(@Harouny)
• Daniel Janczak (@DanielJanczak)
And we all work for Readify
4. Page
Why NoSQL?
Scale
Pros
NoSQL technologies can offer
more scalability than relational
databases
Cons
Often lose some benefits of
relational databases e.g.
database-wide transactions
Better Formats
Pros
Avoiding ORMs
Cons
Limited BI tooling, Data
hard to share outside the
application
Work in more flexible
ways
Pros
No fixed schemas
Cons
Fixed schemas help
prevent errors
7. What is RavenDB?
› Open source NoSQL database for .NET
› Developed by Hibernating Rhinos Ltd
› Available
› As a standalone package
› As embedded version
› On Azure and Amazon
8. Schema-free
Store data as Plain Old C# Object
(POCO)
Or directly as JSON
var author = new Author {
Name = "George Martin",
Books = new[] {
new Book {Title = "A Game of Thrones"},
new Book {Title = "A Clash of Kings"}
}
};
session.Store(author);
{
"Name": "George Martin",
"Books": [
{
"Title": "A Game of Thrones",
"Genre": "Fantasy"
},
{
"Title": "A Clash of Kings",
"Genre": "Fantasy
}
]
}
14. Eventual consistency
Good or bad ?
Newspaper Good! Very few writes, thousands of
reads, doesn’t have to be immediate
Product signup Good! All applicant information held
together
Banking system What if the credits and debits sometimes
don’t match?
Patient drug
monitoring
What if the drug doses are not updated
on time?
15. Our test data
public class Author {
public string Name {get; set;}
public string Surname {get; set;}
public Book[] Books {get; set;}
}
public class Book {
public string Title {get; set;}
public string Genre {get; set;}
public DateTime? PublishedOn {get; set;}
public decimal? Price {get; set;}
}
16. Our test data continues…
• George Martin
• A Game of Thrones, Fantasy, 6/8/1996, $12.34
• A Clash of Kings, Fantasy, 16/11/1998
• Dan Simmons
• Hyperion, Science Fiction, 1/1/1989, $34.56
• Terror, Thriller, 8/1/2007, $23.45
• John Tolkien
• Hobbit, Fantasy, 21/9/1937, $12.34
• Stephen King
• The Stand, Horror, 1/9/1978, $12.34
17. A quick look at code
Code samples:
https://github.com/danieljanczak/ddd-melb2015-ravendb-samples
22. MongoDB Features
The best features of key/values stores, document databases and relational
databases in one.
• Open source
• Cross platform
23. The best features of key/values stores, document databases and relational
databases in one.
MongoDB Features
• Open source
• Cross platform
• Strong indexing
24. The best features of key/values stores, document databases and relational
databases in one.
MongoDB Features
• Open source
• Cross platform
• Strong indexing
• Strong query language
25. The best features of key/values stores, document databases and relational
databases in one.
MongoDB Features
• Open source
• Cross platform
• Strong indexing
• Strong query language
• Replication
26. The best features of key/values stores, document databases and relational
databases in one.
MongoDB Features
• Open source
• Cross platform
• Strong indexing
• Strong query language
• Replication
• Sharding
27. The best features of key/values stores, document databases and relational
databases in one.
MongoDB Features
• Open source
• Cross platform
• Strong indexing
• Strong query language
• Replication
• Sharding
• Native drivers
28. The best features of key/values stores, document databases and relational
databases in one.
MongoDB Features
• Open source
• Cross platform
• Strong indexing
• Strong query language
• Replication
• Sharding
• Native drivers
• File storage (GridFS)
29. The best features of key/values stores, document databases and relational
databases in one.
MongoDB Features
• Open source
• Cross platform
• Strong indexing
• Strong query language
• Replication
• Sharding
• Native drivers
• File storage (GridFS)
• Aggregation framework + MapReduce
33. How would you access it?
Application
Database
Collections
Stored
Procedures
Triggers UDFs
Documents
Libraries:
.NET, JavaScript,
Node.js, Python,
…
REST
Collections
Database
34. DocumentDB
Working with data
RESTful access methods
For Create/Read/Update/Delete
(CRUD) operations
DocumentDB SQL
A query language with SQL-
derived syntax
Example:
SELECT c.Books
FROM Authors c
WHERE c.name =
“George"
Executing logic in the DB
( Javascript)
Stored procedures
Triggers
User-defined functions
(UDFs)
- Allow extending
DocumentDB SQL
35. Sharding and Transaction
Collection Collection Collection
Database
JSON
JSON
JSON
JSON
JSON
JSON
JSON
JSON
Collection
JSON
JSON
JSON
The unit of
Sharding is a
collection is a
collection
Atomic transactions can span
only a single collection
collection
36. .NET Development
• You need following from the portal after provision a database
• Endpoint Url
• Authorization key
• Nu-get package
• Install-Package Microsoft.Azure.DocumentDB
• Use them to create DocumentClient
39. In a nutshell - ACID
• RavenDB
• Fully transactional writes on document level, even between collections
• Reads use eventual consistency
• Support for DTS
• MongoDB
• Only supports ACID transactions at the document level.
• Multi-collection (table) transactions not supported.
• Users can choose the level of consistency using (WriteConcern and ReadPreference)
• DocumentDB
• Full ACID support on the collection level
• 4 levels of consistency, Extremely rich query model.
40. In a nutshell - availability and tooling
• RavenDB
• On-prem (Windows), Cloud (RavenHQ) on Azure and Amazon, Embedded version / unit testing
• Built in web based Management Studio
• .NET and JAVA drivers, REST API
• Possible access through LinqPad
• MongoDB
• On-prem (Windows, Linx, Mac), Cloud (i.e MongoLab)
• MongoDB Cloud Manager (monitoring, automated management)
• Mongo Shell
• Lots of 3rd party GUIs i.e. Robomongo
• DocumentDB
• Accessed from virtually “any” system. Available in “Azure” only at this point.
• Azure Web Portal
• DocumentDB Studio
• LinqPad
• SDK available for various platform and REST based access
41. In a nutshell – querying and indexing
• RavenDB
• Only very basic queries supported without static indexes
• Map-reduce
• Full text search
• Eventual consistency by default
• MongoDB
• Strong indexing (_id indexed by default, custom indexes, compound, unique, text). Async indexing.
• Query from primary by default (Strong consistency)
• Query from “nearest” if needed (Eventual consistency)
• When writing you choose when to consider write operation successful.
• DocumentDB
• Very rich query model – Supports Linq to objects, Lambda expression and SQL style syntax. Supports
Stored procedure, triggers and UDFs
• Auto index by default.
• Support lazy indexing if required.
• 4 levels of consistency to choose from (Strong, Session, Bounded Stalesness,Eventual consistency
42. In a nutshell - performance
• RavenDB
• Writes directly to disk (safety over speed)
• Indexing in the background
• Query results are instantly available in indexes
• May need to wait for specific update
• MongoDB
• TCP/IP based communication.
• Uses BSON (lightweight, traversable, efficient).
• MMAPv1 storage engine uses OS memory mapped files.
• Wiredtiger storage engine (7,000,000 op/second 28 threads).
• Choose level of performance vs consistency.
• DocumentDB
• 3 Levels of performance offered.
• Normalised CPU, memory, Disk and network as Request Units (RU)
• TCP/HTTPS based access available
43. In a nutshell – gotchas!
• RavenDB
• Need to reset underlying Lucene indexes – they do crash
• Database defaults (winning numbers: 15, 50, 16384; 128, 1024)
• Static indexes
• Eventual consistency
• MongoDB
• 32-bit limitations
• Max document size 16 MB
• Modeling currency
• DocumentDB
• Max response size : 1MB
• Max request size including attachment: 512KB.
• Consistency is set at database level not at collection level.