SlideShare a Scribd company logo
1 of 68
Download to read offline
CFCOUCHBASE
2.0
INTRODUCING
AARON BENTON @bentonam BRAD WOOD @bdw429s@ORTUSSOLUTIONS
• HISTORY
• CF EXAMPLES
• N1QL EXAMPLES
• SDK CHANGES
• SDK ROADMAP
• LOAD DEMO
PROJECT INCEPTION
Started Oct. 2013
RELEASE 1.0
1.0 was released Feb. 2014
RELEASE 1.1
1.1 was released Jan. 2015
RELEASE 2.0
2.0 was released June 2016
RELEASE 2.0 BRINGS
• JAVA SDK (2.2.5)
• N1QL SUPPORT W/ GSI INDEXES
• REPLICA READS
• DOCUMENT LOCKING
• PREPARED STATEMENTS
• DESIGN DOCUMENT MANAGEMENT
• EXPANDED CONFIG
• AND MORE…
METHOD HOUSEKEEPING
asyncCounter
asyncDecr
asyncGet
asyncGetAndTouch
asyncGetMulti
asyncGetWithCAS
asyncIncr
asyncSaveView
getDocStats
UNSUPPORTED METHODS
add
decr
incr
delete
newQuery
deleteDesignDocument
set
setMulti
setWithCAS
DEPRECATED METHODS NEW METHODS
counter
exists
getAndLock
getEnvironment
getFromReplica
getIndexes
getUtility
insert
invalidateQueryCache
n1qlQuery
publishDesignDocument
remove
removeDesignDocument
replaceWithCAS
unlock
upset
upsertMulti
viewQuery
METHOD HOUSEKEEPING
1.* METHODS 2.* METHODS
add insert
incr
decr
counter
delete remove
newQuery
n1qlQuery
viewQuery
deleteDesignDocument removeDesignDocument
set
upsert
replace
setMulti upsertMulti
setWithCAS replaceWithCAS
The query() method will still be implemented and is a
facade for both n1ql and view queries
CONFIG HOUSEKEEPING
bucketName
dataMarshaller
defaultTimeout
password
servers
useClassLoader
viewTimeout
SUPPORTED OPTIONS
maxReconnectDelay
obsPollInterval
obsPollMax
opQueueMaxBlockTime
opTimeout
readBufferSize
shouldOptimize
timeoutExceptionThreshold
UNSUPPORTED OPTIONS
autoReleaseAfter
bootstrapCarrierDirectPort
bootstrapCarrierEnabled
bootstrapCarrierSslPort
bootstrapHttpDirectPort
bootstrapHttpEnabled
bootstrapHttpSslPort
bufferPoolingEnabled
caseSensitiveKeys
computationPoolSize
connectTimeout
dcpEnabled
defaultMetricsLoggingConsumer
disconnectTimeout
dnsSrvEnabled
eventBus
ioPool
ioPoolSize
keepAliveInterval
kvEndpoints
kvTimeout
managementTimeout
NEW OPTIONS
maxRequestLifetime
mutationTokensEnabled
networkLatencyMetricsCollectorConfig
observeIntervalDelay
packageNameAndVersion
queryEnabled
queryEndpoints
queryPort
queryTimeout
reconnectDelay
requestBufferSize
responseBufferSize
retryDelay
retryStrategy
runtimeMetricsCollectorConfig
scheduler
sslEnabled
sslKeystoreFile
sslKeystorePassword
tcpNodelayEnabled
userAgent
viewEndpoints
NEW OPTIONS CONT'D
http://developer.couchbase.com/
documentation/server/current/sdks/java-2.2/
env-config.html#story-h2-2
N1QL- Non 1st Normal Form Query Language

- Familiar SQL like syntax

- MapReduce Views

- Steep Learning Curve

- Limited Capabilities

- N1QL requires Couchbase Server 4.0+
ARITHMETIC + - * / % -val
COLLECTION ANY EVERY ARRAY FIRST EXISTS IN WITHIN
COMPARISON
= == != <> > >= < <=
(NOT) BETWEEN (NOT) LIKE IS (NOT) NULL
IS (NOT) MISSING IS (NOT) VALUED
CONDITIONAL CASE expression WHEN value THEN expression
CONSTRUCTION
Array [ value, value, ... ]
Object { key:value, key:value, ... }
LOGICAL AND OR NOT
STRING ||
N1QL OPERATORS
http://developer.couchbase.com/
documentation/server/current/n1ql/n1ql-
language-reference/operators.html
N1QL FUNCTIONS
ABS(EXP)
ACOS(EXP)
ASIN(EXP)
ATAN(EXP)
ATAN2(EXP1, EXP2)
CEIL(EXP)
COS(EXP)
DEGREES(EXP)
E()
EXP(EXP)
LN(EXP)
LOG(EXP)
FLOOR(EXP)
PI()
POWER(EXP1, EXP2)
RADIANS(EXP)
RANDOM([ EXP ])
ROUND(EXP [, DIGITS ])
SIGN(EXP)
SIN(EXP)
SQRT(EXP)
TAN(EXP)
TRUNC(EXP [, DIGITS ])
NUMBER FUNCTIONS
IFMISSING(EXP1, EXP2, ...)
IFMISSINGORNULL(EXP1, EXP2, ...)
IFNULL(EXP1, EXP2, ...)
MISSINGIF(EXP1, EXP2)
NULLIF(EXP1, EXP2)
CONDITIONALS - UNKNOWNS
IFINF(EXP1, EXP2, ...)
IFNAN(EXP1, EXP2, ...)
IFNANORINF(EXP1, EXP2, ...)
NANIF(EXP1, EXP2)
NEGINFIF(EXP1, EXP2)
POSINFIF(EXP1, EXP2)
CONDITIONALS - NUMBERS
ARRAY_AGG(EXP)
ARRAY_AGG(DISTINCT EXP)
AVG(EXP)
AVG(DISTINCT EXP)
COUNT(*)
COUNT(EXP)
COUNT(DISTINCT EXP)
MAX(EXP)
MIN(EXP)
SUM(EXP)
SUM(DISTINCT EXP)
AGGREGATE FUNCTIONS
OBJECT_LENGTH(EXP)
OBJECT_NAMES(EXP)
OBJECT_PAIRS(EXP)
OBJECT_VALUES(EXP)
OBJECT FUNCTIONS
GREATEST(EXP1, EXP2)
LEAST(EXP1, EXP2)
COMPARISON FUNCTIONS
DECODE_JSON(EXP)
ENCODE_JSON(EXP)
ENCODED_SIZE(EXP)
POLY_LENGTH(EXP)
JSON FUNCTIONS
BASE64(EXP)
BASE64_ENCODE(EXP)
BASE64_DECODE(EXP)
META(EXP)
UUID
META AND UUID FUNCTIONS
TYPE CHECKING FUNCTIONS
ISARRAY(EXP)
ISATOM(EXP)
ISBOOLEAN(EXP)
ISNUMBER(EXP)
ISOBJECT(EXP)
ISSTRING(EXP)
TYPE(EXP)
TYPE CONVERSION FUNCTIONS
TOARRAY(EXP)
TOATOM(EXP)
TOBOOLEAN(EXP)
TONUMBER(EXP)
TOOBJECT(EXP)
TOSTRING(EXP)
http://developer.couchbase.com/
documentation/server/4.1/n1ql/n1ql-
language-reference/functions.html
N1QL FUNCTIONS
ARRAY_APPEND(EXP, VAL)
ARRAY_AVG(EXP)
ARRAY_CONCAT(EXP1, EXP2)
ARRAY_CONTAINS(EXP, VAL)
ARRAY_COUNT(EXP)
ARRAY_DISTINCT(EXP)
ARRAY_IFNULL(EXP)
ARRAY_LENGTH(EXP)
ARRAY_MAX(EXP)
ARRAY_MIN(EXP)
ARRAY_POSITION(EXP, VAL)
ARRAY_PREPEND(VAL, EXP)
ARRAY_PUT(EXP, VAL)
ARRAY_RANGE(START, END [,STEP])
ARRAY_REMOVE(EXP, VAL)
ARRAY_REPEAT(VAL, N)
ARRAY_REPLACE(EXP, VAL1, VAL2 [,N])
ARRAY_REVERSE(EXP)
ARRAY_SORT(EXP)
ARRAY_SUM(EXP)
ARRAY FUNCTIONS
CONTAINS(EXP, SUBSTRING)
INITCAP(EXP )
TITLE(EXP)
LENGTH(EXP)
LOWER(EXP)
LTRIM(EXP [,CHARACTERS ])
POSITION(EXP, SUBSTRING)
REPEAT(EXP, N)
REPLACE(EXP, SBSTR, REPL [, N ])
RTRIM(EXP, [,CHARACTERS ])
SPLIT(EXP [, SEP ])
SUBSTR(EXP, POS[, LEN ])
TRIM(EXP [, CHARACTERS ])
UPPER(EXP)
STRING FUNCTIONS
CLOCK_MILLIS()
CLOCK_STR ([FMT ])
DATE_ADD_MILLIS(EXP, N, PART)
DATE_ADD_STR(EXP, N,PART)
DATE_DIFF_MILLIS(EXP1, EXP2, PART)
DATE_DIFF_STR(EXP1, EXP2, PART)
DATE_PART_MILLIS(EXP, PART)
DATE_PART_STR(EXP, PART)
DATE_TRUNC_MILLIS(EXP, PART)
DATE_TRUNC_STR(EXP, PART)
MILLIS(EXP)
STR_TO_MILLIS(EXP)
MILLIS_TO_STR(EXP [, FMT ])
MILLIS_TO_UTC(EXP [, FMT ])
MILLIS_TO_ZONE_NAME(EX, TZ[,FMT])
NOW_MILLIS()
NOW_STR([ FMT ])
STR_TO_MILLIS(EXP)
MILLIS(EXP)
STR_TO_UTC(EXP)
STR_TO_ZONE_NAME(EXP, TZ_NAME)
DATE FUNCTIONS
REGEXP_CONTAINS(EXP, PATTERN)
REGEXP_LIKE(EXP, PATTERN)
REGEXP_POSITION(EXP, PATTERN)
REGEXP_REPLACE(EXP, PTRN, REPL [, N ])
PATTERN MATCHING FUNCTIONS
RETRIEVE DOCUMENT
SELECT *
FROM users
USE KEYS "user_101"
SELECT BY DOCUMENT ID
couchbase.get("user_101");
STANDARD SDK OPERATION
[
{
"users": {
"email": "Judah66@gmail.com",
"first_name": "Albin",
"last_name": "Price",
"user_id": 101,
"username": "Eudora43"
}
}
]
RESULTS
GET MULTIPLE DOCUMENTS
SELECT *
FROM users
USE KEYS ["user_101","user_454"]
SELECT BY DOCUMENT IDS
couchbase.getMulti([
"user_101",
"user_454"
]);
STANDARD SDK OPERATION
[
{
"users": {
"email": "Judah66@gmail.com",
"first_name": "Albin",
"last_name": "Price",
"user_id": 101,
"username": "Eudora43"
}
},
{
"users": {
"email": "Chase.Kohler63@gmail.com",
"first_name": "Donnell",
"last_name": "Ortiz",
"user_id": 454,
"username": "Garett31"
}
}
]
RESULTS
SELECT DIFFERENCES
SELECT *
FROM users
USE KEYS "user_101"
SELECT *
[
{
"users": {
"email": "Judah66@gmail.com",
"first_name": "Albin",
"last_name": "Price",
"user_id": 101,
"username": "Eudora43"
}
}
]
RESULTS
SELECT users.*
FROM users
USE KEYS "user_101"
SELECT ENTITY.*
[
{
"email": "Judah66@gmail.com",
"first_name": "Albin",
"last_name": "Price",
"user_id": 101,
"username": "Eudora43"
}
]
RESULTS
Documents are projected as the bucket or alias
name
?
QUERY BY VALUE
SELECT u.*
FROM users AS u
WHERE u.username = 'Eudora43'
QUERY BY USERNAME
[
{
"email": "Judah66@gmail.com",
"first_name": "Albin",
"last_name": "Price",
"user_id": 197,
"username": "Eudora43"
}
]
RESULTS
STANDARD SDK OPERATION
// perform view query
results = couchbase.query(
designDocumentName="user",
viewName="username",
options={
key: "eudora43"
}
);
// get the document by the id
// returned from the query
user = couchbase.get(results[1].id);
function (doc, meta) {
if( doc.username ){
emit(
doc.username.toLowerCase(),
null
);
}
}
VIEW MAP FUNCTION
These queries are different. The N1QL query
does not take case into consideration the
View does. N1QL string operators are case
sensitive, the N1QL query would need to use
the LOWER() function
This can be done without N1QL, however it
would require a MapReduce View
This query would require a secondary
index
DML
OPERATIONS
INSERT
INSERT INTO users (KEY, VALUE)
VALUES ("user_1021", {
"user_id": 1021,
"name": "John Smith",
"email": "john.smith@email.com"
})
N1QL INSERT
couchbase.insert("user_1021", {
"user_id": 1021,
"name": "John Smith",
"email": "john.smith@email.com"
});
STANDARD SDK OPERATION
{
"results": [],
"metrics": {
"elapsedTime": "50.361685ms",
"executionTime": "50.322691ms",
"resultCount": 0,
"resultSize": 0,
"mutationCount": 1
}
}
RESULTS
RETURNING
INSERT INTO users (KEY, VALUE)
VALUES ("user_2343", {
"user_id": 2343,
"name": "John Smith",
"email": "john.smith@gmail.com"
})
RETURNING *
N1QL INSERT W/ RETURNING
// write the user document
couchbase.insert("user_2343", {
"user_id": 2343,
"name": "John Smith",
"email": "john.smith@gmail.com"
});
// get the user document
user = couchbase.get("user_2343");
STANDARD SDK OPERATION
[
{
"users": {
"email": "john.smith@gmail.com",
"name": "John Smith",
"user_id": 2343
}
}
]
RESULTS
UPDATE
// get the user document
user = couchbase.get("user_2343");
// change the email value
user.email = "elmariachi@gmail.com";
// update the entire document
couchbase.replace("user_2343", user);
// read the write to get just the
// updated email value
user = couchbase.get("user_2343");
email = user.email;
STANDARD SDK OPERATIONN1QL UPDATE
UPDATE users
USE KEYS "user_2343"
SET email =
"elmariachi@gmail.com"
RETURNING email
SDK replace() will error if the document does
not exist, N1QL won’t
[
{
"email": "elmariachi@gmail.com"
}
]
RESULTS
UPSERT
// write the user document
couchbase.upsert("user_2343", {
"user_id": 2343,
"name": "John Smith",
"email": "john.smith@gmail.com"
});
// get the user document
user = couchbase.get("user_2343");
UPSERT INTO users (KEY, VALUE)
VALUES ("user_2343", {
"user_id": 2343,
"name": "John Smith",
"email": "john.smith@gmail.com"
})
RETURNING META().id AS document_id
N1QL UPSERT STANDARD SDK OPERATION
[
{
"document_id": "user2343"
}
]
RESULTS
DELETE
DELETE
FROM users
USE KEYS "user_2343"
N1QL DELETE
// write the user document
couchbase.remove("user_2343");
STANDARD SDK OPERATION
{
"results": [],
"metrics": {
"elapsedTime": "21.591512ms",
"executionTime": "21.559566ms",
"resultCount": 0,
"resultSize": 0
}
}
RESULTS SDK remove() will error if the document does not
exist, N1QL won’t
INDEXING
VIEW VS. GSI INDEXES
GSI INDEXES VIEW INDEXES
Partitioning Model
• Part of Index Service
• Each GSI Must be Unique
• Can only be placed on one node
• Can respond without scatter-gather
• Auto-partitioned with Data Service
• Expensive scatter-gather operations
Scaling Model • Scales with Index Service • Scales with Data Service
Performance
• Singleton Lookups or Range Scans can
respond without scatter-gather
• Singleton Lookups or Range Scans can
respond without scatter-gather
Managed Cache
• Comes with Managed Cache that performs
better during Index Maintenance and Scans
• Depends on File System Cache
Storage Engine • ForestDB • Couchstore
High Availability
• Multiple indexes with identical definitions
across multiple nodes
• Built-in replicas with smart placement
INDEXES
http://developer.couchbase.com/
documentation/server/current/indexes/gsi-
for-n1ql.html
• "USE KEYS" statements do not
require GSI Indexes
• All other N1QL queries require
GSI Indexes
• PRIMARY Index contains all keys
• Ad hoc queries require a
PRIMARY Index
• Index Names must be unique
• USING GSI is the default
CREATE PRIMARY INDEX idx_users_primary
ON users
USING GSI
PRIMARY INDEX
CREATE INDEX idx_users_username
ON users( username )
USING GSI
SECONDARY INDEX
DROP PRIMARY INDEX ON idx_users_primary
ON users
USING GSI
DROP PRIMARY INDEX
DROP PRIMARY INDEX idx_users_primary
ON users
USING GSI
DROP SECONDARY INDEX
EXPLAIN
EXPLAIN
SELECT u.*
FROM users AS u
WHERE u.username = 'Eudora43'
QUERY BY USERNAME
[
{
"plan": {
"#operator": "Sequence",
"~children": [
{
"#operator": "IndexScan",
"index": "idx_users_username",
"index_id": "240a59c3f5ced906",
"keyspace": "social",
"namespace": "default",
...
],
"using": "gsi"
},
...
]
},
...
}
]
RESULTS
http://developer.couchbase.com/
documentation/server/current/n1ql/n1ql-
language-reference/explain.html
INDEX SCAN TYPES
• PRIMARYSCAN
• INDEXSCAN
• KEYSCAN
• PARENTSCAN
• VALUESCAN
• DUMMYSCAN
• COUNTSCAN
• INTERSECTSCAN
Reference https://github.com/couchbase/query#plan
for Scan explanations
INDEXES
CREATE INDEX idx_users_username
ON users( username )
WHERE username IS NOT MISSING
USING GSI
FILTERED INDEX
CREATE INDEX idx_users_username
ON users( username )
WHERE doc_type = 'user'
USING GSI
FILTERED INDEX
• Definitions can be duplicated
• Indexes can be partitioned
• Randomly assigned to an Index
nodes in the cluster
INDEXES
CREATE INDEX idx_users_username
ON users( username )
WHERE username IS NOT MISSING
USING GSI
WITH { "defer_build": true }
DEFER THE INDEX BUILD
BUILD INDEX
ON users(
idx_users_username,
idx_users_email
)
USING GSI
BUILD AN INDEX
CREATE INDEX idx_users_username
ON users( username )
WHERE username IS NOT MISSING
USING GSI
WITH { "nodes": "node3:8091"}
ADD INDEX TO SPECIFIC NODE
• Index builds can be deferred
• Multiple Indexes can be built at a
time
• Indexes can be deployed to
specific nodes
• GSI Indexes are not rebalanced
INDEX LOAD BALANCING
CREATE INDEX idx_users_username_AtoM
ON users( username )
WHERE doc_type = 'user'
AND username BETWEEN 'A' AND 'N'
USING GSI
RANGE PARTITIONED INDEXES
CREATE INDEX idx_users_username_NtoZ
ON users( username )
WHERE doc_type = 'user'
AND username BETWEEN 'N' AND 'Z'
USING GSI
- A round robin algorithm is used to distribute index load

- If a node responsible for a index goes down, a PrimaryIndexScan would be
used, or the query would fail if there is no PRIMARY INDEX
CREATE INDEX idx_users_username_AtoM_node1
ON users( username )
WHERE doc_type = 'user'
AND username BETWEEN 'A' AND 'N'
USING GSI WITH { "nodes": "node1:8091"}
REDUNDANT / REPLICATED INDEXES
CREATE INDEX idx_users_username_NtoZ_node1
ON users( username )
WHERE doc_type = 'user'
AND username BETWEEN 'N' AND 'Z'
USING GSI WITH { "nodes": "node1:8091"}
CREATE INDEX idx_users_username_AtoM_node2
ON users( username )
WHERE doc_type = 'user'
AND username BETWEEN 'A' AND 'N'
USING GSI WITH { "nodes": "node2:8091"}
CREATE INDEX idx_users_username_NtoZ_node2
ON users( username )
WHERE doc_type = 'user'
AND username BETWEEN 'N' AND 'Z'
USING GSI WITH { "nodes": "node2:8091"}
ADVANCED SELECTS
MISSING ATTRIBUTES
SELECT u.user_id, u.email, u.name
FROM users AS u
USE KEYS "user_197"
QUERY
{
"email": "Judah66@gmail.com",
"first_name": "Albin",
"last_name": "Price",
"user_id": 197,
"username": "Eudora43"
}
SAMPLE DOCUMENT
[
{
"email": "Judah66@gmail.com"
"user_id": 197
}
]
RESULT
Requested attributes that are missing are not returned as
part of the response
MISSING ATTRIBUTES
SELECT u.user_id, u.email,
IFMISSING(
u.name,
u.first_name || ' ' || u.last_name
) AS name
FROM users AS u
USE KEYS "user_197"
QUERY
{
"email": "Judah66@gmail.com",
"first_name": "Albin",
"last_name": "Price",
"user_id": 197,
"username": "Eudora43"
}
SAMPLE DOCUMENT
[
{
"email": "Judah66@gmail.com",
"name": "Albin Price",
"user_id": 197
}
]
RESULT
SELECT airlines.airline_id,
airlines.airline_name,
IFNULL(
airlines.airline_iata,
airlines.airline_icao
) AS airline_code
FROM `flight-data` AS codes
USE KEYS 'airline_code_DL'
INNER JOIN `flight-data` AS airlines
ON KEYS 'airline_' || TOSTRING( codes.id )
LIMIT 1
QUERY
[
{
"airline_code": "DL",
"airline_id": 2009,
"airline_name": "Delta Air Lines"
}
]
RESULT
{
"_id": "airline_code_DL",
"code": "DL",
"code_type": "iata",
"designation": "airline",
"doc_type": "code",
"id": 2009
}
AIRLINE LOOKUP DOCUMENT
{
"_id": "airline_2009",
"active": true,
"airline_iata": "DL",
"airline_icao": "DAL",
"airline_id": 2009,
"airline_name": "Delta Air Lines",
"callsign": "DELTA",
"doc_type": "airline",
"iso_country": "US"
}
AIRLINE DOCUMENT
JOINS
MULTIPLE JOINS
{
"_id": "route_6cea35d1-37e1-5425-913f-42c29e248285",
"airline_code": "DL",
"destination_airport_code": "MSP",
"doc_type": "route",
"route_id": "6cea35d1-37e1-5425-913f-42c29e248285",
"source_airport_code": "BWI"
}
ROUTE DOCUMENT
{
"_id": "airline_code_DL",
"code": "DL",
"code_type": "iata",
"designation": "airline",
"doc_type": "code",
"id": 2009
}
AIRLINE LOOKUP DOCUMENT
{
"_id": "airline_2009",
"airline_iata": "DL",
"airline_icao": "DAL",
"airline_id": 2009,
"airline_name": "Delta Air Lines",
"doc_type": "airline"
}
AIRLINE DOCUMENT
{
"_id": "airport_code_BWI",
"code": "BWI",
"code_type": "iata",
"designation": "airport",
"doc_type": "code",
"id": 3435
}
AIRPORT LOOKUP DOCUMENT
{
"_id": "airport_3435",
"airport_iata": "BWI",
"airport_icao": "KBWI",
"airport_id": 3435,
"airport_ident": "KBWI",
"airport_name": "Baltimore Washington Intl",
"airport_type": "large_airport",
"doc_type": "airport",
"elevation": 146,
"geo": {
"latitude": 39.17539978,
"longitude": -76.66829681
},
"iso_continent": "NA",
"iso_country": "US",
"iso_region": "US-MD",
"municipality": "Baltimore"
}
AIRPORT DOCUMENT
SELECT airlines.airline_name,
IFNULL(
airlines.airline_iata,
airlines.airline_icao
) AS airline_code,
source_airports.airport_name,
source_airports.iso_country,
source_airports.iso_region,
IFNULL(
source_airports.airport_iata,
source_airports.airport_icao,
source_airports.airport_ident
) AS airport_code
FROM `flight-data` AS routes
/* get airline details */
INNER JOIN `flight-data` AS airline_codes
ON KEYS 'airline_code_' || routes.airline_code
INNER JOIN `flight-data` AS airlines
ON KEYS 'airline_' || TOSTRING( airline_codes.id )
/* get airport details */
INNER JOIN `flight-data` AS airport_codes
ON KEYS 'airport_code_' || routes.source_airport_code
INNER JOIN `flight-data` AS source_airports
ON KEYS 'airport_' || TOSTRING( airport_codes.id )
WHERE routes.destination_airport_code = 'MSP'
AND routes.source_airport_code IS NOT NULL
AND routes.doc_type = 'route'
AND routes.active = true
ORDER BY source_airports.airport_name ASC,
airlines.airline_name ASC
QUERY
[
{
"airline_code": "DL",
"airline_name": "Delta Air Lines",
"airport_code": "ABR",
"airport_name": "Aberdeen Regional Airport",
"iso_country": "US",
"iso_region": "US-SD"
},
{
"airline_code": "DL",
"airline_name": "Delta Air Lines",
"airport_code": "ABQ",
"airport_name": "Albuquerque International Sunport",
"iso_country": "US",
"iso_region": "US-NM"
},
{
"airline_code": "DL",
"airline_name": "Delta Air Lines",
"airport_code": "BWI",
"airport_name": "Baltimore Washington Intl",
"iso_country": "US",
"iso_region": "US-MD"
},
...
]
RESULT
MULTIPLE JOINS
ANY
ANY variable ( IN | WITHIN ) expression
[ , variable ( IN | WITHIN ) expression ]*
SATISFIES condition END
EVERY
EVERY variable ( IN | WITHIN ) expression
[ , variable ( IN | WITHIN ) expression ]*
SATISFIES condition END
ARRAY
ARRAY expression FOR variable ( IN | WITHIN ) expression
[ , variable ( IN | WITHIN ) expression ]* [ ( WHEN condition) ] END
FIRST
FIRST expression FOR variable ( IN | WITHIN )
expression [ , variable ( IN | WITHIN ) expression]*
[ ( WHEN condition) ] END
EXISTS EXISTS expression
IN expression [ NOT ] IN expression
WITHIN expression [NOT] WITHIN expression
COLLECTION OPERATORS
http://developer.couchbase.com/
documentation/server/current/n1ql/
n1ql-language-reference/
collectionops.html
ANY OPERATOR
SELECT u.name, u.children
FROM users AS u
WHERE u.doc_type = 'user'
AND ANY child IN u.children
SATISFIES child.age > 10
END
ORDER BY ARRAY_LENGTH(u.children) ASC
QUERY
[
{
"children": [
{
"age": 12,
"first_name": "Emelia",
"gender": "M"
}
],
"name": "Elyse Rempel"
},
{
"children": [
{
"age": 15,
"first_name": "Yvette",
"gender": "F"
},
{
"age": 8,
"first_name": "Rodolfo",
"gender": "M"
}
],
"name": "Reynold Mohr"
},
...
]
RESULT
EVERY OPERATOR
SELECT u.name, u.children
FROM users AS u
WHERE u.doc_type = 'user'
AND EVERY child IN u.children
SATISFIES child.age > 10
AND child.gender = 'M'
END
ORDER BY ARRAY_LENGTH(u.children) ASC
QUERY
[
{
"children": [
{
"age": 16,
"first_name": "Grant",
"gender": "M"
}
],
"name": "Joseph Stiedemann"
},
{
"children": [
{
"age": 11,
"first_name": "Ezekiel",
"gender": "M"
},
{
"age": 12,
"first_name": "Brooks",
"gender": "M"
}
],
"name": "Aiden Christiansen"
},
...
]
RESULT
ARRAY INDEXING
[
{
"chat_id": "61bf583b2cd0",
"created_on": "2016-05-29T13:31:29.759Z"
},
{
"chat_id": "d63a56047e08",
"created_on": "2016-05-29T10:07:18.824Z"
},
...
]
RESULT
SELECT chats.chat_id,
MILLIS_TO_STR(chats.created_on) AS
created_on
FROM social AS chats
WHERE ANY user_id IN chats.users
SATISFIES user_id = 100
END
AND chats.doc_type = 'chat'
ORDER BY chats.created_on DESC
QUERY
CREATE INDEX idx_chats_users ON social(
DISTINCT ARRAY user_id
FOR user_id IN users
WHEN user_id IS NOT NULL
END,
doc_type
)
WHERE doc_type = 'chat'
INDEX
{
"_id": "chat_4ff8dd056225",
"doc_type": "chat",
"chat_id": "4ff8dd056225",
"created_on": 1464518742983,
"users": [
861,
492
]
}
SAMPLE DOCUMENT
{
"_id": "user_764",
"doc_type": "user",
"user_id": 764,
"first_name": "Geovanny",
"last_name": "Parker",
"phones": [
{
"type": "Mobile",
"phone_number": "676.825.8926",
"extension": null
},
{
"type": "Mobile",
"phone_number": "792.877.3144",
"extension": "3644"
},
{
"type": "Home",
"phone_number": "(730) 490-6734",
"extension": null
}
]
}
SAMPLE DOCUMENT
SELECT u.phones
FROM users AS u
USE KEYS 'user_764'
QUERY
[
{
"phones": [
{
"type": "Mobile",
"phone_number": "676.825.8926",
"extension": null
},
{
"type": "Mobile",
"phone_number": "792.877.3144",
"extension": "3644"
},
{
"type": "Home",
"phone_number": "(730) 490-6734",
"extension": null
}
]
}
]
RESULT
UNNEST
This query will only provide a single result
with a nested "phones" attribute that is an
array
UNNEST
{
"_id": "user_764",
"doc_type": "user",
"user_id": 764,
"first_name": "Geovanny",
"last_name": "Parker",
"phones": [
{
"type": "Mobile",
"phone_number": "676.825.8926",
"extension": null
},
{
"type": "Mobile",
"phone_number": "792.877.3144",
"extension": "3644"
},
{
"type": "Home",
"phone_number": "(730) 490-6734",
"extension": null
}
]
}
SAMPLE DOCUMENT
SELECT phone_numbers.*
FROM users AS u
USE KEYS 'user_23'
UNNEST u.phones AS phone_numbers
QUERY
[
{
"type": "Mobile",
"phone_number": "676.825.8926",
"extension": null
},
{
"type": "Mobile",
"phone_number": "792.877.3144",
"extension": "3644"
},
{
"type": "Home",
"phone_number": "(730) 490-6734",
"extension": null
}
]
RESULT
UNNSET performs a JOIN on a
documents attribute that is an array
with its parent document
NEST
SELECT u.first_name, u.last_name,
user_phones
FROM users AS u
USE KEYS 'user_581'
INNER NEST users AS user_phones
ON KEYS 'user_' || TOSTRING( u.user_id ) || '_phones'
QUERY
[
{
"first_name": "Geovanny",
"last_name": "Parker",
"user_phones": [
{
"_id": "user_581_phones",
"doc_type": "user-phones",
"phones": [
{
"extension": null,
"phone_number": "872-201-8963",
"phone_type": "Mobile"
},
{
"extension": "9324",
"phone_number": "720.194.5604",
"phone_type": "Other"
}
],
"user_id": 581
}
]
}
]
RESULT
{
"_id": "user_581_phones",
"doc_type": "user-phones",
"user_id": 581,
"phones": [
{
"extension": null,
"phone_number": "872-201-8963",
"phone_type": "Mobile"
},
{
"extension": "9324",
"phone_number": "720.194.5604",
"phone_type": "Other"
}
]
}
{
"_id": "user_581",
"doc_type": "user",
"user_id": 581,
"first_name": "Geovanny",
"last_name": "Parker"
}
USER DOCUMENT
USER PHONES DOCUMENT
A NEST operation will take 0 or more
documents and nest them as a single entry in
the result

INNER NEST will omit records if the right side
is missing

LEFT NEST will return records whether or not
the right side was found or not
NEST
SELECT u.first_name, u.last_name,
user_phones[0].phones
FROM users AS u
USE KEYS 'user_581'
INNER NEST users AS user_phones
ON KEYS 'user_' ||
TOSTRING( u.user_id ) || '_phones'
QUERY
[
{
"first_name": "Geovanny",
"last_name": "Parker",
"phones": [
{
"extension": null,
"phone_number": "872-201-8963",
"phone_type": "Mobile"
},
{
"extension": "9324",
"phone_number": "720.194.5604",
"phone_type": "Other"
}
]
}
]
RESULT
{
"_id": "user_581_phones",
"doc_type": "user-phones",
"user_id": 581,
"phones": [
{
"extension": null,
"phone_number": "872-201-8963",
"phone_type": "Mobile"
},
{
"extension": "9324",
"phone_number": "720.194.5604",
"phone_type": "Other"
}
]
}
{
"_id": "user_581",
"doc_type": "user",
"user_id": 581,
"first_name": "Geovanny",
"last_name": "Parker"
}
USER DOCUMENT
USER PHONES DOCUMENT
NEST
SELECT u.user_id, u.first_name, u.last_name,
ARRAY
{
"phone_number": phone.phone_number,
"phone_type": phone.phone_type,
"extension": phone.extension
}
FOR phone IN IFMISSING(phones, [])
END AS phones
FROM users AS u
USE KEYS 'user_581'
LEFT NEST users AS phones ON KEYS (
ARRAY phone.phone_id FOR phone IN (
SELECT 'phone_' || phone_id AS phone_id
FROM users AS phone_lookup
USE KEYS
'user_' || TOSTRING(u.user_id) || '_phones'
UNNEST phone_lookup.phones AS phone_id
) END
)
QUERY
{
"_id": "user_581_phones",
"doc_type": "user-phones",
"user_id": 581,
"phones": [
"886350f7-3f97-5405-b6d0-294ecf469f05",
"8ee1bd2e-7b88-5d2c-a211-05a61ac367f8"
]
}
{
"_id": "user_581",
"doc_type": "user",
"user_id": 581,
"first_name": "Geovanny",
"last_name": "Parker"
}
USER DOCUMENT
USER PHONES DOCUMENT
{
"_id": "phone_886350f7-3f97-5405-b6d0-294ecf469f05",
"doc_type": "phone",
"phone_id": "886350f7-3f97-5405-b6d0-294ecf469f05",
"user_id": 581,
"phone_type": "Other",
"phone_number": "872-201-8963",
"extension": null
}
USER PHONES DOCUMENT
generates the same exact results as the
previous slide
AGGREGATES
SELECT COUNT(*) AS total_users
FROM social AS users
WHERE users.doc_type = 'user'
QUERY
[
{
"total_users": 1000
}
]
RESULT
SELECT ARRAY_COUNT(users.children) AS children
FROM social AS users
USE KEYS 'user_132'
QUERY
[
{
"children": 3
}
]
RESULT
SELECT MIN(children.age) AS min_age,
MAX(children.age) AS max_age,
AVG(children.age) AS avg_age
FROM social AS users
UNNEST users.children AS children
WHERE users.doc_type = 'user'
QUERY
[
{
"avg_age": 8.94,
"max_age": 17,
"min_age": 1
}
]
RESULT
http://developer.couchbase.com/
documentation/server/current/n1ql/n1ql-
language-reference/aggregatefun.html
CF EXAMPLES
POSITIONAL PARAMS
couchbase.n1qlQuery(
statement="
SELECT airport_id,
airport_name, airport_type,
municipality, geo, timezone,
airport_iata, airport_icao,
FROM `flight-data`
WHERE iso_country = $1
AND iso_region = $2
AND doc_type = $3
ORDER BY airport_name ASC
LIMIT 2
",
parameters=[
"US",
"US-ME",
"airport"
]
);
SDK QUERY RESULT
NAMED PARAMS
couchbase.n1qlQuery(
statement="
SELECT airport_id,
airport_name, airport_type,
municipality, geo, timezone,
airport_iata, airport_icao,
FROM `flight-data`
WHERE iso_country = $country
AND iso_region = $region
AND doc_type = $doc_type
ORDER BY airport_name ASC
LIMIT 2
",
parameters={
'country': "US",
'region': "US-VT",
'doc_type': "airport"
}
);
SDK QUERY RESULT
PREPARED STATEMENTS
couchbase.n1qlQuery("
PREPARE
SELECT COUNT(1) AS total_airports
FROM `flight-data`
WHERE iso_country = 'US'
AND iso_region IS NOT NULL
AND doc_type = 'airport'
");
PREPARE QUERY PREPARE RESULT
couchbase.n1qlQuery("
EXECUTE
'9e4e1d3d-f1c3-4292-81cb-824af1fae87e'
");
EXECUTE QUERY
EXECUTE RESULT
Prepared Statements are stored in memory
until a server restart
NAMED PREPARED STATEMENTS
couchbase.n1qlQuery("
PREPARE us_airports
FROM
SELECT COUNT(1) AS total_airports
FROM `flight-data`
WHERE iso_country = 'US'
AND iso_region IS NOT NULL
AND doc_type = 'airport'
");
PREPARE QUERY RESULT
couchbase.n1qlQuery("
EXECUTE 'us_airports'
");
EXECUTE QUERY RESULT
DOCUMENT LOCKING
user = couchbase.getAndLock("user_101");
PREPARE QUERY
RESULT
• Default lock time is 5 seconds
• Max lock time is 30 seconds
• Prevents retrieval while Locked
• Prevents updates while Locked
• An update requires CAS
UNLOCKING A DOCUMENT
// replace the document using CAS
couchbase.replaceWithCAS(
id="user_101",
value=doc,
cas=user.cas
);
REPLACING THE DOCUMENT W/ CAS
RESULT
// update the document
couchbase.upsert("user_101", doc);
// replace the document
couchbase.replace("user_101", doc);
WHILE LOCKED UPSERT AND REPLACE WILL FAIL
ERROR
// unlock the document, returns true or
// false if the unlock was successful
couchbase.unlock(
id="user_101",
cas=user.cas
);
USING UNLOCK
Documents will automatically unlock after the
"lock time" has been exceeded
REPLICA READS
try {
// retrieve and lock the document
result = couchbase.get("user_100");
} catch(any e){
// attempt to get the document from a replica
result = couchbase.getFromReplica("user_100");
}
// replica reads return an array for each
// configured replica in the bucket
if (isArray(result)) {
result = result[0];
}
GET DOCUMENT FROM REPLICA(S)
COUNTERS
// increment the user counter
next_id = couchbase.counter(
id="user_counter",
value=1,
defaultValue=0
);
couchbase.insert("user_" & next_id, {
"user_id": next_id,
"name": "John Smith",
"email": "john.smith@email.com"
});
INCREMENT VALUE
// decrement the tickets counter
available_tix = couchbase.counter(
id="tickets_counter",
value=-1,
defaultValue=1000
);
// if there are no more tickets
// available redirect the user
if (available_tix < 1) {
cflocation(url="sold-out.cfm",
addtoken=false);
}
...
DECREMENT VALUE
Counter operations are Atomic
CF TRAVEL SAMPLE
bit.ly/cfcouchbase-travel
• AIRLINES
• AIRPORTS
• HOTELS
• LANDMARKS
• ROUTES
N1QL EXAMPLES
FLIGHT DATA
bit.ly/flight-data-n1ql
• AIRLINE REVIEWS
• AIRLINES
• AIRPORT AIRLINES
• AIRPORT FREQUENCIES
• AIRPORT NAVAIDS
• AIRPORT REVIEWS
• AIRPORT RUNWAYS
• AIRPORTS
• CODES
• CONTINENTS
• COUNTRIES
• FREQUENCIES
• NAVAIDS
• REGIONS
• ROUTES
• RUNWAYS
• USERS
SOCIAL DATA
bit.ly/social-data-n1ql
• ADDRESSES
• AUTH
• CHAT MESSAGES
• CHATS
• EMAILS
• FRIENDS
• LIKES
• PHONES
• POSTS
• USER ADDRESSES
• USER EMAILS
• USER PHONES
• USERS
ROADMAP
• FULL TEXT SEARCHING (COMING IN CB 4.5)
• SUBDOC SUPPORT
• REMOVAL OF DEPRECATED METHODS
• SDK ALIGNMENT
• REGULAR RELEASE CYCLE
• MORE EXAMPLES
DOWNLOAD NOW
www.ortussolutions.com/products/cfcouchbase
bit.ly/cfcouchbase
OR
box install cfcouchbase
OR
CONTRIBUTE
github.com/Ortus-Solutions/cfcouchbase-sdk
bit.ly/cfcouchbasesdk
OR
DRAWING
SELECT drawing.attendees[
FLOOR(
RANDOM() *
ARRAY_LENGTH(drawing.attendees)
)
] AS winner
FROM `default` AS drawing
USE KEYS 'drawing'
QUESTIONS?
RESOURCES
bit.ly/n1ql-41-language-reference
bit.ly/node-fakeit
bit.ly/cfcouchbase-travel bit.ly/cfcouchbase-samples
blog.couchbase.com
developer.couchbase.com query.pub.couchbase.com
thepolyglotdeveloper.com
ortussolutions.com/products/cfcouchbase
ortussolutions.com/blog/category/cfcouchbase

More Related Content

What's hot

Node 관계형 데이터베이스_바인딩
Node 관계형 데이터베이스_바인딩Node 관계형 데이터베이스_바인딩
Node 관계형 데이터베이스_바인딩
HyeonSeok Choi
 
Intravert Server side processing for Cassandra
Intravert Server side processing for CassandraIntravert Server side processing for Cassandra
Intravert Server side processing for Cassandra
Edward Capriolo
 
Introduction to Apache Spark / PUT 06.2014
Introduction to Apache Spark / PUT 06.2014Introduction to Apache Spark / PUT 06.2014
Introduction to Apache Spark / PUT 06.2014
bbogacki
 

What's hot (20)

Rxjs swetugg
Rxjs swetuggRxjs swetugg
Rxjs swetugg
 
Apache Cassandra & Data Modeling
Apache Cassandra & Data ModelingApache Cassandra & Data Modeling
Apache Cassandra & Data Modeling
 
Noinject
NoinjectNoinject
Noinject
 
2 презентация rx java+android
2 презентация rx java+android2 презентация rx java+android
2 презентация rx java+android
 
COScheduler In Depth
COScheduler In DepthCOScheduler In Depth
COScheduler In Depth
 
Node 관계형 데이터베이스_바인딩
Node 관계형 데이터베이스_바인딩Node 관계형 데이터베이스_바인딩
Node 관계형 데이터베이스_바인딩
 
The Ring programming language version 1.3 book - Part 7 of 88
The Ring programming language version 1.3 book - Part 7 of 88The Ring programming language version 1.3 book - Part 7 of 88
The Ring programming language version 1.3 book - Part 7 of 88
 
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...
 
Dun ddd
Dun dddDun ddd
Dun ddd
 
Compose Async with RxJS
Compose Async with RxJSCompose Async with RxJS
Compose Async with RxJS
 
Diving into MySQL 5.7: advanced features
Diving into MySQL 5.7: advanced featuresDiving into MySQL 5.7: advanced features
Diving into MySQL 5.7: advanced features
 
Bangun datar dan bangun ruang
Bangun datar dan bangun ruangBangun datar dan bangun ruang
Bangun datar dan bangun ruang
 
Intravert Server side processing for Cassandra
Intravert Server side processing for CassandraIntravert Server side processing for Cassandra
Intravert Server side processing for Cassandra
 
Introduction to Apache Spark / PUT 06.2014
Introduction to Apache Spark / PUT 06.2014Introduction to Apache Spark / PUT 06.2014
Introduction to Apache Spark / PUT 06.2014
 
Flexviews materialized views for my sql
Flexviews materialized views for my sqlFlexviews materialized views for my sql
Flexviews materialized views for my sql
 
Apache Spark - Basics of RDD & RDD Operations | Big Data Hadoop Spark Tutoria...
Apache Spark - Basics of RDD & RDD Operations | Big Data Hadoop Spark Tutoria...Apache Spark - Basics of RDD & RDD Operations | Big Data Hadoop Spark Tutoria...
Apache Spark - Basics of RDD & RDD Operations | Big Data Hadoop Spark Tutoria...
 
Oop assignment 02
Oop assignment 02Oop assignment 02
Oop assignment 02
 
Apache Cassandra Lesson: Data Modelling and CQL3
Apache Cassandra Lesson: Data Modelling and CQL3Apache Cassandra Lesson: Data Modelling and CQL3
Apache Cassandra Lesson: Data Modelling and CQL3
 
Py spark cheat sheet by cheatsheetmaker.com
Py spark cheat sheet by cheatsheetmaker.comPy spark cheat sheet by cheatsheetmaker.com
Py spark cheat sheet by cheatsheetmaker.com
 
Dagger & rxjava & retrofit
Dagger & rxjava & retrofitDagger & rxjava & retrofit
Dagger & rxjava & retrofit
 

Viewers also liked

Viewers also liked (6)

Zephyr™ Digital Airflow Sensors
Zephyr™ Digital Airflow SensorsZephyr™ Digital Airflow Sensors
Zephyr™ Digital Airflow Sensors
 
AIX Guide
AIX GuideAIX Guide
AIX Guide
 
[Did you know] 모바일 전략과 웹앱 20130703
[Did you know] 모바일 전략과 웹앱  20130703[Did you know] 모바일 전략과 웹앱  20130703
[Did you know] 모바일 전략과 웹앱 20130703
 
Art and Science of Web Sites Performance: A Front-end Approach
Art and Science of Web Sites Performance: A Front-end ApproachArt and Science of Web Sites Performance: A Front-end Approach
Art and Science of Web Sites Performance: A Front-end Approach
 
Processo de Startup do Linux
Processo de Startup do LinuxProcesso de Startup do Linux
Processo de Startup do Linux
 
Eclipse Memory Analyzer MAJUG November 2008
Eclipse Memory Analyzer MAJUG November 2008Eclipse Memory Analyzer MAJUG November 2008
Eclipse Memory Analyzer MAJUG November 2008
 

Similar to CFCouchbase 2.0 and N1QL

The Art Of Readable Code
The Art Of Readable CodeThe Art Of Readable Code
The Art Of Readable Code
Baidu, Inc.
 
1403 app dev series - session 5 - analytics
1403   app dev series - session 5 - analytics1403   app dev series - session 5 - analytics
1403 app dev series - session 5 - analytics
MongoDB
 

Similar to CFCouchbase 2.0 and N1QL (20)

Die Neuheiten in MariaDB 10.2 und MaxScale 2.1
Die Neuheiten in MariaDB 10.2 und MaxScale 2.1Die Neuheiten in MariaDB 10.2 und MaxScale 2.1
Die Neuheiten in MariaDB 10.2 und MaxScale 2.1
 
03 2017Emea_RoadshowMilan-WhatsNew-Mariadbserver10_2andmaxscale 2_1
03 2017Emea_RoadshowMilan-WhatsNew-Mariadbserver10_2andmaxscale 2_103 2017Emea_RoadshowMilan-WhatsNew-Mariadbserver10_2andmaxscale 2_1
03 2017Emea_RoadshowMilan-WhatsNew-Mariadbserver10_2andmaxscale 2_1
 
What’s New in MariaDB Server 10.2
What’s New in MariaDB Server 10.2What’s New in MariaDB Server 10.2
What’s New in MariaDB Server 10.2
 
How Angular2 Can Improve Your AngularJS Apps Today!
How Angular2 Can Improve Your AngularJS Apps Today!How Angular2 Can Improve Your AngularJS Apps Today!
How Angular2 Can Improve Your AngularJS Apps Today!
 
Angularjs Test Driven Development (TDD)
Angularjs Test Driven Development (TDD)Angularjs Test Driven Development (TDD)
Angularjs Test Driven Development (TDD)
 
Fast querying indexing for performance (4)
Fast querying   indexing for performance (4)Fast querying   indexing for performance (4)
Fast querying indexing for performance (4)
 
The Art Of Readable Code
The Art Of Readable CodeThe Art Of Readable Code
The Art Of Readable Code
 
Test-Driven Development of AngularJS Applications
Test-Driven Development of AngularJS ApplicationsTest-Driven Development of AngularJS Applications
Test-Driven Development of AngularJS Applications
 
AngularJS Testing Strategies
AngularJS Testing StrategiesAngularJS Testing Strategies
AngularJS Testing Strategies
 
Going Native: Leveraging the New JSON Native Datatype in Oracle 21c
Going Native: Leveraging the New JSON Native Datatype in Oracle 21cGoing Native: Leveraging the New JSON Native Datatype in Oracle 21c
Going Native: Leveraging the New JSON Native Datatype in Oracle 21c
 
Angular 2 - Core Concepts
Angular 2 - Core ConceptsAngular 2 - Core Concepts
Angular 2 - Core Concepts
 
Webinar: Applikationsentwicklung mit MongoDB : Teil 5: Reporting & Aggregation
Webinar: Applikationsentwicklung mit MongoDB: Teil 5: Reporting & AggregationWebinar: Applikationsentwicklung mit MongoDB: Teil 5: Reporting & Aggregation
Webinar: Applikationsentwicklung mit MongoDB : Teil 5: Reporting & Aggregation
 
Emerging Languages: A Tour of the Horizon
Emerging Languages: A Tour of the HorizonEmerging Languages: A Tour of the Horizon
Emerging Languages: A Tour of the Horizon
 
Angular 2: core concepts
Angular 2: core conceptsAngular 2: core concepts
Angular 2: core concepts
 
Couchbas for dummies
Couchbas for dummiesCouchbas for dummies
Couchbas for dummies
 
Analytics Metrics delivery and ML Feature visualization: Evolution of Data Pl...
Analytics Metrics delivery and ML Feature visualization: Evolution of Data Pl...Analytics Metrics delivery and ML Feature visualization: Evolution of Data Pl...
Analytics Metrics delivery and ML Feature visualization: Evolution of Data Pl...
 
1403 app dev series - session 5 - analytics
1403   app dev series - session 5 - analytics1403   app dev series - session 5 - analytics
1403 app dev series - session 5 - analytics
 
Introduction to-mongo db-execution-plan-optimizer-final
Introduction to-mongo db-execution-plan-optimizer-finalIntroduction to-mongo db-execution-plan-optimizer-final
Introduction to-mongo db-execution-plan-optimizer-final
 
Introduction to Mongodb execution plan and optimizer
Introduction to Mongodb execution plan and optimizerIntroduction to Mongodb execution plan and optimizer
Introduction to Mongodb execution plan and optimizer
 
Angular2 for Beginners
Angular2 for BeginnersAngular2 for Beginners
Angular2 for Beginners
 

Recently uploaded

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 

Recently uploaded (20)

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 

CFCouchbase 2.0 and N1QL

  • 1. CFCOUCHBASE 2.0 INTRODUCING AARON BENTON @bentonam BRAD WOOD @bdw429s@ORTUSSOLUTIONS
  • 2. • HISTORY • CF EXAMPLES • N1QL EXAMPLES • SDK CHANGES • SDK ROADMAP • LOAD DEMO
  • 4. RELEASE 1.0 1.0 was released Feb. 2014
  • 5. RELEASE 1.1 1.1 was released Jan. 2015
  • 6.
  • 7. RELEASE 2.0 2.0 was released June 2016
  • 8. RELEASE 2.0 BRINGS • JAVA SDK (2.2.5) • N1QL SUPPORT W/ GSI INDEXES • REPLICA READS • DOCUMENT LOCKING • PREPARED STATEMENTS • DESIGN DOCUMENT MANAGEMENT • EXPANDED CONFIG • AND MORE…
  • 9. METHOD HOUSEKEEPING asyncCounter asyncDecr asyncGet asyncGetAndTouch asyncGetMulti asyncGetWithCAS asyncIncr asyncSaveView getDocStats UNSUPPORTED METHODS add decr incr delete newQuery deleteDesignDocument set setMulti setWithCAS DEPRECATED METHODS NEW METHODS counter exists getAndLock getEnvironment getFromReplica getIndexes getUtility insert invalidateQueryCache n1qlQuery publishDesignDocument remove removeDesignDocument replaceWithCAS unlock upset upsertMulti viewQuery
  • 10. METHOD HOUSEKEEPING 1.* METHODS 2.* METHODS add insert incr decr counter delete remove newQuery n1qlQuery viewQuery deleteDesignDocument removeDesignDocument set upsert replace setMulti upsertMulti setWithCAS replaceWithCAS The query() method will still be implemented and is a facade for both n1ql and view queries
  • 11. CONFIG HOUSEKEEPING bucketName dataMarshaller defaultTimeout password servers useClassLoader viewTimeout SUPPORTED OPTIONS maxReconnectDelay obsPollInterval obsPollMax opQueueMaxBlockTime opTimeout readBufferSize shouldOptimize timeoutExceptionThreshold UNSUPPORTED OPTIONS autoReleaseAfter bootstrapCarrierDirectPort bootstrapCarrierEnabled bootstrapCarrierSslPort bootstrapHttpDirectPort bootstrapHttpEnabled bootstrapHttpSslPort bufferPoolingEnabled caseSensitiveKeys computationPoolSize connectTimeout dcpEnabled defaultMetricsLoggingConsumer disconnectTimeout dnsSrvEnabled eventBus ioPool ioPoolSize keepAliveInterval kvEndpoints kvTimeout managementTimeout NEW OPTIONS maxRequestLifetime mutationTokensEnabled networkLatencyMetricsCollectorConfig observeIntervalDelay packageNameAndVersion queryEnabled queryEndpoints queryPort queryTimeout reconnectDelay requestBufferSize responseBufferSize retryDelay retryStrategy runtimeMetricsCollectorConfig scheduler sslEnabled sslKeystoreFile sslKeystorePassword tcpNodelayEnabled userAgent viewEndpoints NEW OPTIONS CONT'D http://developer.couchbase.com/ documentation/server/current/sdks/java-2.2/ env-config.html#story-h2-2
  • 12. N1QL- Non 1st Normal Form Query Language - Familiar SQL like syntax - MapReduce Views - Steep Learning Curve - Limited Capabilities - N1QL requires Couchbase Server 4.0+
  • 13. ARITHMETIC + - * / % -val COLLECTION ANY EVERY ARRAY FIRST EXISTS IN WITHIN COMPARISON = == != <> > >= < <= (NOT) BETWEEN (NOT) LIKE IS (NOT) NULL IS (NOT) MISSING IS (NOT) VALUED CONDITIONAL CASE expression WHEN value THEN expression CONSTRUCTION Array [ value, value, ... ] Object { key:value, key:value, ... } LOGICAL AND OR NOT STRING || N1QL OPERATORS http://developer.couchbase.com/ documentation/server/current/n1ql/n1ql- language-reference/operators.html
  • 14. N1QL FUNCTIONS ABS(EXP) ACOS(EXP) ASIN(EXP) ATAN(EXP) ATAN2(EXP1, EXP2) CEIL(EXP) COS(EXP) DEGREES(EXP) E() EXP(EXP) LN(EXP) LOG(EXP) FLOOR(EXP) PI() POWER(EXP1, EXP2) RADIANS(EXP) RANDOM([ EXP ]) ROUND(EXP [, DIGITS ]) SIGN(EXP) SIN(EXP) SQRT(EXP) TAN(EXP) TRUNC(EXP [, DIGITS ]) NUMBER FUNCTIONS IFMISSING(EXP1, EXP2, ...) IFMISSINGORNULL(EXP1, EXP2, ...) IFNULL(EXP1, EXP2, ...) MISSINGIF(EXP1, EXP2) NULLIF(EXP1, EXP2) CONDITIONALS - UNKNOWNS IFINF(EXP1, EXP2, ...) IFNAN(EXP1, EXP2, ...) IFNANORINF(EXP1, EXP2, ...) NANIF(EXP1, EXP2) NEGINFIF(EXP1, EXP2) POSINFIF(EXP1, EXP2) CONDITIONALS - NUMBERS ARRAY_AGG(EXP) ARRAY_AGG(DISTINCT EXP) AVG(EXP) AVG(DISTINCT EXP) COUNT(*) COUNT(EXP) COUNT(DISTINCT EXP) MAX(EXP) MIN(EXP) SUM(EXP) SUM(DISTINCT EXP) AGGREGATE FUNCTIONS OBJECT_LENGTH(EXP) OBJECT_NAMES(EXP) OBJECT_PAIRS(EXP) OBJECT_VALUES(EXP) OBJECT FUNCTIONS GREATEST(EXP1, EXP2) LEAST(EXP1, EXP2) COMPARISON FUNCTIONS DECODE_JSON(EXP) ENCODE_JSON(EXP) ENCODED_SIZE(EXP) POLY_LENGTH(EXP) JSON FUNCTIONS BASE64(EXP) BASE64_ENCODE(EXP) BASE64_DECODE(EXP) META(EXP) UUID META AND UUID FUNCTIONS TYPE CHECKING FUNCTIONS ISARRAY(EXP) ISATOM(EXP) ISBOOLEAN(EXP) ISNUMBER(EXP) ISOBJECT(EXP) ISSTRING(EXP) TYPE(EXP) TYPE CONVERSION FUNCTIONS TOARRAY(EXP) TOATOM(EXP) TOBOOLEAN(EXP) TONUMBER(EXP) TOOBJECT(EXP) TOSTRING(EXP) http://developer.couchbase.com/ documentation/server/4.1/n1ql/n1ql- language-reference/functions.html
  • 15. N1QL FUNCTIONS ARRAY_APPEND(EXP, VAL) ARRAY_AVG(EXP) ARRAY_CONCAT(EXP1, EXP2) ARRAY_CONTAINS(EXP, VAL) ARRAY_COUNT(EXP) ARRAY_DISTINCT(EXP) ARRAY_IFNULL(EXP) ARRAY_LENGTH(EXP) ARRAY_MAX(EXP) ARRAY_MIN(EXP) ARRAY_POSITION(EXP, VAL) ARRAY_PREPEND(VAL, EXP) ARRAY_PUT(EXP, VAL) ARRAY_RANGE(START, END [,STEP]) ARRAY_REMOVE(EXP, VAL) ARRAY_REPEAT(VAL, N) ARRAY_REPLACE(EXP, VAL1, VAL2 [,N]) ARRAY_REVERSE(EXP) ARRAY_SORT(EXP) ARRAY_SUM(EXP) ARRAY FUNCTIONS CONTAINS(EXP, SUBSTRING) INITCAP(EXP ) TITLE(EXP) LENGTH(EXP) LOWER(EXP) LTRIM(EXP [,CHARACTERS ]) POSITION(EXP, SUBSTRING) REPEAT(EXP, N) REPLACE(EXP, SBSTR, REPL [, N ]) RTRIM(EXP, [,CHARACTERS ]) SPLIT(EXP [, SEP ]) SUBSTR(EXP, POS[, LEN ]) TRIM(EXP [, CHARACTERS ]) UPPER(EXP) STRING FUNCTIONS CLOCK_MILLIS() CLOCK_STR ([FMT ]) DATE_ADD_MILLIS(EXP, N, PART) DATE_ADD_STR(EXP, N,PART) DATE_DIFF_MILLIS(EXP1, EXP2, PART) DATE_DIFF_STR(EXP1, EXP2, PART) DATE_PART_MILLIS(EXP, PART) DATE_PART_STR(EXP, PART) DATE_TRUNC_MILLIS(EXP, PART) DATE_TRUNC_STR(EXP, PART) MILLIS(EXP) STR_TO_MILLIS(EXP) MILLIS_TO_STR(EXP [, FMT ]) MILLIS_TO_UTC(EXP [, FMT ]) MILLIS_TO_ZONE_NAME(EX, TZ[,FMT]) NOW_MILLIS() NOW_STR([ FMT ]) STR_TO_MILLIS(EXP) MILLIS(EXP) STR_TO_UTC(EXP) STR_TO_ZONE_NAME(EXP, TZ_NAME) DATE FUNCTIONS REGEXP_CONTAINS(EXP, PATTERN) REGEXP_LIKE(EXP, PATTERN) REGEXP_POSITION(EXP, PATTERN) REGEXP_REPLACE(EXP, PTRN, REPL [, N ]) PATTERN MATCHING FUNCTIONS
  • 16. RETRIEVE DOCUMENT SELECT * FROM users USE KEYS "user_101" SELECT BY DOCUMENT ID couchbase.get("user_101"); STANDARD SDK OPERATION [ { "users": { "email": "Judah66@gmail.com", "first_name": "Albin", "last_name": "Price", "user_id": 101, "username": "Eudora43" } } ] RESULTS
  • 17. GET MULTIPLE DOCUMENTS SELECT * FROM users USE KEYS ["user_101","user_454"] SELECT BY DOCUMENT IDS couchbase.getMulti([ "user_101", "user_454" ]); STANDARD SDK OPERATION [ { "users": { "email": "Judah66@gmail.com", "first_name": "Albin", "last_name": "Price", "user_id": 101, "username": "Eudora43" } }, { "users": { "email": "Chase.Kohler63@gmail.com", "first_name": "Donnell", "last_name": "Ortiz", "user_id": 454, "username": "Garett31" } } ] RESULTS
  • 18. SELECT DIFFERENCES SELECT * FROM users USE KEYS "user_101" SELECT * [ { "users": { "email": "Judah66@gmail.com", "first_name": "Albin", "last_name": "Price", "user_id": 101, "username": "Eudora43" } } ] RESULTS SELECT users.* FROM users USE KEYS "user_101" SELECT ENTITY.* [ { "email": "Judah66@gmail.com", "first_name": "Albin", "last_name": "Price", "user_id": 101, "username": "Eudora43" } ] RESULTS Documents are projected as the bucket or alias name
  • 19. ? QUERY BY VALUE SELECT u.* FROM users AS u WHERE u.username = 'Eudora43' QUERY BY USERNAME [ { "email": "Judah66@gmail.com", "first_name": "Albin", "last_name": "Price", "user_id": 197, "username": "Eudora43" } ] RESULTS STANDARD SDK OPERATION // perform view query results = couchbase.query( designDocumentName="user", viewName="username", options={ key: "eudora43" } ); // get the document by the id // returned from the query user = couchbase.get(results[1].id); function (doc, meta) { if( doc.username ){ emit( doc.username.toLowerCase(), null ); } } VIEW MAP FUNCTION These queries are different. The N1QL query does not take case into consideration the View does. N1QL string operators are case sensitive, the N1QL query would need to use the LOWER() function This can be done without N1QL, however it would require a MapReduce View This query would require a secondary index
  • 21. INSERT INSERT INTO users (KEY, VALUE) VALUES ("user_1021", { "user_id": 1021, "name": "John Smith", "email": "john.smith@email.com" }) N1QL INSERT couchbase.insert("user_1021", { "user_id": 1021, "name": "John Smith", "email": "john.smith@email.com" }); STANDARD SDK OPERATION { "results": [], "metrics": { "elapsedTime": "50.361685ms", "executionTime": "50.322691ms", "resultCount": 0, "resultSize": 0, "mutationCount": 1 } } RESULTS
  • 22. RETURNING INSERT INTO users (KEY, VALUE) VALUES ("user_2343", { "user_id": 2343, "name": "John Smith", "email": "john.smith@gmail.com" }) RETURNING * N1QL INSERT W/ RETURNING // write the user document couchbase.insert("user_2343", { "user_id": 2343, "name": "John Smith", "email": "john.smith@gmail.com" }); // get the user document user = couchbase.get("user_2343"); STANDARD SDK OPERATION [ { "users": { "email": "john.smith@gmail.com", "name": "John Smith", "user_id": 2343 } } ] RESULTS
  • 23. UPDATE // get the user document user = couchbase.get("user_2343"); // change the email value user.email = "elmariachi@gmail.com"; // update the entire document couchbase.replace("user_2343", user); // read the write to get just the // updated email value user = couchbase.get("user_2343"); email = user.email; STANDARD SDK OPERATIONN1QL UPDATE UPDATE users USE KEYS "user_2343" SET email = "elmariachi@gmail.com" RETURNING email SDK replace() will error if the document does not exist, N1QL won’t [ { "email": "elmariachi@gmail.com" } ] RESULTS
  • 24. UPSERT // write the user document couchbase.upsert("user_2343", { "user_id": 2343, "name": "John Smith", "email": "john.smith@gmail.com" }); // get the user document user = couchbase.get("user_2343"); UPSERT INTO users (KEY, VALUE) VALUES ("user_2343", { "user_id": 2343, "name": "John Smith", "email": "john.smith@gmail.com" }) RETURNING META().id AS document_id N1QL UPSERT STANDARD SDK OPERATION [ { "document_id": "user2343" } ] RESULTS
  • 25. DELETE DELETE FROM users USE KEYS "user_2343" N1QL DELETE // write the user document couchbase.remove("user_2343"); STANDARD SDK OPERATION { "results": [], "metrics": { "elapsedTime": "21.591512ms", "executionTime": "21.559566ms", "resultCount": 0, "resultSize": 0 } } RESULTS SDK remove() will error if the document does not exist, N1QL won’t
  • 27. VIEW VS. GSI INDEXES GSI INDEXES VIEW INDEXES Partitioning Model • Part of Index Service • Each GSI Must be Unique • Can only be placed on one node • Can respond without scatter-gather • Auto-partitioned with Data Service • Expensive scatter-gather operations Scaling Model • Scales with Index Service • Scales with Data Service Performance • Singleton Lookups or Range Scans can respond without scatter-gather • Singleton Lookups or Range Scans can respond without scatter-gather Managed Cache • Comes with Managed Cache that performs better during Index Maintenance and Scans • Depends on File System Cache Storage Engine • ForestDB • Couchstore High Availability • Multiple indexes with identical definitions across multiple nodes • Built-in replicas with smart placement
  • 28. INDEXES http://developer.couchbase.com/ documentation/server/current/indexes/gsi- for-n1ql.html • "USE KEYS" statements do not require GSI Indexes • All other N1QL queries require GSI Indexes • PRIMARY Index contains all keys • Ad hoc queries require a PRIMARY Index • Index Names must be unique • USING GSI is the default CREATE PRIMARY INDEX idx_users_primary ON users USING GSI PRIMARY INDEX CREATE INDEX idx_users_username ON users( username ) USING GSI SECONDARY INDEX DROP PRIMARY INDEX ON idx_users_primary ON users USING GSI DROP PRIMARY INDEX DROP PRIMARY INDEX idx_users_primary ON users USING GSI DROP SECONDARY INDEX
  • 29. EXPLAIN EXPLAIN SELECT u.* FROM users AS u WHERE u.username = 'Eudora43' QUERY BY USERNAME [ { "plan": { "#operator": "Sequence", "~children": [ { "#operator": "IndexScan", "index": "idx_users_username", "index_id": "240a59c3f5ced906", "keyspace": "social", "namespace": "default", ... ], "using": "gsi" }, ... ] }, ... } ] RESULTS http://developer.couchbase.com/ documentation/server/current/n1ql/n1ql- language-reference/explain.html
  • 30. INDEX SCAN TYPES • PRIMARYSCAN • INDEXSCAN • KEYSCAN • PARENTSCAN • VALUESCAN • DUMMYSCAN • COUNTSCAN • INTERSECTSCAN Reference https://github.com/couchbase/query#plan for Scan explanations
  • 31. INDEXES CREATE INDEX idx_users_username ON users( username ) WHERE username IS NOT MISSING USING GSI FILTERED INDEX CREATE INDEX idx_users_username ON users( username ) WHERE doc_type = 'user' USING GSI FILTERED INDEX • Definitions can be duplicated • Indexes can be partitioned • Randomly assigned to an Index nodes in the cluster
  • 32. INDEXES CREATE INDEX idx_users_username ON users( username ) WHERE username IS NOT MISSING USING GSI WITH { "defer_build": true } DEFER THE INDEX BUILD BUILD INDEX ON users( idx_users_username, idx_users_email ) USING GSI BUILD AN INDEX CREATE INDEX idx_users_username ON users( username ) WHERE username IS NOT MISSING USING GSI WITH { "nodes": "node3:8091"} ADD INDEX TO SPECIFIC NODE • Index builds can be deferred • Multiple Indexes can be built at a time • Indexes can be deployed to specific nodes • GSI Indexes are not rebalanced
  • 33. INDEX LOAD BALANCING CREATE INDEX idx_users_username_AtoM ON users( username ) WHERE doc_type = 'user' AND username BETWEEN 'A' AND 'N' USING GSI RANGE PARTITIONED INDEXES CREATE INDEX idx_users_username_NtoZ ON users( username ) WHERE doc_type = 'user' AND username BETWEEN 'N' AND 'Z' USING GSI - A round robin algorithm is used to distribute index load - If a node responsible for a index goes down, a PrimaryIndexScan would be used, or the query would fail if there is no PRIMARY INDEX CREATE INDEX idx_users_username_AtoM_node1 ON users( username ) WHERE doc_type = 'user' AND username BETWEEN 'A' AND 'N' USING GSI WITH { "nodes": "node1:8091"} REDUNDANT / REPLICATED INDEXES CREATE INDEX idx_users_username_NtoZ_node1 ON users( username ) WHERE doc_type = 'user' AND username BETWEEN 'N' AND 'Z' USING GSI WITH { "nodes": "node1:8091"} CREATE INDEX idx_users_username_AtoM_node2 ON users( username ) WHERE doc_type = 'user' AND username BETWEEN 'A' AND 'N' USING GSI WITH { "nodes": "node2:8091"} CREATE INDEX idx_users_username_NtoZ_node2 ON users( username ) WHERE doc_type = 'user' AND username BETWEEN 'N' AND 'Z' USING GSI WITH { "nodes": "node2:8091"}
  • 35. MISSING ATTRIBUTES SELECT u.user_id, u.email, u.name FROM users AS u USE KEYS "user_197" QUERY { "email": "Judah66@gmail.com", "first_name": "Albin", "last_name": "Price", "user_id": 197, "username": "Eudora43" } SAMPLE DOCUMENT [ { "email": "Judah66@gmail.com" "user_id": 197 } ] RESULT Requested attributes that are missing are not returned as part of the response
  • 36. MISSING ATTRIBUTES SELECT u.user_id, u.email, IFMISSING( u.name, u.first_name || ' ' || u.last_name ) AS name FROM users AS u USE KEYS "user_197" QUERY { "email": "Judah66@gmail.com", "first_name": "Albin", "last_name": "Price", "user_id": 197, "username": "Eudora43" } SAMPLE DOCUMENT [ { "email": "Judah66@gmail.com", "name": "Albin Price", "user_id": 197 } ] RESULT
  • 37. SELECT airlines.airline_id, airlines.airline_name, IFNULL( airlines.airline_iata, airlines.airline_icao ) AS airline_code FROM `flight-data` AS codes USE KEYS 'airline_code_DL' INNER JOIN `flight-data` AS airlines ON KEYS 'airline_' || TOSTRING( codes.id ) LIMIT 1 QUERY [ { "airline_code": "DL", "airline_id": 2009, "airline_name": "Delta Air Lines" } ] RESULT { "_id": "airline_code_DL", "code": "DL", "code_type": "iata", "designation": "airline", "doc_type": "code", "id": 2009 } AIRLINE LOOKUP DOCUMENT { "_id": "airline_2009", "active": true, "airline_iata": "DL", "airline_icao": "DAL", "airline_id": 2009, "airline_name": "Delta Air Lines", "callsign": "DELTA", "doc_type": "airline", "iso_country": "US" } AIRLINE DOCUMENT JOINS
  • 38. MULTIPLE JOINS { "_id": "route_6cea35d1-37e1-5425-913f-42c29e248285", "airline_code": "DL", "destination_airport_code": "MSP", "doc_type": "route", "route_id": "6cea35d1-37e1-5425-913f-42c29e248285", "source_airport_code": "BWI" } ROUTE DOCUMENT { "_id": "airline_code_DL", "code": "DL", "code_type": "iata", "designation": "airline", "doc_type": "code", "id": 2009 } AIRLINE LOOKUP DOCUMENT { "_id": "airline_2009", "airline_iata": "DL", "airline_icao": "DAL", "airline_id": 2009, "airline_name": "Delta Air Lines", "doc_type": "airline" } AIRLINE DOCUMENT { "_id": "airport_code_BWI", "code": "BWI", "code_type": "iata", "designation": "airport", "doc_type": "code", "id": 3435 } AIRPORT LOOKUP DOCUMENT { "_id": "airport_3435", "airport_iata": "BWI", "airport_icao": "KBWI", "airport_id": 3435, "airport_ident": "KBWI", "airport_name": "Baltimore Washington Intl", "airport_type": "large_airport", "doc_type": "airport", "elevation": 146, "geo": { "latitude": 39.17539978, "longitude": -76.66829681 }, "iso_continent": "NA", "iso_country": "US", "iso_region": "US-MD", "municipality": "Baltimore" } AIRPORT DOCUMENT
  • 39. SELECT airlines.airline_name, IFNULL( airlines.airline_iata, airlines.airline_icao ) AS airline_code, source_airports.airport_name, source_airports.iso_country, source_airports.iso_region, IFNULL( source_airports.airport_iata, source_airports.airport_icao, source_airports.airport_ident ) AS airport_code FROM `flight-data` AS routes /* get airline details */ INNER JOIN `flight-data` AS airline_codes ON KEYS 'airline_code_' || routes.airline_code INNER JOIN `flight-data` AS airlines ON KEYS 'airline_' || TOSTRING( airline_codes.id ) /* get airport details */ INNER JOIN `flight-data` AS airport_codes ON KEYS 'airport_code_' || routes.source_airport_code INNER JOIN `flight-data` AS source_airports ON KEYS 'airport_' || TOSTRING( airport_codes.id ) WHERE routes.destination_airport_code = 'MSP' AND routes.source_airport_code IS NOT NULL AND routes.doc_type = 'route' AND routes.active = true ORDER BY source_airports.airport_name ASC, airlines.airline_name ASC QUERY [ { "airline_code": "DL", "airline_name": "Delta Air Lines", "airport_code": "ABR", "airport_name": "Aberdeen Regional Airport", "iso_country": "US", "iso_region": "US-SD" }, { "airline_code": "DL", "airline_name": "Delta Air Lines", "airport_code": "ABQ", "airport_name": "Albuquerque International Sunport", "iso_country": "US", "iso_region": "US-NM" }, { "airline_code": "DL", "airline_name": "Delta Air Lines", "airport_code": "BWI", "airport_name": "Baltimore Washington Intl", "iso_country": "US", "iso_region": "US-MD" }, ... ] RESULT MULTIPLE JOINS
  • 40. ANY ANY variable ( IN | WITHIN ) expression [ , variable ( IN | WITHIN ) expression ]* SATISFIES condition END EVERY EVERY variable ( IN | WITHIN ) expression [ , variable ( IN | WITHIN ) expression ]* SATISFIES condition END ARRAY ARRAY expression FOR variable ( IN | WITHIN ) expression [ , variable ( IN | WITHIN ) expression ]* [ ( WHEN condition) ] END FIRST FIRST expression FOR variable ( IN | WITHIN ) expression [ , variable ( IN | WITHIN ) expression]* [ ( WHEN condition) ] END EXISTS EXISTS expression IN expression [ NOT ] IN expression WITHIN expression [NOT] WITHIN expression COLLECTION OPERATORS http://developer.couchbase.com/ documentation/server/current/n1ql/ n1ql-language-reference/ collectionops.html
  • 41. ANY OPERATOR SELECT u.name, u.children FROM users AS u WHERE u.doc_type = 'user' AND ANY child IN u.children SATISFIES child.age > 10 END ORDER BY ARRAY_LENGTH(u.children) ASC QUERY [ { "children": [ { "age": 12, "first_name": "Emelia", "gender": "M" } ], "name": "Elyse Rempel" }, { "children": [ { "age": 15, "first_name": "Yvette", "gender": "F" }, { "age": 8, "first_name": "Rodolfo", "gender": "M" } ], "name": "Reynold Mohr" }, ... ] RESULT
  • 42. EVERY OPERATOR SELECT u.name, u.children FROM users AS u WHERE u.doc_type = 'user' AND EVERY child IN u.children SATISFIES child.age > 10 AND child.gender = 'M' END ORDER BY ARRAY_LENGTH(u.children) ASC QUERY [ { "children": [ { "age": 16, "first_name": "Grant", "gender": "M" } ], "name": "Joseph Stiedemann" }, { "children": [ { "age": 11, "first_name": "Ezekiel", "gender": "M" }, { "age": 12, "first_name": "Brooks", "gender": "M" } ], "name": "Aiden Christiansen" }, ... ] RESULT
  • 43. ARRAY INDEXING [ { "chat_id": "61bf583b2cd0", "created_on": "2016-05-29T13:31:29.759Z" }, { "chat_id": "d63a56047e08", "created_on": "2016-05-29T10:07:18.824Z" }, ... ] RESULT SELECT chats.chat_id, MILLIS_TO_STR(chats.created_on) AS created_on FROM social AS chats WHERE ANY user_id IN chats.users SATISFIES user_id = 100 END AND chats.doc_type = 'chat' ORDER BY chats.created_on DESC QUERY CREATE INDEX idx_chats_users ON social( DISTINCT ARRAY user_id FOR user_id IN users WHEN user_id IS NOT NULL END, doc_type ) WHERE doc_type = 'chat' INDEX { "_id": "chat_4ff8dd056225", "doc_type": "chat", "chat_id": "4ff8dd056225", "created_on": 1464518742983, "users": [ 861, 492 ] } SAMPLE DOCUMENT
  • 44. { "_id": "user_764", "doc_type": "user", "user_id": 764, "first_name": "Geovanny", "last_name": "Parker", "phones": [ { "type": "Mobile", "phone_number": "676.825.8926", "extension": null }, { "type": "Mobile", "phone_number": "792.877.3144", "extension": "3644" }, { "type": "Home", "phone_number": "(730) 490-6734", "extension": null } ] } SAMPLE DOCUMENT SELECT u.phones FROM users AS u USE KEYS 'user_764' QUERY [ { "phones": [ { "type": "Mobile", "phone_number": "676.825.8926", "extension": null }, { "type": "Mobile", "phone_number": "792.877.3144", "extension": "3644" }, { "type": "Home", "phone_number": "(730) 490-6734", "extension": null } ] } ] RESULT UNNEST This query will only provide a single result with a nested "phones" attribute that is an array
  • 45. UNNEST { "_id": "user_764", "doc_type": "user", "user_id": 764, "first_name": "Geovanny", "last_name": "Parker", "phones": [ { "type": "Mobile", "phone_number": "676.825.8926", "extension": null }, { "type": "Mobile", "phone_number": "792.877.3144", "extension": "3644" }, { "type": "Home", "phone_number": "(730) 490-6734", "extension": null } ] } SAMPLE DOCUMENT SELECT phone_numbers.* FROM users AS u USE KEYS 'user_23' UNNEST u.phones AS phone_numbers QUERY [ { "type": "Mobile", "phone_number": "676.825.8926", "extension": null }, { "type": "Mobile", "phone_number": "792.877.3144", "extension": "3644" }, { "type": "Home", "phone_number": "(730) 490-6734", "extension": null } ] RESULT UNNSET performs a JOIN on a documents attribute that is an array with its parent document
  • 46. NEST SELECT u.first_name, u.last_name, user_phones FROM users AS u USE KEYS 'user_581' INNER NEST users AS user_phones ON KEYS 'user_' || TOSTRING( u.user_id ) || '_phones' QUERY [ { "first_name": "Geovanny", "last_name": "Parker", "user_phones": [ { "_id": "user_581_phones", "doc_type": "user-phones", "phones": [ { "extension": null, "phone_number": "872-201-8963", "phone_type": "Mobile" }, { "extension": "9324", "phone_number": "720.194.5604", "phone_type": "Other" } ], "user_id": 581 } ] } ] RESULT { "_id": "user_581_phones", "doc_type": "user-phones", "user_id": 581, "phones": [ { "extension": null, "phone_number": "872-201-8963", "phone_type": "Mobile" }, { "extension": "9324", "phone_number": "720.194.5604", "phone_type": "Other" } ] } { "_id": "user_581", "doc_type": "user", "user_id": 581, "first_name": "Geovanny", "last_name": "Parker" } USER DOCUMENT USER PHONES DOCUMENT A NEST operation will take 0 or more documents and nest them as a single entry in the result INNER NEST will omit records if the right side is missing LEFT NEST will return records whether or not the right side was found or not
  • 47. NEST SELECT u.first_name, u.last_name, user_phones[0].phones FROM users AS u USE KEYS 'user_581' INNER NEST users AS user_phones ON KEYS 'user_' || TOSTRING( u.user_id ) || '_phones' QUERY [ { "first_name": "Geovanny", "last_name": "Parker", "phones": [ { "extension": null, "phone_number": "872-201-8963", "phone_type": "Mobile" }, { "extension": "9324", "phone_number": "720.194.5604", "phone_type": "Other" } ] } ] RESULT { "_id": "user_581_phones", "doc_type": "user-phones", "user_id": 581, "phones": [ { "extension": null, "phone_number": "872-201-8963", "phone_type": "Mobile" }, { "extension": "9324", "phone_number": "720.194.5604", "phone_type": "Other" } ] } { "_id": "user_581", "doc_type": "user", "user_id": 581, "first_name": "Geovanny", "last_name": "Parker" } USER DOCUMENT USER PHONES DOCUMENT
  • 48. NEST SELECT u.user_id, u.first_name, u.last_name, ARRAY { "phone_number": phone.phone_number, "phone_type": phone.phone_type, "extension": phone.extension } FOR phone IN IFMISSING(phones, []) END AS phones FROM users AS u USE KEYS 'user_581' LEFT NEST users AS phones ON KEYS ( ARRAY phone.phone_id FOR phone IN ( SELECT 'phone_' || phone_id AS phone_id FROM users AS phone_lookup USE KEYS 'user_' || TOSTRING(u.user_id) || '_phones' UNNEST phone_lookup.phones AS phone_id ) END ) QUERY { "_id": "user_581_phones", "doc_type": "user-phones", "user_id": 581, "phones": [ "886350f7-3f97-5405-b6d0-294ecf469f05", "8ee1bd2e-7b88-5d2c-a211-05a61ac367f8" ] } { "_id": "user_581", "doc_type": "user", "user_id": 581, "first_name": "Geovanny", "last_name": "Parker" } USER DOCUMENT USER PHONES DOCUMENT { "_id": "phone_886350f7-3f97-5405-b6d0-294ecf469f05", "doc_type": "phone", "phone_id": "886350f7-3f97-5405-b6d0-294ecf469f05", "user_id": 581, "phone_type": "Other", "phone_number": "872-201-8963", "extension": null } USER PHONES DOCUMENT generates the same exact results as the previous slide
  • 49. AGGREGATES SELECT COUNT(*) AS total_users FROM social AS users WHERE users.doc_type = 'user' QUERY [ { "total_users": 1000 } ] RESULT SELECT ARRAY_COUNT(users.children) AS children FROM social AS users USE KEYS 'user_132' QUERY [ { "children": 3 } ] RESULT SELECT MIN(children.age) AS min_age, MAX(children.age) AS max_age, AVG(children.age) AS avg_age FROM social AS users UNNEST users.children AS children WHERE users.doc_type = 'user' QUERY [ { "avg_age": 8.94, "max_age": 17, "min_age": 1 } ] RESULT http://developer.couchbase.com/ documentation/server/current/n1ql/n1ql- language-reference/aggregatefun.html
  • 51. POSITIONAL PARAMS couchbase.n1qlQuery( statement=" SELECT airport_id, airport_name, airport_type, municipality, geo, timezone, airport_iata, airport_icao, FROM `flight-data` WHERE iso_country = $1 AND iso_region = $2 AND doc_type = $3 ORDER BY airport_name ASC LIMIT 2 ", parameters=[ "US", "US-ME", "airport" ] ); SDK QUERY RESULT
  • 52. NAMED PARAMS couchbase.n1qlQuery( statement=" SELECT airport_id, airport_name, airport_type, municipality, geo, timezone, airport_iata, airport_icao, FROM `flight-data` WHERE iso_country = $country AND iso_region = $region AND doc_type = $doc_type ORDER BY airport_name ASC LIMIT 2 ", parameters={ 'country': "US", 'region': "US-VT", 'doc_type': "airport" } ); SDK QUERY RESULT
  • 53. PREPARED STATEMENTS couchbase.n1qlQuery(" PREPARE SELECT COUNT(1) AS total_airports FROM `flight-data` WHERE iso_country = 'US' AND iso_region IS NOT NULL AND doc_type = 'airport' "); PREPARE QUERY PREPARE RESULT couchbase.n1qlQuery(" EXECUTE '9e4e1d3d-f1c3-4292-81cb-824af1fae87e' "); EXECUTE QUERY EXECUTE RESULT Prepared Statements are stored in memory until a server restart
  • 54. NAMED PREPARED STATEMENTS couchbase.n1qlQuery(" PREPARE us_airports FROM SELECT COUNT(1) AS total_airports FROM `flight-data` WHERE iso_country = 'US' AND iso_region IS NOT NULL AND doc_type = 'airport' "); PREPARE QUERY RESULT couchbase.n1qlQuery(" EXECUTE 'us_airports' "); EXECUTE QUERY RESULT
  • 55. DOCUMENT LOCKING user = couchbase.getAndLock("user_101"); PREPARE QUERY RESULT • Default lock time is 5 seconds • Max lock time is 30 seconds • Prevents retrieval while Locked • Prevents updates while Locked • An update requires CAS
  • 56. UNLOCKING A DOCUMENT // replace the document using CAS couchbase.replaceWithCAS( id="user_101", value=doc, cas=user.cas ); REPLACING THE DOCUMENT W/ CAS RESULT // update the document couchbase.upsert("user_101", doc); // replace the document couchbase.replace("user_101", doc); WHILE LOCKED UPSERT AND REPLACE WILL FAIL ERROR // unlock the document, returns true or // false if the unlock was successful couchbase.unlock( id="user_101", cas=user.cas ); USING UNLOCK Documents will automatically unlock after the "lock time" has been exceeded
  • 57. REPLICA READS try { // retrieve and lock the document result = couchbase.get("user_100"); } catch(any e){ // attempt to get the document from a replica result = couchbase.getFromReplica("user_100"); } // replica reads return an array for each // configured replica in the bucket if (isArray(result)) { result = result[0]; } GET DOCUMENT FROM REPLICA(S)
  • 58. COUNTERS // increment the user counter next_id = couchbase.counter( id="user_counter", value=1, defaultValue=0 ); couchbase.insert("user_" & next_id, { "user_id": next_id, "name": "John Smith", "email": "john.smith@email.com" }); INCREMENT VALUE // decrement the tickets counter available_tix = couchbase.counter( id="tickets_counter", value=-1, defaultValue=1000 ); // if there are no more tickets // available redirect the user if (available_tix < 1) { cflocation(url="sold-out.cfm", addtoken=false); } ... DECREMENT VALUE Counter operations are Atomic
  • 59. CF TRAVEL SAMPLE bit.ly/cfcouchbase-travel • AIRLINES • AIRPORTS • HOTELS • LANDMARKS • ROUTES
  • 61. FLIGHT DATA bit.ly/flight-data-n1ql • AIRLINE REVIEWS • AIRLINES • AIRPORT AIRLINES • AIRPORT FREQUENCIES • AIRPORT NAVAIDS • AIRPORT REVIEWS • AIRPORT RUNWAYS • AIRPORTS • CODES • CONTINENTS • COUNTRIES • FREQUENCIES • NAVAIDS • REGIONS • ROUTES • RUNWAYS • USERS
  • 62. SOCIAL DATA bit.ly/social-data-n1ql • ADDRESSES • AUTH • CHAT MESSAGES • CHATS • EMAILS • FRIENDS • LIKES • PHONES • POSTS • USER ADDRESSES • USER EMAILS • USER PHONES • USERS
  • 63. ROADMAP • FULL TEXT SEARCHING (COMING IN CB 4.5) • SUBDOC SUPPORT • REMOVAL OF DEPRECATED METHODS • SDK ALIGNMENT • REGULAR RELEASE CYCLE • MORE EXAMPLES
  • 66. DRAWING SELECT drawing.attendees[ FLOOR( RANDOM() * ARRAY_LENGTH(drawing.attendees) ) ] AS winner FROM `default` AS drawing USE KEYS 'drawing'