6. * theoretically up to 2 billion
Timestamp used for conflict resolution (Last Write Wins)
Column→Row→ColumnFamily(Table)
A Row composed of Row Key and multiple Columns
Column Family composed of multiple Rows
(Table)
7. CQL(Cassandra Query Language)
Single Partition Key
CREATE TABLE books (
title text,
author text,
year int,
PRIMARY KEY (title)
);
INSERT INTO books (title, author, year)
VALUES ('Patriot Games', 'Tom Clancy', 1987);
INSERT INTO books (title, author, year)
VALUES ('Without Remorse', 'Tom Clancy', 1993);
select * from books;
title | author | year
----------------+------------+------
Without Remorse| Tom Clancy | 1993
Patriot Games| Tom Clancy | 1987
RowKey: Without Remorse
=> (name=, value=, timestamp=1393102991499)
=> (name=author, value=Tom Clancy, timestamp=1393102991499)
=> (name=year, value=1993, timestamp=1393102991499)
RowKey: Patriot Games
=> (name=, value=, timestamp=1393102991499100)
=> (name=author, value=Tom Clancy, timestamp=1393102991499100)
=> (name=year, value=1987, timestamp=1393102991499100)
ColumnName
ColumnValue
ColumnName ColumnValue
CREATE TABLE users (
id timeuuid,
lastname varchar,
firstname varchar,
dateOfBirth timestamp,
PRIMARY KEY(id)
);
PRIMARY KEY只有⼀一个PK
和传统的数据库没什么区别
由于PK是timeuuid,每次插⼊入
数据的timeuuid总是不同的,
所以插⼊入都是全新的记录。
12. Compound Keys
CREATE TABLE authors (
name text,
year int,
title text,
isbn text,
publisher text,
PRIMARY KEY (name, year, title)
);
CREATE TABLE authors (
name text,
year int,
title text,
isbn text,
publisher text,
PRIMARY KEY (name, year, title)
) WITH CLUSTERING ORDER BY (year DESC);
name | year | title | isbn | publisher
------------+------+-----------------+---------------+-----------
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam
RowKey: Tom Clancy
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4)
=> (name=1987:Patriot Games:publisher, value=Putnam)
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0)
=> (name=1993:Without Remorse:publisher, value=Putnam)
name | year | title | isbn | publisher
------------+------+-----------------+---------------+-----------
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam
RowKey: Tom Clancy
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0)
=> (name=1993:Without Remorse:publisher, value=Putnam)
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4)
=> (name=1987:Patriot Games:publisher, value=Putnam)
insert into authors(name,year,title,isbn,publisher) values ('Tom Clancy',1987,'Patriot Games','0-399-13241-4','Putnam');
insert into authors(name,year,title,isbn,publisher) values ('Tom Clancy',1993,'Without Remorse','0-399-13825-0','Putnam');
RowKey |1987:Patriot Games:ISBN|1987:Patriot Games:publisher|1993:Without Remorse:ISBN|1993:Without Remorse:publisher
----------+-----------------------+----------------------------+-------------------------+------------------------------
Tom Clancy| 0-399-13241-4 | Putnam | 0-399-13825-0 | Putnam
<year>:<title>:ISBN <year>:<title>:publisher
Clustering Keys
Ordinary Columns
One Row
13. Composite Partition Keys
CREATE TABLE authors (
name text,
year int,
title text,
isbn text,
publisher text,
PRIMARY KEY ((name, year), title)
);
name | year | title | isbn | publisher
------------+------+-----------------+---------------+-----------
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam
RowKey: Tom Clancy:1987
=> (name=Patriot Games:isbn, value=0-399-13825-0)
=> (name=Patriot Games:publisher, value=Putnam
---------------------------------------------------
RowKey: Tom Clancy:1993
=> (name=Without Remorse:isbn, value=0-399-13241-4)
=> (name=Without Remorse:publisher, value=Putnam)
RowKey |Patriot Games:ISBN|Patriot Games:publisher
---------------+------------------+------------------------
Tom Clancy:1987| 0-399-13241-4 | Putnam
RowKey |Without Remorse:ISBN|Without Remorse:publisher
---------------+--------------------+-------------------------
Tom Clancy:1993| 0-399-13825-0 | Putnam
RowKey |1987:Patriot Games:ISBN|1987:Patriot Games:publisher|1993:Without Remorse:ISBN|1993:Without Remorse:publisher
----------+-----------------------+----------------------------+-------------------------+------------------------------
Tom Clancy| 0-399-13241-4 | Putnam | 0-399-13825-0 | Putnam
#Row 1
#Row 2
Two Row(based on Partition Key)
CREATE TABLE authors (
name text,
year int,
title text,
isbn text,
publisher text,
PRIMARY KEY (name, year, title)
);
14. age role
john 37 dev
age role
eric 38 ceo
name | age | role
-----+-----+-----
john | 37 | dev
eric | 38 | ceo
CREATE TABLE employees (
name text PRIMARY KEY,
age int,
role text
);
CREATE TABLE employees (
company text,
name text,
age int,
role text,
PRIMARY KEY (company,name)
);
company | name | age | role
--------+------+-----+-----
OSC | eric | 38 | ceo
OSC | john | 37 | dev
RKG | anya | 29 | lead
RKG | ben | 27 | dev
RKG | chad | 35 | ops
eric:ag
e
eric:role john:ag
e
john:role
OSC 38 dev 37 dev
anya:age anya:role ben:ag
e
ben:role chad:age chad:role
RKG 29 lead 27 dev 35 ops
CREATE TABLE example (
A text,
B text,
C text,
D text,
E text,
F text,
PRIMARY KEY ((A,B),C,D)
);
A | B | C | D | E | F
--+---+---+---+---+---
a | b | c | d | e | f
a | b | c | g | h | i
a | b | j | k | l | m
a | n | o | p | q | r
s | t | u | v | w | x
c:d:E c:d:F c:g:E c:g:F j:k:E j:k:F
a:b e f h i l m
o:p:E o:p:F
a:n q r
u:v:E u:v:F
s:t w x
More Examples(1:Basic)
15. SETS
CREATE TABLE mytable(
X text,
Y text,
myset set<int>,
PRIMARY KEY (X,Y)
);
X | Y | myset
---+---+------------
a | b | {1,2}
a | c | {3,4,5}
b:myset:1 b:myset:2 c:myset:3 c:myset:4 c:myset:5
a
X | Y | mylist
---+---+------------
a | b | [1,2]
b:mylist:f7e5450039..8d b:mylist:f7e5450139..8d
a 1 2
LISTS
CREATE TABLE mytable(
X text,
Y text,
mylist list<int>,
PRIMARY KEY (X,Y)
);
MAPS
CREATE TABLE mytable(
X text,
Y text,
mymap map<text,int>,
PRIMARY KEY (X,Y)
);
X | Y | mymap
---+---+------------
a | b | {m:1,n:2}
a | c |{n:3,p:4,q:5}
b:mymap:m b:mymap:n c:mymap:n c:mymap:p c:mymap:q
a 1 2 3 4 5
More Examples(2:Collection)
key
value
mapName<ClusterKey>
ColumnName
List elements
timeuuidlistName
ColumnName
<ClusterKey>
setName
ColumnName
<ClusterKey>
Set Elements
16. name | year | title | publisher
------------------+------+-----------------+--------------
Tom Clancy | 1987 | Patriot Games | Putnam
Dean Koontz | 1991 | Cold Fire | Headline
Anne Rice | 1998 | Pandora | Random House
Charles Dickens | 1838 | Oliver Twist | Random House
Secondary Index are the only type of index that Cassandra will manage for you, so the terms “index” and “secondary index”
actually refer to the same mechanism. The purpose of an index is to allow query-by-value functionality(not by primary key).
索引(⼆二级索引)
At the storage layer, a secondary index is simply another
column family,where the key is the value of the indexed column,
and the columns contain the row keys of the indexed table.
⼆二级索引和IN查询类似,因为要查询索引条⺫⽬目的所有Row Keys
IN查询可以确定具体查询哪些节点,但是⼆二级索引查询所有节点
因为⼆二级索引是本地的,不是全局,即使节点没有索引值,也要
去这个节点查询,⽐比如empty index所在节点仍然需要查询。
本地:每个节点建⽴立各⾃自的索引,⽆无法感知集群其他节点的索引
$
Node1
Node2
publisher | names
---------------+----------------------------
Putnam | [Tom Clancy]
Headline | [Dean Koontz]
Random House | [Anne Rice, Charles Dickens]
% &
RowKey: Putnam
=> (name=Tom Clancy, value=)
RowKey: Headline
=> (name=Dean Koontz, value=)
RowKey: Random House
=> (name=Anne Rice, value=)
=> (name=Charles Dickens, value=)
Index Column Row Keys
CREATE INDEX author_publisher ON author (publisher);
19. 》insert, flush
[
{"key": "16","columns": [["","",1417814256390], ["col2","26",1417814256390], ["col3","36",1417814256390], ["id","id16",1417814256390]]},
{"key": "15","columns": [["","",1417814244766], ["col2","25",1417814244766], ["col3","35",1417814244766], ["id","id15",1417814244766]]},
{"key": "14","columns": [["","",1417814230711], ["col2","24",1417814230711], ["col3","34",1417814230711], ["id","id14",1417814230711]]},
{"key": "13","columns": [["","",1417814218246], ["col2","23",1417814218246], ["col3","33",1417814218246], ["id","id13",1417814218246]]},
{"key": "12","columns": [["","",1417814207910], ["col2","22",1417814207910], ["col3","32",1417814207910], ["id","id12",1417814207910]]},
{"key": "11","columns": [["","",1417814197094], ["col2","21",1417814197094], ["col3","31",1417814197094], ["id","id11",1417814197094]]},
{"key": "1","columns": [["","",1417814185270], ["col2","2",1417814185270], ["col3","3",1417814185270], ["id","id1",1417814185270]]}
]
》delete ROW, delete COLUMN, flush
delete from ts1 WHERE col1 = '1';
delete id from ts1 WHERE col1 = ’11';
delete col2 from ts1 WHERE col1 = '12';
[
{"key": "1","metadata": {"deletionInfo": {"markedForDeleteAt":1417814302304,"localDeletionTime":1417814302}},"columns": []},
{"key": "11","columns": [["id","54822130",1417814320400,"d"]]},
{"key": "12","columns": [["col2","5482220b",1417814539434,"d"]]}
]
》compact
[
{"key": "16","columns": [["","",1417814256390], ["col2","26",1417814256390], ["col3","36",1417814256390], ["id","id16",1417814256390]]},
{"key": "13","columns": [["","",1417814218246], ["col2","23",1417814218246], ["col3","33",1417814218246], ["id","id13",1417814218246]]},
{"key": "15","columns": [["","",1417814244766], ["col2","25",1417814244766], ["col3","35",1417814244766], ["id","id15",1417814244766]]},
{"key": "14","columns": [["","",1417814230711], ["col2","24",1417814230711], ["col3","34",1417814230711], ["id","id14",1417814230711]]},
{"key": "12","columns": [["","",1417814207910], ["col2","5482220b",1417814539434,"d"], ["col3","32",1417814207910], ["id","id12",1417814207910]]},
{"key": "11","columns": [["","",1417814197094], ["col2","21",1417814197094], ["col3","31",1417814197094], ["id","54822130",1417814320400,"d"]]},
{"key": "1","metadata": {"deletionInfo": {"markedForDeleteAt":1417814302304,"localDeletionTime":1417814302}},"columns": []}
]
》gc_grace_seconds, compact, delete/clean-up Tombstone
[
{"key": "16","columns": [["","",1417814256390], ["col2","26",1417814256390], ["col3","36",1417814256390], ["id","id16",1417814256390]]},
{"key": "13","columns": [["","",1417814218246], ["col2","23",1417814218246], ["col3","33",1417814218246], ["id","id13",1417814218246]]},
{"key": "15","columns": [["","",1417814244766], ["col2","25",1417814244766], ["col3","35",1417814244766], ["id","id15",1417814244766]]},
{"key": "14","columns": [["","",1417814230711], ["col2","24",1417814230711], ["col3","34",1417814230711], ["id","id14",1417814230711]]},
{"key": "12","columns": [["","",1417814207910], ["col3","32",1417814207910], ["id","id12",1417814207910]]},
{"key": "11","columns": [["","",1417814197094], ["col2","21",1417814197094], ["col3","31",1417814197094]]}
]
column or row deleted before really disappear after gc_grace_seconds
column or row still stay on sstable, but with tombstone(d) marked
delete command will marked as tombstone after flush to sstable
after flush as sstable, you can use sstablejson to check data
row-key=1 deleted
row-key=11, column=id deleted
row -key=12, column=col2 deleted
use sstablejson to see tombstone
http://stackoverflow.com/questions/27776337/what-types-of-tombstones-does-cassandra-support?rq=1
CREATE TABLE ts1 (
col1 text,
col2 text,
col3 text,
id text,
PRIMARY KEY ((col1))
)
e - expired TTL
d - deleted value (tombstone)
t - deleted range of values (range tombstone)
20. C* INSERT Flow: Coordinator(replicas)→CommitLog→Memtable
How to Check Data…
1.insert data
2.nodetool flush
3.sstabledump Data.db
Demo Time…
21. DELETE FLOW SAME AS INSERT FLOW…(coord,log,mem)
Single Partition Query: [sstable+memtable→merge]
Row Tombstone,Should Be 3 Tombstone Cells?Why 0??
1.Determing replicas for mutation
2.Appending to commitlog
3.Adding to #table memtable
🔍
关于Collection类型的删除:http://thelastpickle.com/blog/2016/07/27/about-deletes-and-tombstones.html
22. Delete NonExist Row,
Query Return Empty,
But Produce Tombstone
name | year | title | isbn | publisher
------------+------+-----------------+---------------+-----------
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam
❌
这条Row Key记录根本就不存在
不过Row Key即使不存在,
Delete也会产⽣生⼀一个Tombstone
24. insert into authors(name,year,title,isbn,publisher) values ('Tom Clancy',1987,'Patriot Games','0-399-13241-4','Putnam');
insert into authors(name,year,title,isbn,publisher) values ('Tom Clancy',1993,'Without Remorse','0-399-13825-0','Putnam');
DELETE FROM authors WHERE name = 'Tom Clancy' AND year = 1987 AND title = 'Patriot Gamess';
SELECT * FROM authors WHERE name = 'Tom Clancy' AND year = 1987 AND title = 'Patriot Games';
DELETE isbn FROM authors WHERE name = 'Tom Clancy' AND year = 1983 AND title = 'Without Remorse';
DELETE FROM authors WHERE name = 'Tom Clancy' AND year = 1983 AND title = 'Without Remorse';
SELECT * FROM authors WHERE name = 'Tom Clancy' AND year = 1983 AND title = 'Without Remorse';
DELETE isbn FROM authors WHERE name = 'Tom Clancy' AND year = 1993 AND title = 'Without Remorse';
DELETE FROM authors WHERE name = 'Tom Clancy' AND year = 1993 AND title = 'Without Remorse';
SELECT * FROM authors WHERE name = 'Tom Clancy' AND year = 1993 AND title = 'Without Remorse';
删除不存在的记录,仍然会产⽣生Tombstone
先Delete Column,再Delete Row,最后还是只剩下Row Tombstone(⼤大覆盖⼩小)
Row Tombstone
sstabledump(tombstone)
37. Tombstone(Read) can’t use Bloom Filter
BF⽤用来判断RowKey是否存在于SSTable中,
如果BF判定为True,有可能存在,有可能不存在
如果BF判定为False,则⼀一定不会存在!
但是即使判定为False,有可能存在Tombstone!
所以还是需要读取SSTable⽂文件中的Tombstone!
A query for a key in an sstable that has only tombstones associated
with it will still pass through the bloom filter, because the system must
reconcile(协调) tombstones with other replicas. Since the bloom filter
is designed to prevent unnecessary reads for missing data, this means
Cassandra will perform extra reads after data has been deleted.
BloomFilter FalseRead(“row1”) ❌
BloomFilter FalseRead(“row1”) 仍然要读取⽂文件,因为存在Tombstone
tombstone
读取Tombstone时,BloomFilter不能正常使⽤用!
Read(“row1”)
既然BloomFilter没有发挥应有的义务,就不需要BF了!
❎
✅
✅