SlideShare ist ein Scribd-Unternehmen logo
1 von 87
A NEW PLATFORM FOR A NEW ERA
2© Copyright 2015 Pivotal. All rights reserved. 2© Copyright 2015 Pivotal. All rights reserved.
Open Sourcing
GemFire
An Introduction to Apache Geode (incubating)
The open source distributed in-memory database
Anthony Baker
abaker@apache.org
@metatype
Dan Smith
dsmith@pivotal.io
3© Copyright 2015 Pivotal. All rights reserved.
Topics
 GemFire history, architecture, and use cases
 Geode, the open source core of GemFire
 Example application and demo
 Technical deep dive
4© Copyright 2015 Pivotal. All rights reserved.
It’s all about the
DATAand how you use it
5© Copyright 2015 Pivotal. All rights reserved.
2004 2008 2014
• Massive increase in data
volumes
• Falling margins per
transaction
• Increasing cost of IT
maintenance
• Need for elasticity in
systems
• Financial Services
Providers (every major
Wall Street bank)
• Department of Defense
• Real Time response needs
• Time to market constraints
• Need for flexible data
models across enterprise
• Distributed development
• Persistence + In-memory
• Global data visibility needs
• Fast Ingest needs for data
• Need to allow devices to
hook into enterprise data
• Always on
• Largest travel Portal
• Airlines
• Trade clearing
• Online gambling
• Largest Telcos
• Large mfrers
• Largest Payroll processor
• Auto insurance giants
• Largest rail systems on
earth
Hybrid Transactional
/Analytics grids
Our GemFire Journey Over The Years
6© Copyright 2015 Pivotal. All rights reserved.
Apps at Scale Have Unique Needs
Pivotal GemFire is the distributed, in-memory database for
apps that need:
1. Elastic scale-out performance
2. High performance database capabilities in distributed systems
3. Mission critical availability and resiliency
4. Flexibility for developers to create unique applications
5. Easy administration of distributed data grids
7© Copyright 2015 Pivotal. All rights reserved.
1. Elastic Scale-Out Performance
Nodes
Ops /
SecLinear scalability
Elastic capacity +/-
Latency-minimizing
data distribution
China Railway
Corporation
“The system is operating with solid
performance and uptime. Now, we have a
reliable, economically sound production
system that supports record volumes and
has room to grow”
Dr. Jiansheng Zhu, Vice Director of China
Academy of Railway Sciences
• 4.5 million ticket purchases & 20
million users per day.
• Spikes of 15,000 tickets sold per
minute, 40,000 visits per second.
8© Copyright 2015 Pivotal. All rights reserved.
2. High Performance Database Capabilities
Performance-
optimized
persistence
Configurable
consistency Partitioned Replicated Disabled
Distributed &
continuous queries
Distributed transactions, indexing, archival
Newedge
“Our global deployment of Pivotal
GemFire’s distributed cache gives me a
single version of the trade – resolving
hard-to-test-for synchronization issues
that exist within any globally distributed
business application architecture”
Michael Benillouche, Global Head of Data
Management
9© Copyright 2015 Pivotal. All rights reserved.
3. Mission Critical Availability and Resiliency
Cluster to cluster
WAN connectivity
Cluster resilience
& failover
XX Gire
“We can track and collect money at our
4,000+ kiosks and branches – even without
a reliable Internet connection. GemFire
provides the core data grid and a significant
amount of related functionality to help us
handle this unreliable network problem”
Gustavo Valdez, Chief of Architecture and
Development
• 19 million payment transactions per
month
• 4000+ points of sale with intermittent
Internet connectivity
10© Copyright 2015 Pivotal. All rights reserved.
4. Flexibility for Developers to Create Unique
Apps
• Data Structures:
– User-defined classes
– Documents (JSON)
• Native language support:
– Java, C++, C#
• API’s
– Java: Hashmap
– Memcache
– Spring Data GemFire
– C++ Serialization API’s
• Embedded query authoring
– Object Query Language (OQL)
• Publish & subscribe framework for
continous query & reliable
asynchronous event queues
• App-server embedded functionality:
– Web app session state caching
– L2 Hibernate
11© Copyright 2015 Pivotal. All rights reserved.
5. Easy Administration of Distributed Data
Grids
• Auto tuning of distributed computing resources to optimize
performance
• Cluster monitoring dashboard
– Cluster and node status & performance
• Offline performance statistics analysis tool
– View historical logs and events to diagnose performance and resource bottlenecks
• Command-line tool for taking action on clusters and nodes
12© Copyright 2015 Pivotal. All rights reserved.
What makes it fast?
 Design center is RAM not HDD
 Really demanding customers
 Avoid and minimize, particularly on the critical read/write paths:
– Network hops, copying data, contention, distributed locking, disk seeks, garbage
 Lots and lots of testing
– Establish and monitor performance baselines
– Distributed systems testing is difficult!
13© Copyright 2015 Pivotal. All rights reserved.
Horizontal Scaling for GemFire (Geode) Reads
With Consistent Latency and CPU
• Scaled from 256 clients and 2 servers to 1280 clients and 10 servers
• Partitioned region with redundancy and 1K data size
0
2
4
6
8
10
12
14
16
18
0
1
2
3
4
5
6
2 4 6 8 10
Speedup
Server Hosts
speedup
latency (ms)
CPU %
14© Copyright 2015 Pivotal. All rights reserved.
GemFire (Geode) 3.5-4.5X Faster Than Cassandra
for YCSB
15© Copyright 2015 Pivotal. All rights reserved.
Roadmap
 HDFS persistence
 Off-heap storage
 Lucene indexes
 Spark integration
 Cloud Foundry service
…and other ideas from the Geode community!
16© Copyright 2015 Pivotal. All rights reserved. 16© Copyright 2015 Pivotal. All rights reserved.
Use cases and patterns
17© Copyright 2015 Pivotal. All rights reserved.
“Low touch” Usage Patterns
Simple template for TCServer, TC, App servers
Shared nothing persistence, Global session state
HTTP Session management
Set Cache in hibernate.cfg.xml
Support for query and entity caching
Hibernate L2 Cache plugin
Servers understand the memcached wire protocol
Use any memcached client
Memcached protocol
<bean id="cacheManager"
class="org.springframework.data.gemfire.support.GemfireCacheManager"Spring Cache Abstraction
18© Copyright 2015 Pivotal. All rights reserved.
Application Patterns
 Caching for speed and scale
– Read-through, Write-through, Write-behind
 Geode as the OLTP system of record
– Data in-memory for low latency, on disk for durability
 Parallel compute engine
 Real-time analytics
19© Copyright 2015 Pivotal. All rights reserved.
Development Patterns
 Configure the cluster programmatically or declaratively
using cache.xml, gfsh, or SpringDataGemFire
<cache>
<region name="turbineSensorData" refid="PARTITION_PERSISTENT">
<partition-attributes redundant-copies="1" total-num-buckets="43"/>
</region>
</cache>
20© Copyright 2015 Pivotal. All rights reserved.
Development Patterns
 Write key-value data into a Region using { create | put |
putAll | remove }
– The value can be flat data, nested objects, JSON, …
Region sensorData = cache.getRegion("TurbineSensorData");
SensorKey key = new SensorKey(31415926, "2013-05-19T19:22Z"); // turbineId, timestamp
sensorData.put(key, new TurbineReading()
.setAmbientTemp(75)
.setOperatingTemp(80)
.setWindDirection(0)
.setWindSpeed(30)
.setPowerOutput(5000)
.setRPM(5));
21© Copyright 2015 Pivotal. All rights reserved.
Development Patterns
 Read values from a Region by key using { get | getAll }
TurbineReading data = sensorData.get(new SensorKey(31415926,"2013-05-
19T19:22Z"));
 Query values using OQL
// finds all sensor readings for the given turbine
SELECT * from /TurbineSensorData.entrySet WHERE key.turbineId = 31415926
// finds all sensor readings where the operating temp exceeds a threshold
SELECT * from /TurbineSensorData.entrySet WHERE value.operatingTemp > 120
 Apply indexes to optimize queries
22© Copyright 2015 Pivotal. All rights reserved.
Development Patterns
 Execute functions to operate on local data in parallel
 Respond to updates using CacheListeners and Events
 Automatic redundancy, partitioning, distribution, consistency,
network partition detection & recovery, load balancing, …
23© Copyright 2015 Pivotal. All rights reserved. 23© Copyright 2015 Pivotal. All rights reserved.
Apache Geode
(incubating)
The open source core of GemFire
24© Copyright 2015 Pivotal. All rights reserved.
Why OSS? Why Now? Why Apache?
 Open Source Software is fundamentally changing buying patterns
– Developers have to endorse product selection (No longer CIO handshake)
– Community endorsement is key to product visibility
– Open source credentials attract the best developers
– Vendor credibility directly tied to street credibility of product
 Align with the tides of history
– Customers increasingly asking to participate in product development
– Allow product development to happen with full transparency
 Apache is where you go to build Open Source street cred
– Transparent, meritocracy which puts developers in charge
– Roman keeps shouting “Apache!” every few hours
25© Copyright 2015 Pivotal. All rights reserved.
Geode Will Be A Significant Apache Project
 Over a 1000 person years invested into cutting edge R&D
 Thousands of production customers in very demanding verticals
 Cutting edge use cases that have shaped product thinking
 Tens of thousands of distributed , scaled up tests that can randomize
every aspect of the product
 A core technology team that has stayed together since founding
 Performance differentiators that are baked into every aspect of the
product
26© Copyright 2015 Pivotal. All rights reserved.
Geode or GemFire?
 Geode is a project, GemFire is a product
 We donated everything but the kitchen sink*
 More code drops imminent; going forward all development
happens OSS-style (“The Apache Way”)
* Multi-site WAN replication, continuous queries, and native (C/C++) client driver
27© Copyright 2015 Pivotal. All rights reserved.
28© Copyright 2015 Pivotal. All rights reserved.
X
Apache
29© Copyright 2015 Pivotal. All rights reserved.
30© Copyright 2015 Pivotal. All rights reserved. 30© Copyright 2015 Pivotal. All rights reserved.
Geode Demo
31© Copyright 2015 Pivotal. All rights reserved.
Post Region
Partitioned
People Region
Partitioned
Social Network
Person
Name: String
Description:String
Post
Id: PostId(name, date)
Text: String
32© Copyright 2015 Pivotal. All rights reserved.
Partition put
Client
Server 1
Server 2
Server 3
Bucket 1
Bucket 1
Bucket 2
Bucket 2
#(LOL)=1
Put LOL
33© Copyright 2015 Pivotal. All rights reserved.
Partition put
Client
Server 1
Server 2
Server 3
LOL
LOL
Bucket 2
Bucket 2
Replicate
To Secondary
34© Copyright 2015 Pivotal. All rights reserved.
“User” Use Case – Save Objects
public interface PersonRepository extends CrudRepository<Person, String> {
}
@Autowired
PersonRepository people;
public static void main(String[] args) {
people.save(new Person(name));
posts.save(new Post(new PostId(name, date), text));
}
Nested Objects,
Compound Keys
35© Copyright 2015 Pivotal. All rights reserved.
“User” Use Case – Save Objects
public interface PersonRepository extends CrudRepository<Person, String> {
}
@Autowired
PersonRepository people;
public static void main(String[] args) {
people.save(new Person(name));
posts.save(new Post(new PostId(name, date), text));
}
Automatically Serialized
With PDX
36© Copyright 2015 Pivotal. All rights reserved.
Configuration
<gfe:partitioned-region id="people" copies="1"/>
<bean id="pdxSerializer”
class="com.gemstone.gemfire.pdx.ReflectionBasedAutoSerializer">
<constructor-arg value="io.pivotal.happysocial.model.*"/>
</bean>
<gfe:cache pdx-serializer-ref="pdxSerializer"/>
37© Copyright 2015 Pivotal. All rights reserved.
Data Analyst – Determine Sentiment
 Find all of the posts for a user
 Analyze their content
38© Copyright 2015 Pivotal. All rights reserved.
First try – Just use a Query
public interface PostRepository extends
GemfireRepository<Post, PostId> {
@Query("select * from /posts where id.person=$1")
public Collection<Post> findPosts(String personName);
}
Collection<Post> posts = postRepository.findPosts(personName);
String sentiment = sentimentAnalyzer.analyze(posts);
39© Copyright 2015 Pivotal. All rights reserved.
First try – Just use a Query
public interface PostRepository extends
GemfireRepository<Post, PostId> {
@Query("select * from /posts where id.person=$1")
public Collection<Post> findPosts(String personName);
}
Collection<Post> posts = postRepository.findPosts(personName);
String sentiment = sentimentAnalyzer.analyze(posts);
Query Nested Objects
40© Copyright 2015 Pivotal. All rights reserved.
Use an Index
<gfe:index
id="postAuthor"
expression="id.person"
from="/posts”
/>
41© Copyright 2015 Pivotal. All rights reserved.
Still could be more efficient
Client
Server 1
Server 2
Server 3
Joe: LOL!!
Joe: LOL!!
EJ: arrg
Maya: Hii
Jess: sup
Jess: ok
Hitting multiple
Nodes
Bringing too much
Data to the client
42© Copyright 2015 Pivotal. All rights reserved.
Colocate the data
Client
Server 1
Server 2
Server 3
Joe: LOL!! Joe: LOL!!
EJ: arrgMaya: Hii
Jess: sup Jess: ok
<gfe:partitioned-region id="posts" copies="1"
colocated-with="people”>
<gfe:partition-resolver ref="partitionResolver"/>
</gfe:partitioned-region>
43© Copyright 2015 Pivotal. All rights reserved.
Send behavior to data
Client
Server 1
Server 2
Server 3
Joe: LOL!! Joe: LOL!!
EJ: arrgMaya: Hii
Jess: sup Jess: ok
Execution function
getSentiment
On Joe, Jess
Execute on Joe
Execute on Jess
44© Copyright 2015 Pivotal. All rights reserved.
Sample Function – Client Side
@Component
@OnRegion(region = "posts")
public interface FunctionClient {
public List<SentimentResult> getSentiment(@Filter Set<String> people);
}
45© Copyright 2015 Pivotal. All rights reserved.
Sample Function – Server Side
@GemfireFunction(HA=true)
public SentimentResult getSentiment(Region<PostId, Post> localPosts,
@Filter Set<String> personNames)
throws Exception {
String personName = personNames.iterator().next();
Collection<Post> posts = localPosts.query("id.person='" personName + "'");
String sentiment = sentimentAnalyzer.analyze(posts);
return new SentimentResult(sentiment, personName);
}
46© Copyright 2015 Pivotal. All rights reserved.
Sample Function – Server Side
@GemfireFunction(HA=true)
public SentimentResult getSentiment(Region<PostId, Post> localPosts,
@Filter Set<String> personNames)
throws Exception {
String personName = personNames.iterator().next();
Collection<Post> posts = localPosts.query("id.person='" personName + "'");
String sentiment = sentimentAnalyzer.analyze(posts);
return new SentimentResult(sentiment, personName);
}
47© Copyright 2015 Pivotal. All rights reserved.
Sample Function – Server Side
@GemfireFunction(HA=true)
public SentimentResult getSentiment(Region<PostId, Post> localPosts,
@Filter Set<String> personNames)
throws Exception {
String personName = personNames.iterator().next();
Collection<Post> posts = localPosts.query("id.person='" personName + "'");
String sentiment = sentimentAnalyzer.analyze(posts);
return new SentimentResult(sentiment, personName);
}
48© Copyright 2015 Pivotal. All rights reserved. 48© Copyright 2015 Pivotal. All rights reserved.
Demo
49© Copyright 2015 Pivotal. All rights reserved. 49© Copyright 2015 Pivotal. All rights reserved.
Design
• Persistence
• PDX Serialization
50© Copyright 2015 Pivotal. All rights reserved.
Persistence – Design Goals
 High availability
 Low latency, high throughput
 Durability
 Minimal impact to in memory speed
– Minimize IO operations
– Minimize locking
– Minimize deserialization
51© Copyright 2015 Pivotal. All rights reserved.
Persistence – Shared Nothing
Server 3Server 2
Bucket 3
*primary*
Bucket 1
secondary
Bucket 3
secondary
Bucket 2
*primary*
Server 1
Bucket 2
secondary
Bucket 1
*primary*
52© Copyright 2015 Pivotal. All rights reserved.
Persistence – Shared Nothing
Server 1 Server 3Server 2
Bucket 2
secondary
Bucket 1
*primary*
Bucket 3
*primary*
Bucket 1
secondary
Bucket 3
secondary
Bucket 2
*primary*
What if a server
crashes?
53© Copyright 2015 Pivotal. All rights reserved.
Persistence – Shared Nothing
Server 1 Server 3Server 2
Bucket 2
secondary
Bucket 1
*primary*
Bucket 3
*primary*
Bucket 1
*primary*
Bucket 3
secondary
Bucket 2
*primary*
Bucket 2
secondary
Primary
Failover
Restore
Redundancy
Bucket 1
secondary
54© Copyright 2015 Pivotal. All rights reserved.
Persistence – Shared Nothing
Server 1 Server 3Server 2
Bucket 2
secondary
Bucket 1
*primary*
Bucket 3
*primary*
Bucket 1
*primary*
Bucket 3
secondary
Bucket 2
*primary*
Bucket 2
secondary
Bucket 1
secondary
What if they all crash?
55© Copyright 2015 Pivotal. All rights reserved.
Persistence – Shared Nothing
Server 3Server 2
Bucket 3
*primary*
Bucket 1
*primary*
Bucket 3
secondary
Bucket 2
*primary*
Bucket 2
secondary
Bucket 1
secondary
Server 1
Bucket 2
secondary
Bucket 1
*primary*
Compare Persistent View
On Restart
56© Copyright 2015 Pivotal. All rights reserved.
Persistence – Shared Nothing
Server 1
Bucket 2
secondary
Bucket 1
*primary*
Server 3Server 2
Bucket 3
*primary*
Bucket 1
secondary
Bucket 3
secondary
Bucket 2
*primary*
Bucket 2
secondary
Bucket 1
secondary
Recover Latest
Data &
Rebalance
57© Copyright 2015 Pivotal. All rights reserved.
Modify
k1->v5
Create
k6->v6
Create
k1->v1
Create
k2->v2
Modify
k1->v3
Create
k4->v4
Modify
k1->v5
Create
k6->v6
Member
1
Persistence - Operation Logs
Put k6->v6
Oplog2.crf
Oplog1.crf
Append to
operation log
58© Copyright 2015 Pivotal. All rights reserved.
Modify
k1->v5
Create
k6->v6
Create
k1->v1
Modify
k1->v3
Member
1
Persistence - Compaction
Oplog2.crf
Oplog1.crf
Create
k2->v2
Create
k4->v4
Copy live
data forward
59© Copyright 2015 Pivotal. All rights reserved.
Modify
k1->v5
Create
k6->v6
Persistence - Compaction
Create
k2->v2
Create
k4->v4
Oplog2.crf
Member
1
Modify
k4->v7Oplog3.crf
Put k4->v7
60© Copyright 2015 Pivotal. All rights reserved. 60© Copyright 2013 Pivotal. All rights reserved.
Design
• Persistence
• PDX Serialization
61© Copyright 2015 Pivotal. All rights reserved.
Fixed or flexible schema?
id name age pet_id
{
id : 1,
name : “Fred”,
age : 42,
pet : {
name : “Barney”,
type : “dino”
}
}
OR
62© Copyright 2015 Pivotal. All rights reserved.
But how to serialize data?
http://blog.pivotal.io/pivotal/products/data-serialization-how-to-run-multiple-big-data-apps-at-once-with-gemfire
63© Copyright 2015 Pivotal. All rights reserved.
Portable Data eXchange
C#, C++, Java, JSON
| header | data |
| pdx | length | dsid | typeId | fields | offsets |
64© Copyright 2015 Pivotal. All rights reserved.
Efficient for queries
SELECT p.name from /Person p WHERE p.pet.type = “dino”
{
id : 1,
name : “Fred”,
age : 42,
pet : {
name : “Barney”,
type : “dino”
}
}
single field
deserialization
65© Copyright 2015 Pivotal. All rights reserved.
Easy to use
 Access from Java, C#, C++, JSON
 Domain objects not required
 Automatic type definition
– No IDL compiler or schema required
– No hand-coded read/write methods
66© Copyright 2015 Pivotal. All rights reserved.
Distributed type registry
Member A Member B
Distributed Type Definitions
Person p1 = …
region.put(“Fred”, p1);
67© Copyright 2015 Pivotal. All rights reserved.
Distributed type registry
Member A Member B
Distributed Type Definitions
Person p1 = …
region.put(“Fred”, p1);
automatic
definition
68© Copyright 2015 Pivotal. All rights reserved.
Distributed type registry
Member A Member B
Distributed Type Definitions
Person p1 = …
region.put(“Fred”, p1);
automatic
definition
69© Copyright 2015 Pivotal. All rights reserved.
Distributed type registry
Member A Member B
Distributed Type Definitions
Person p1 = …
region.put(“Fred”, p1);
automatic
definition
replicate serialized data
containing typeId
70© Copyright 2015 Pivotal. All rights reserved.
Schema evolution
PDX provides forwards and backwards
compatibility, no code required
Member A Member B
Distributed Type Definitions
v
1
v
2
Application
#2
Application
#1
v2 objects preserve data
from missing fields
v1 objects use default values
to fill in new fields
71© Copyright 2015 Pivotal. All rights reserved.
Questions?
 http://geode.incubator.apache.org
 dev@geode.incubator.apache.org
 user@geode.incubator.apache.org
 http://github.com/apache/incubator-geode
72© Copyright 2015 Pivotal. All rights reserved. 72© Copyright 2013 Pivotal. All rights reserved.
Bonus Content
73© Copyright 2015 Pivotal. All rights reserved. 73© Copyright 2015 Pivotal. All rights reserved.
Design
• Persistence
• Asynchronous Events
• Client Subscriptions
• PDX Serialization
74© Copyright 2015 Pivotal. All rights reserved.
Asynchronous Events – Design Goals
 High availability
 Low latency, high throughput
 Deliver events to a receiver without impacting the write path
75© Copyright 2015 Pivotal. All rights reserved.
Member 3
Member 1
Serial Queues
Member 2
LOL!!put
Primary Queue
Secondary Queue
Enqueue
sup
LOL!!
sup
sup
LOL!!
Replicate
76© Copyright 2015 Pivotal. All rights reserved.
Member 3
Member 1
Member 2
Serial Queues
sup
LOL!!
sup
LOL!!
AsyncEventListener
{
processBatch()
}
LOL!!
sup
Dispatch Events
from Primary
77© Copyright 2015 Pivotal. All rights reserved.
put
Secondary Queue
(Partition 1)
LOL!!
sup
sup
LOL!!
Primary Queue
(Partition 2)
Parallel Queues
Primary Queue
(Partition 1)
LOL!!
sup
sup
LOL!!
Word
Word
78© Copyright 2015 Pivotal. All rights reserved.
LOL!!
sup
sup
LOL!!
Parallel Queues
LOL!!
sup
sup
LOL!!
Word
Word
AsyncEventListener
{
processBatch()
}
AsyncEventListener
{
processBatch()
}
Parallel
Dispatch
79© Copyright 2015 Pivotal. All rights reserved. 79© Copyright 2013 Pivotal. All rights reserved.
Design
• Persistence
• Asynchronous Events
• Client Subscriptions
• PDX Serialization
80© Copyright 2015 Pivotal. All rights reserved.
“There are only two hard things in Computer Science: cache
invalidation, naming things, and off-by-one errors.”
– the internet
81© Copyright 2015 Pivotal. All rights reserved.
Client driver
 Intelligent – understands data distribution for single hop
network access
 Caching – can be configured to locally cache data for even
faster access
 Events – registerInterest() in keys to receive push
notifications
82© Copyright 2015 Pivotal. All rights reserved.
Client subscriptions
 Useful for refreshing client cache, pushing events (“topics”)
to multiple clients
 Highly available & scalable via in-memory replicated queues
 Events are ordered, at-least-once delivery
 Durable subscriptions, conflation optional
83© Copyright 2015 Pivotal. All rights reserved.
Client subscriptions
Member A Member B
region.registerInterest(“Fred”);
Client
Client
84© Copyright 2015 Pivotal. All rights reserved.
Client subscriptions
Member A Member B
Person p1 = …
region.put(“Fred”, p1);
region.registerInterest(“Fred”);
Client
Client
85© Copyright 2015 Pivotal. All rights reserved.
Client subscriptions
Member A Member B
Person p1 = …
region.put(“Fred”, p1);
region.registerInterest(“Fred”);
Client
Client
86© Copyright 2015 Pivotal. All rights reserved.
Client subscriptions
Member A Member B
Person p1 = …
region.put(“Fred”, p1);
region.registerInterest(“Fred”);
Client
Client
87© Copyright 2015 Pivotal. All rights reserved.
Client subscriptions
Member A Member B
Person p1 = …
region.put(“Fred”, p1);
region.registerInterest(“Fred”);
Client
Client

Weitere ähnliche Inhalte

Was ist angesagt?

GemFire In Memory Data Grid
GemFire In Memory Data GridGemFire In Memory Data Grid
GemFire In Memory Data GridDmitry Buzdin
 
Apache Geode Meetup, Cork, Ireland at CIT
Apache Geode Meetup, Cork, Ireland at CITApache Geode Meetup, Cork, Ireland at CIT
Apache Geode Meetup, Cork, Ireland at CITApache Geode
 
Getting Started with Apache Geode
Getting Started with Apache GeodeGetting Started with Apache Geode
Getting Started with Apache GeodeJohn Blum
 
Build your first Internet of Things app today with Open Source
Build your first Internet of Things app today with Open SourceBuild your first Internet of Things app today with Open Source
Build your first Internet of Things app today with Open SourceApache Geode
 
Scale Out Your Big Data Apps: The Latest on Pivotal GemFire and GemFire XD
Scale Out Your Big Data Apps: The Latest on Pivotal GemFire and GemFire XDScale Out Your Big Data Apps: The Latest on Pivotal GemFire and GemFire XD
Scale Out Your Big Data Apps: The Latest on Pivotal GemFire and GemFire XDVMware Tanzu
 
Apache Geode - The First Six Months
Apache Geode -  The First Six MonthsApache Geode -  The First Six Months
Apache Geode - The First Six MonthsAnthony Baker
 
Apache Geode Meetup, London
Apache Geode Meetup, LondonApache Geode Meetup, London
Apache Geode Meetup, LondonApache Geode
 
Building Apps with Distributed In-Memory Computing Using Apache Geode
Building Apps with Distributed In-Memory Computing Using Apache GeodeBuilding Apps with Distributed In-Memory Computing Using Apache Geode
Building Apps with Distributed In-Memory Computing Using Apache GeodePivotalOpenSourceHub
 
Hive LLAP: A High Performance, Cost-effective Alternative to Traditional MPP ...
Hive LLAP: A High Performance, Cost-effective Alternative to Traditional MPP ...Hive LLAP: A High Performance, Cost-effective Alternative to Traditional MPP ...
Hive LLAP: A High Performance, Cost-effective Alternative to Traditional MPP ...DataWorks Summit
 
Hive 3 - a new horizon
Hive 3 - a new horizonHive 3 - a new horizon
Hive 3 - a new horizonThejas Nair
 
YARN Containerized Services: Fading The Lines Between On-Prem And Cloud
YARN Containerized Services: Fading The Lines Between On-Prem And CloudYARN Containerized Services: Fading The Lines Between On-Prem And Cloud
YARN Containerized Services: Fading The Lines Between On-Prem And CloudDataWorks Summit
 
Gemfire
GemfireGemfire
GemfireFNian
 
Apache Geode (incubating) Introduction with Docker
Apache Geode (incubating) Introduction with DockerApache Geode (incubating) Introduction with Docker
Apache Geode (incubating) Introduction with DockerWilliam Markito Oliveira
 
SpringCamp 2016 - Apache Geode 와 Spring Data Gemfire
SpringCamp 2016 - Apache Geode 와 Spring Data GemfireSpringCamp 2016 - Apache Geode 와 Spring Data Gemfire
SpringCamp 2016 - Apache Geode 와 Spring Data GemfireJay Lee
 
Running Enterprise Workloads in the Cloud
Running Enterprise Workloads in the CloudRunning Enterprise Workloads in the Cloud
Running Enterprise Workloads in the CloudDataWorks Summit
 
How to Design for Database High Availability
How to Design for Database High AvailabilityHow to Design for Database High Availability
How to Design for Database High AvailabilityEDB
 

Was ist angesagt? (20)

ApexMeetup Geode - Talk1 2016-03-17
ApexMeetup Geode - Talk1 2016-03-17ApexMeetup Geode - Talk1 2016-03-17
ApexMeetup Geode - Talk1 2016-03-17
 
GemFire In Memory Data Grid
GemFire In Memory Data GridGemFire In Memory Data Grid
GemFire In Memory Data Grid
 
Apache Geode Meetup, Cork, Ireland at CIT
Apache Geode Meetup, Cork, Ireland at CITApache Geode Meetup, Cork, Ireland at CIT
Apache Geode Meetup, Cork, Ireland at CIT
 
Geode Meetup Apachecon
Geode Meetup ApacheconGeode Meetup Apachecon
Geode Meetup Apachecon
 
Getting Started with Apache Geode
Getting Started with Apache GeodeGetting Started with Apache Geode
Getting Started with Apache Geode
 
Build your first Internet of Things app today with Open Source
Build your first Internet of Things app today with Open SourceBuild your first Internet of Things app today with Open Source
Build your first Internet of Things app today with Open Source
 
Scale Out Your Big Data Apps: The Latest on Pivotal GemFire and GemFire XD
Scale Out Your Big Data Apps: The Latest on Pivotal GemFire and GemFire XDScale Out Your Big Data Apps: The Latest on Pivotal GemFire and GemFire XD
Scale Out Your Big Data Apps: The Latest on Pivotal GemFire and GemFire XD
 
Apache Geode - The First Six Months
Apache Geode -  The First Six MonthsApache Geode -  The First Six Months
Apache Geode - The First Six Months
 
Apache Geode Meetup, London
Apache Geode Meetup, LondonApache Geode Meetup, London
Apache Geode Meetup, London
 
Building Apps with Distributed In-Memory Computing Using Apache Geode
Building Apps with Distributed In-Memory Computing Using Apache GeodeBuilding Apps with Distributed In-Memory Computing Using Apache Geode
Building Apps with Distributed In-Memory Computing Using Apache Geode
 
Hive LLAP: A High Performance, Cost-effective Alternative to Traditional MPP ...
Hive LLAP: A High Performance, Cost-effective Alternative to Traditional MPP ...Hive LLAP: A High Performance, Cost-effective Alternative to Traditional MPP ...
Hive LLAP: A High Performance, Cost-effective Alternative to Traditional MPP ...
 
Database as a Service - Tutorial @ICDE 2010
Database as a Service - Tutorial @ICDE 2010Database as a Service - Tutorial @ICDE 2010
Database as a Service - Tutorial @ICDE 2010
 
Hive 3 - a new horizon
Hive 3 - a new horizonHive 3 - a new horizon
Hive 3 - a new horizon
 
YARN Containerized Services: Fading The Lines Between On-Prem And Cloud
YARN Containerized Services: Fading The Lines Between On-Prem And CloudYARN Containerized Services: Fading The Lines Between On-Prem And Cloud
YARN Containerized Services: Fading The Lines Between On-Prem And Cloud
 
Gemfire
GemfireGemfire
Gemfire
 
Apache Geode (incubating) Introduction with Docker
Apache Geode (incubating) Introduction with DockerApache Geode (incubating) Introduction with Docker
Apache Geode (incubating) Introduction with Docker
 
SpringCamp 2016 - Apache Geode 와 Spring Data Gemfire
SpringCamp 2016 - Apache Geode 와 Spring Data GemfireSpringCamp 2016 - Apache Geode 와 Spring Data Gemfire
SpringCamp 2016 - Apache Geode 와 Spring Data Gemfire
 
What's New in Apache Hive
What's New in Apache HiveWhat's New in Apache Hive
What's New in Apache Hive
 
Running Enterprise Workloads in the Cloud
Running Enterprise Workloads in the CloudRunning Enterprise Workloads in the Cloud
Running Enterprise Workloads in the Cloud
 
How to Design for Database High Availability
How to Design for Database High AvailabilityHow to Design for Database High Availability
How to Design for Database High Availability
 

Andere mochten auch

DSD-INT 2015 - Fast-data management toolbox using Open Earth building blocks ...
DSD-INT 2015 - Fast-data management toolbox using Open Earth building blocks ...DSD-INT 2015 - Fast-data management toolbox using Open Earth building blocks ...
DSD-INT 2015 - Fast-data management toolbox using Open Earth building blocks ...Deltares
 
China & the environment
China & the environmentChina & the environment
China & the environmentTrent_Ledford
 
Архитектура Apache Ignite .NET
Архитектура Apache Ignite .NETАрхитектура Apache Ignite .NET
Архитектура Apache Ignite .NETMikhail Shcherbakov
 
Building Wall St Risk Systems with Apache Geode
Building Wall St Risk Systems with Apache GeodeBuilding Wall St Risk Systems with Apache Geode
Building Wall St Risk Systems with Apache GeodeAndre Langevin
 
JBoss Community Introduction
JBoss Community IntroductionJBoss Community Introduction
JBoss Community Introductionjbugkorea
 
Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347
Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347
Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347Manik Surtani
 
Apache conbigdata2015 christiantzolov-federated sql on hadoop and beyond- lev...
Apache conbigdata2015 christiantzolov-federated sql on hadoop and beyond- lev...Apache conbigdata2015 christiantzolov-federated sql on hadoop and beyond- lev...
Apache conbigdata2015 christiantzolov-federated sql on hadoop and beyond- lev...Christian Tzolov
 
Infinispan Servers: Beyond peer-to-peer data grids
Infinispan Servers: Beyond peer-to-peer data gridsInfinispan Servers: Beyond peer-to-peer data grids
Infinispan Servers: Beyond peer-to-peer data gridsGalder Zamarreño
 
Infinspan: In-memory data grid meets NoSQL
Infinspan: In-memory data grid meets NoSQLInfinspan: In-memory data grid meets NoSQL
Infinspan: In-memory data grid meets NoSQLManik Surtani
 

Andere mochten auch (12)

DSD-INT 2015 - Fast-data management toolbox using Open Earth building blocks ...
DSD-INT 2015 - Fast-data management toolbox using Open Earth building blocks ...DSD-INT 2015 - Fast-data management toolbox using Open Earth building blocks ...
DSD-INT 2015 - Fast-data management toolbox using Open Earth building blocks ...
 
BILLY THE PUPPET
BILLY THE PUPPETBILLY THE PUPPET
BILLY THE PUPPET
 
Eyesus(1)
Eyesus(1)Eyesus(1)
Eyesus(1)
 
China & the environment
China & the environmentChina & the environment
China & the environment
 
Nosferatu
NosferatuNosferatu
Nosferatu
 
Архитектура Apache Ignite .NET
Архитектура Apache Ignite .NETАрхитектура Apache Ignite .NET
Архитектура Apache Ignite .NET
 
Building Wall St Risk Systems with Apache Geode
Building Wall St Risk Systems with Apache GeodeBuilding Wall St Risk Systems with Apache Geode
Building Wall St Risk Systems with Apache Geode
 
JBoss Community Introduction
JBoss Community IntroductionJBoss Community Introduction
JBoss Community Introduction
 
Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347
Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347
Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347
 
Apache conbigdata2015 christiantzolov-federated sql on hadoop and beyond- lev...
Apache conbigdata2015 christiantzolov-federated sql on hadoop and beyond- lev...Apache conbigdata2015 christiantzolov-federated sql on hadoop and beyond- lev...
Apache conbigdata2015 christiantzolov-federated sql on hadoop and beyond- lev...
 
Infinispan Servers: Beyond peer-to-peer data grids
Infinispan Servers: Beyond peer-to-peer data gridsInfinispan Servers: Beyond peer-to-peer data grids
Infinispan Servers: Beyond peer-to-peer data grids
 
Infinspan: In-memory data grid meets NoSQL
Infinspan: In-memory data grid meets NoSQLInfinspan: In-memory data grid meets NoSQL
Infinspan: In-memory data grid meets NoSQL
 

Ähnlich wie Open Sourcing GemFire - Apache Geode

Presentazione SimpliVity @ VMUGIT UserCon 2015
Presentazione SimpliVity @ VMUGIT UserCon 2015Presentazione SimpliVity @ VMUGIT UserCon 2015
Presentazione SimpliVity @ VMUGIT UserCon 2015VMUG IT
 
EMC Pivotal overview deck
EMC Pivotal overview deckEMC Pivotal overview deck
EMC Pivotal overview deckmister_moun
 
Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platform
Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platformPivotal deep dive_on_pivotal_hd_world_class_hdfs_platform
Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platformEMC
 
times ten in-memory database for extreme performance
times ten in-memory database for extreme performancetimes ten in-memory database for extreme performance
times ten in-memory database for extreme performanceOracle Korea
 
Embedded-ml(ai)applications - Bjoern Staender
Embedded-ml(ai)applications - Bjoern StaenderEmbedded-ml(ai)applications - Bjoern Staender
Embedded-ml(ai)applications - Bjoern StaenderDataconomy Media
 
Simplifying Real-Time Architectures for IoT with Apache Kudu
Simplifying Real-Time Architectures for IoT with Apache KuduSimplifying Real-Time Architectures for IoT with Apache Kudu
Simplifying Real-Time Architectures for IoT with Apache KuduCloudera, Inc.
 
There are 250 Database products, are you running the right one?
There are 250 Database products, are you running the right one?There are 250 Database products, are you running the right one?
There are 250 Database products, are you running the right one?Aerospike, Inc.
 
Times ten 18.1_overview_meetup
Times ten 18.1_overview_meetupTimes ten 18.1_overview_meetup
Times ten 18.1_overview_meetupByung Ho Lee
 
Does Big Data Spell Big Costs- Impetus Webinar
Does Big Data Spell Big Costs- Impetus WebinarDoes Big Data Spell Big Costs- Impetus Webinar
Does Big Data Spell Big Costs- Impetus WebinarImpetus Technologies
 
Best Practices for a Complete Postgres Enterprise Architecture Setup
Best Practices for a Complete Postgres Enterprise Architecture SetupBest Practices for a Complete Postgres Enterprise Architecture Setup
Best Practices for a Complete Postgres Enterprise Architecture SetupEDB
 
Beyond PowerPlay: Choose the Right OLAP Tool for Your BI Environment (Cognos...
 Beyond PowerPlay: Choose the Right OLAP Tool for Your BI Environment (Cognos... Beyond PowerPlay: Choose the Right OLAP Tool for Your BI Environment (Cognos...
Beyond PowerPlay: Choose the Right OLAP Tool for Your BI Environment (Cognos...Senturus
 
Hadoop and Manufacturing
Hadoop and ManufacturingHadoop and Manufacturing
Hadoop and ManufacturingCloudera, Inc.
 
Software Defined Infrastructure
Software Defined InfrastructureSoftware Defined Infrastructure
Software Defined Infrastructureinside-BigData.com
 
The role of NoSQL in the Next Generation of Financial Informatics
The role of NoSQL in the Next Generation of Financial InformaticsThe role of NoSQL in the Next Generation of Financial Informatics
The role of NoSQL in the Next Generation of Financial InformaticsAerospike, Inc.
 
NVMe and Flash – Make Your Storage Great Again!
NVMe and Flash – Make Your Storage Great Again!NVMe and Flash – Make Your Storage Great Again!
NVMe and Flash – Make Your Storage Great Again!DataCore Software
 
POWER8 the x86 Server Farm - IBM Business Partners use POWER8 to Lower Client...
POWER8 the x86 Server Farm - IBM Business Partners use POWER8 to Lower Client...POWER8 the x86 Server Farm - IBM Business Partners use POWER8 to Lower Client...
POWER8 the x86 Server Farm - IBM Business Partners use POWER8 to Lower Client...Paula Koziol
 
Oracle Big Data Appliance and Big Data SQL for advanced analytics
Oracle Big Data Appliance and Big Data SQL for advanced analyticsOracle Big Data Appliance and Big Data SQL for advanced analytics
Oracle Big Data Appliance and Big Data SQL for advanced analyticsjdijcks
 
Oracle databáze – Konsolidovaná Data Management Platforma
Oracle databáze – Konsolidovaná Data Management PlatformaOracle databáze – Konsolidovaná Data Management Platforma
Oracle databáze – Konsolidovaná Data Management PlatformaMarketingArrowECS_CZ
 
Oracle business continuity for virtualization and cloud infrastructure
Oracle business continuity for virtualization and cloud infrastructureOracle business continuity for virtualization and cloud infrastructure
Oracle business continuity for virtualization and cloud infrastructureOTN Systems Hub
 
Data core overview - haluk-final
Data core overview - haluk-finalData core overview - haluk-final
Data core overview - haluk-finalHaluk Ulubay
 

Ähnlich wie Open Sourcing GemFire - Apache Geode (20)

Presentazione SimpliVity @ VMUGIT UserCon 2015
Presentazione SimpliVity @ VMUGIT UserCon 2015Presentazione SimpliVity @ VMUGIT UserCon 2015
Presentazione SimpliVity @ VMUGIT UserCon 2015
 
EMC Pivotal overview deck
EMC Pivotal overview deckEMC Pivotal overview deck
EMC Pivotal overview deck
 
Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platform
Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platformPivotal deep dive_on_pivotal_hd_world_class_hdfs_platform
Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platform
 
times ten in-memory database for extreme performance
times ten in-memory database for extreme performancetimes ten in-memory database for extreme performance
times ten in-memory database for extreme performance
 
Embedded-ml(ai)applications - Bjoern Staender
Embedded-ml(ai)applications - Bjoern StaenderEmbedded-ml(ai)applications - Bjoern Staender
Embedded-ml(ai)applications - Bjoern Staender
 
Simplifying Real-Time Architectures for IoT with Apache Kudu
Simplifying Real-Time Architectures for IoT with Apache KuduSimplifying Real-Time Architectures for IoT with Apache Kudu
Simplifying Real-Time Architectures for IoT with Apache Kudu
 
There are 250 Database products, are you running the right one?
There are 250 Database products, are you running the right one?There are 250 Database products, are you running the right one?
There are 250 Database products, are you running the right one?
 
Times ten 18.1_overview_meetup
Times ten 18.1_overview_meetupTimes ten 18.1_overview_meetup
Times ten 18.1_overview_meetup
 
Does Big Data Spell Big Costs- Impetus Webinar
Does Big Data Spell Big Costs- Impetus WebinarDoes Big Data Spell Big Costs- Impetus Webinar
Does Big Data Spell Big Costs- Impetus Webinar
 
Best Practices for a Complete Postgres Enterprise Architecture Setup
Best Practices for a Complete Postgres Enterprise Architecture SetupBest Practices for a Complete Postgres Enterprise Architecture Setup
Best Practices for a Complete Postgres Enterprise Architecture Setup
 
Beyond PowerPlay: Choose the Right OLAP Tool for Your BI Environment (Cognos...
 Beyond PowerPlay: Choose the Right OLAP Tool for Your BI Environment (Cognos... Beyond PowerPlay: Choose the Right OLAP Tool for Your BI Environment (Cognos...
Beyond PowerPlay: Choose the Right OLAP Tool for Your BI Environment (Cognos...
 
Hadoop and Manufacturing
Hadoop and ManufacturingHadoop and Manufacturing
Hadoop and Manufacturing
 
Software Defined Infrastructure
Software Defined InfrastructureSoftware Defined Infrastructure
Software Defined Infrastructure
 
The role of NoSQL in the Next Generation of Financial Informatics
The role of NoSQL in the Next Generation of Financial InformaticsThe role of NoSQL in the Next Generation of Financial Informatics
The role of NoSQL in the Next Generation of Financial Informatics
 
NVMe and Flash – Make Your Storage Great Again!
NVMe and Flash – Make Your Storage Great Again!NVMe and Flash – Make Your Storage Great Again!
NVMe and Flash – Make Your Storage Great Again!
 
POWER8 the x86 Server Farm - IBM Business Partners use POWER8 to Lower Client...
POWER8 the x86 Server Farm - IBM Business Partners use POWER8 to Lower Client...POWER8 the x86 Server Farm - IBM Business Partners use POWER8 to Lower Client...
POWER8 the x86 Server Farm - IBM Business Partners use POWER8 to Lower Client...
 
Oracle Big Data Appliance and Big Data SQL for advanced analytics
Oracle Big Data Appliance and Big Data SQL for advanced analyticsOracle Big Data Appliance and Big Data SQL for advanced analytics
Oracle Big Data Appliance and Big Data SQL for advanced analytics
 
Oracle databáze – Konsolidovaná Data Management Platforma
Oracle databáze – Konsolidovaná Data Management PlatformaOracle databáze – Konsolidovaná Data Management Platforma
Oracle databáze – Konsolidovaná Data Management Platforma
 
Oracle business continuity for virtualization and cloud infrastructure
Oracle business continuity for virtualization and cloud infrastructureOracle business continuity for virtualization and cloud infrastructure
Oracle business continuity for virtualization and cloud infrastructure
 
Data core overview - haluk-final
Data core overview - haluk-finalData core overview - haluk-final
Data core overview - haluk-final
 

Kürzlich hochgeladen

Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesZilliz
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
The Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfThe Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfSeasiaInfotech2
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 

Kürzlich hochgeladen (20)

Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector Databases
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
The Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfThe Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdf
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 

Open Sourcing GemFire - Apache Geode

  • 1. A NEW PLATFORM FOR A NEW ERA
  • 2. 2© Copyright 2015 Pivotal. All rights reserved. 2© Copyright 2015 Pivotal. All rights reserved. Open Sourcing GemFire An Introduction to Apache Geode (incubating) The open source distributed in-memory database Anthony Baker abaker@apache.org @metatype Dan Smith dsmith@pivotal.io
  • 3. 3© Copyright 2015 Pivotal. All rights reserved. Topics  GemFire history, architecture, and use cases  Geode, the open source core of GemFire  Example application and demo  Technical deep dive
  • 4. 4© Copyright 2015 Pivotal. All rights reserved. It’s all about the DATAand how you use it
  • 5. 5© Copyright 2015 Pivotal. All rights reserved. 2004 2008 2014 • Massive increase in data volumes • Falling margins per transaction • Increasing cost of IT maintenance • Need for elasticity in systems • Financial Services Providers (every major Wall Street bank) • Department of Defense • Real Time response needs • Time to market constraints • Need for flexible data models across enterprise • Distributed development • Persistence + In-memory • Global data visibility needs • Fast Ingest needs for data • Need to allow devices to hook into enterprise data • Always on • Largest travel Portal • Airlines • Trade clearing • Online gambling • Largest Telcos • Large mfrers • Largest Payroll processor • Auto insurance giants • Largest rail systems on earth Hybrid Transactional /Analytics grids Our GemFire Journey Over The Years
  • 6. 6© Copyright 2015 Pivotal. All rights reserved. Apps at Scale Have Unique Needs Pivotal GemFire is the distributed, in-memory database for apps that need: 1. Elastic scale-out performance 2. High performance database capabilities in distributed systems 3. Mission critical availability and resiliency 4. Flexibility for developers to create unique applications 5. Easy administration of distributed data grids
  • 7. 7© Copyright 2015 Pivotal. All rights reserved. 1. Elastic Scale-Out Performance Nodes Ops / SecLinear scalability Elastic capacity +/- Latency-minimizing data distribution China Railway Corporation “The system is operating with solid performance and uptime. Now, we have a reliable, economically sound production system that supports record volumes and has room to grow” Dr. Jiansheng Zhu, Vice Director of China Academy of Railway Sciences • 4.5 million ticket purchases & 20 million users per day. • Spikes of 15,000 tickets sold per minute, 40,000 visits per second.
  • 8. 8© Copyright 2015 Pivotal. All rights reserved. 2. High Performance Database Capabilities Performance- optimized persistence Configurable consistency Partitioned Replicated Disabled Distributed & continuous queries Distributed transactions, indexing, archival Newedge “Our global deployment of Pivotal GemFire’s distributed cache gives me a single version of the trade – resolving hard-to-test-for synchronization issues that exist within any globally distributed business application architecture” Michael Benillouche, Global Head of Data Management
  • 9. 9© Copyright 2015 Pivotal. All rights reserved. 3. Mission Critical Availability and Resiliency Cluster to cluster WAN connectivity Cluster resilience & failover XX Gire “We can track and collect money at our 4,000+ kiosks and branches – even without a reliable Internet connection. GemFire provides the core data grid and a significant amount of related functionality to help us handle this unreliable network problem” Gustavo Valdez, Chief of Architecture and Development • 19 million payment transactions per month • 4000+ points of sale with intermittent Internet connectivity
  • 10. 10© Copyright 2015 Pivotal. All rights reserved. 4. Flexibility for Developers to Create Unique Apps • Data Structures: – User-defined classes – Documents (JSON) • Native language support: – Java, C++, C# • API’s – Java: Hashmap – Memcache – Spring Data GemFire – C++ Serialization API’s • Embedded query authoring – Object Query Language (OQL) • Publish & subscribe framework for continous query & reliable asynchronous event queues • App-server embedded functionality: – Web app session state caching – L2 Hibernate
  • 11. 11© Copyright 2015 Pivotal. All rights reserved. 5. Easy Administration of Distributed Data Grids • Auto tuning of distributed computing resources to optimize performance • Cluster monitoring dashboard – Cluster and node status & performance • Offline performance statistics analysis tool – View historical logs and events to diagnose performance and resource bottlenecks • Command-line tool for taking action on clusters and nodes
  • 12. 12© Copyright 2015 Pivotal. All rights reserved. What makes it fast?  Design center is RAM not HDD  Really demanding customers  Avoid and minimize, particularly on the critical read/write paths: – Network hops, copying data, contention, distributed locking, disk seeks, garbage  Lots and lots of testing – Establish and monitor performance baselines – Distributed systems testing is difficult!
  • 13. 13© Copyright 2015 Pivotal. All rights reserved. Horizontal Scaling for GemFire (Geode) Reads With Consistent Latency and CPU • Scaled from 256 clients and 2 servers to 1280 clients and 10 servers • Partitioned region with redundancy and 1K data size 0 2 4 6 8 10 12 14 16 18 0 1 2 3 4 5 6 2 4 6 8 10 Speedup Server Hosts speedup latency (ms) CPU %
  • 14. 14© Copyright 2015 Pivotal. All rights reserved. GemFire (Geode) 3.5-4.5X Faster Than Cassandra for YCSB
  • 15. 15© Copyright 2015 Pivotal. All rights reserved. Roadmap  HDFS persistence  Off-heap storage  Lucene indexes  Spark integration  Cloud Foundry service …and other ideas from the Geode community!
  • 16. 16© Copyright 2015 Pivotal. All rights reserved. 16© Copyright 2015 Pivotal. All rights reserved. Use cases and patterns
  • 17. 17© Copyright 2015 Pivotal. All rights reserved. “Low touch” Usage Patterns Simple template for TCServer, TC, App servers Shared nothing persistence, Global session state HTTP Session management Set Cache in hibernate.cfg.xml Support for query and entity caching Hibernate L2 Cache plugin Servers understand the memcached wire protocol Use any memcached client Memcached protocol <bean id="cacheManager" class="org.springframework.data.gemfire.support.GemfireCacheManager"Spring Cache Abstraction
  • 18. 18© Copyright 2015 Pivotal. All rights reserved. Application Patterns  Caching for speed and scale – Read-through, Write-through, Write-behind  Geode as the OLTP system of record – Data in-memory for low latency, on disk for durability  Parallel compute engine  Real-time analytics
  • 19. 19© Copyright 2015 Pivotal. All rights reserved. Development Patterns  Configure the cluster programmatically or declaratively using cache.xml, gfsh, or SpringDataGemFire <cache> <region name="turbineSensorData" refid="PARTITION_PERSISTENT"> <partition-attributes redundant-copies="1" total-num-buckets="43"/> </region> </cache>
  • 20. 20© Copyright 2015 Pivotal. All rights reserved. Development Patterns  Write key-value data into a Region using { create | put | putAll | remove } – The value can be flat data, nested objects, JSON, … Region sensorData = cache.getRegion("TurbineSensorData"); SensorKey key = new SensorKey(31415926, "2013-05-19T19:22Z"); // turbineId, timestamp sensorData.put(key, new TurbineReading() .setAmbientTemp(75) .setOperatingTemp(80) .setWindDirection(0) .setWindSpeed(30) .setPowerOutput(5000) .setRPM(5));
  • 21. 21© Copyright 2015 Pivotal. All rights reserved. Development Patterns  Read values from a Region by key using { get | getAll } TurbineReading data = sensorData.get(new SensorKey(31415926,"2013-05- 19T19:22Z"));  Query values using OQL // finds all sensor readings for the given turbine SELECT * from /TurbineSensorData.entrySet WHERE key.turbineId = 31415926 // finds all sensor readings where the operating temp exceeds a threshold SELECT * from /TurbineSensorData.entrySet WHERE value.operatingTemp > 120  Apply indexes to optimize queries
  • 22. 22© Copyright 2015 Pivotal. All rights reserved. Development Patterns  Execute functions to operate on local data in parallel  Respond to updates using CacheListeners and Events  Automatic redundancy, partitioning, distribution, consistency, network partition detection & recovery, load balancing, …
  • 23. 23© Copyright 2015 Pivotal. All rights reserved. 23© Copyright 2015 Pivotal. All rights reserved. Apache Geode (incubating) The open source core of GemFire
  • 24. 24© Copyright 2015 Pivotal. All rights reserved. Why OSS? Why Now? Why Apache?  Open Source Software is fundamentally changing buying patterns – Developers have to endorse product selection (No longer CIO handshake) – Community endorsement is key to product visibility – Open source credentials attract the best developers – Vendor credibility directly tied to street credibility of product  Align with the tides of history – Customers increasingly asking to participate in product development – Allow product development to happen with full transparency  Apache is where you go to build Open Source street cred – Transparent, meritocracy which puts developers in charge – Roman keeps shouting “Apache!” every few hours
  • 25. 25© Copyright 2015 Pivotal. All rights reserved. Geode Will Be A Significant Apache Project  Over a 1000 person years invested into cutting edge R&D  Thousands of production customers in very demanding verticals  Cutting edge use cases that have shaped product thinking  Tens of thousands of distributed , scaled up tests that can randomize every aspect of the product  A core technology team that has stayed together since founding  Performance differentiators that are baked into every aspect of the product
  • 26. 26© Copyright 2015 Pivotal. All rights reserved. Geode or GemFire?  Geode is a project, GemFire is a product  We donated everything but the kitchen sink*  More code drops imminent; going forward all development happens OSS-style (“The Apache Way”) * Multi-site WAN replication, continuous queries, and native (C/C++) client driver
  • 27. 27© Copyright 2015 Pivotal. All rights reserved.
  • 28. 28© Copyright 2015 Pivotal. All rights reserved. X Apache
  • 29. 29© Copyright 2015 Pivotal. All rights reserved.
  • 30. 30© Copyright 2015 Pivotal. All rights reserved. 30© Copyright 2015 Pivotal. All rights reserved. Geode Demo
  • 31. 31© Copyright 2015 Pivotal. All rights reserved. Post Region Partitioned People Region Partitioned Social Network Person Name: String Description:String Post Id: PostId(name, date) Text: String
  • 32. 32© Copyright 2015 Pivotal. All rights reserved. Partition put Client Server 1 Server 2 Server 3 Bucket 1 Bucket 1 Bucket 2 Bucket 2 #(LOL)=1 Put LOL
  • 33. 33© Copyright 2015 Pivotal. All rights reserved. Partition put Client Server 1 Server 2 Server 3 LOL LOL Bucket 2 Bucket 2 Replicate To Secondary
  • 34. 34© Copyright 2015 Pivotal. All rights reserved. “User” Use Case – Save Objects public interface PersonRepository extends CrudRepository<Person, String> { } @Autowired PersonRepository people; public static void main(String[] args) { people.save(new Person(name)); posts.save(new Post(new PostId(name, date), text)); } Nested Objects, Compound Keys
  • 35. 35© Copyright 2015 Pivotal. All rights reserved. “User” Use Case – Save Objects public interface PersonRepository extends CrudRepository<Person, String> { } @Autowired PersonRepository people; public static void main(String[] args) { people.save(new Person(name)); posts.save(new Post(new PostId(name, date), text)); } Automatically Serialized With PDX
  • 36. 36© Copyright 2015 Pivotal. All rights reserved. Configuration <gfe:partitioned-region id="people" copies="1"/> <bean id="pdxSerializer” class="com.gemstone.gemfire.pdx.ReflectionBasedAutoSerializer"> <constructor-arg value="io.pivotal.happysocial.model.*"/> </bean> <gfe:cache pdx-serializer-ref="pdxSerializer"/>
  • 37. 37© Copyright 2015 Pivotal. All rights reserved. Data Analyst – Determine Sentiment  Find all of the posts for a user  Analyze their content
  • 38. 38© Copyright 2015 Pivotal. All rights reserved. First try – Just use a Query public interface PostRepository extends GemfireRepository<Post, PostId> { @Query("select * from /posts where id.person=$1") public Collection<Post> findPosts(String personName); } Collection<Post> posts = postRepository.findPosts(personName); String sentiment = sentimentAnalyzer.analyze(posts);
  • 39. 39© Copyright 2015 Pivotal. All rights reserved. First try – Just use a Query public interface PostRepository extends GemfireRepository<Post, PostId> { @Query("select * from /posts where id.person=$1") public Collection<Post> findPosts(String personName); } Collection<Post> posts = postRepository.findPosts(personName); String sentiment = sentimentAnalyzer.analyze(posts); Query Nested Objects
  • 40. 40© Copyright 2015 Pivotal. All rights reserved. Use an Index <gfe:index id="postAuthor" expression="id.person" from="/posts” />
  • 41. 41© Copyright 2015 Pivotal. All rights reserved. Still could be more efficient Client Server 1 Server 2 Server 3 Joe: LOL!! Joe: LOL!! EJ: arrg Maya: Hii Jess: sup Jess: ok Hitting multiple Nodes Bringing too much Data to the client
  • 42. 42© Copyright 2015 Pivotal. All rights reserved. Colocate the data Client Server 1 Server 2 Server 3 Joe: LOL!! Joe: LOL!! EJ: arrgMaya: Hii Jess: sup Jess: ok <gfe:partitioned-region id="posts" copies="1" colocated-with="people”> <gfe:partition-resolver ref="partitionResolver"/> </gfe:partitioned-region>
  • 43. 43© Copyright 2015 Pivotal. All rights reserved. Send behavior to data Client Server 1 Server 2 Server 3 Joe: LOL!! Joe: LOL!! EJ: arrgMaya: Hii Jess: sup Jess: ok Execution function getSentiment On Joe, Jess Execute on Joe Execute on Jess
  • 44. 44© Copyright 2015 Pivotal. All rights reserved. Sample Function – Client Side @Component @OnRegion(region = "posts") public interface FunctionClient { public List<SentimentResult> getSentiment(@Filter Set<String> people); }
  • 45. 45© Copyright 2015 Pivotal. All rights reserved. Sample Function – Server Side @GemfireFunction(HA=true) public SentimentResult getSentiment(Region<PostId, Post> localPosts, @Filter Set<String> personNames) throws Exception { String personName = personNames.iterator().next(); Collection<Post> posts = localPosts.query("id.person='" personName + "'"); String sentiment = sentimentAnalyzer.analyze(posts); return new SentimentResult(sentiment, personName); }
  • 46. 46© Copyright 2015 Pivotal. All rights reserved. Sample Function – Server Side @GemfireFunction(HA=true) public SentimentResult getSentiment(Region<PostId, Post> localPosts, @Filter Set<String> personNames) throws Exception { String personName = personNames.iterator().next(); Collection<Post> posts = localPosts.query("id.person='" personName + "'"); String sentiment = sentimentAnalyzer.analyze(posts); return new SentimentResult(sentiment, personName); }
  • 47. 47© Copyright 2015 Pivotal. All rights reserved. Sample Function – Server Side @GemfireFunction(HA=true) public SentimentResult getSentiment(Region<PostId, Post> localPosts, @Filter Set<String> personNames) throws Exception { String personName = personNames.iterator().next(); Collection<Post> posts = localPosts.query("id.person='" personName + "'"); String sentiment = sentimentAnalyzer.analyze(posts); return new SentimentResult(sentiment, personName); }
  • 48. 48© Copyright 2015 Pivotal. All rights reserved. 48© Copyright 2015 Pivotal. All rights reserved. Demo
  • 49. 49© Copyright 2015 Pivotal. All rights reserved. 49© Copyright 2015 Pivotal. All rights reserved. Design • Persistence • PDX Serialization
  • 50. 50© Copyright 2015 Pivotal. All rights reserved. Persistence – Design Goals  High availability  Low latency, high throughput  Durability  Minimal impact to in memory speed – Minimize IO operations – Minimize locking – Minimize deserialization
  • 51. 51© Copyright 2015 Pivotal. All rights reserved. Persistence – Shared Nothing Server 3Server 2 Bucket 3 *primary* Bucket 1 secondary Bucket 3 secondary Bucket 2 *primary* Server 1 Bucket 2 secondary Bucket 1 *primary*
  • 52. 52© Copyright 2015 Pivotal. All rights reserved. Persistence – Shared Nothing Server 1 Server 3Server 2 Bucket 2 secondary Bucket 1 *primary* Bucket 3 *primary* Bucket 1 secondary Bucket 3 secondary Bucket 2 *primary* What if a server crashes?
  • 53. 53© Copyright 2015 Pivotal. All rights reserved. Persistence – Shared Nothing Server 1 Server 3Server 2 Bucket 2 secondary Bucket 1 *primary* Bucket 3 *primary* Bucket 1 *primary* Bucket 3 secondary Bucket 2 *primary* Bucket 2 secondary Primary Failover Restore Redundancy Bucket 1 secondary
  • 54. 54© Copyright 2015 Pivotal. All rights reserved. Persistence – Shared Nothing Server 1 Server 3Server 2 Bucket 2 secondary Bucket 1 *primary* Bucket 3 *primary* Bucket 1 *primary* Bucket 3 secondary Bucket 2 *primary* Bucket 2 secondary Bucket 1 secondary What if they all crash?
  • 55. 55© Copyright 2015 Pivotal. All rights reserved. Persistence – Shared Nothing Server 3Server 2 Bucket 3 *primary* Bucket 1 *primary* Bucket 3 secondary Bucket 2 *primary* Bucket 2 secondary Bucket 1 secondary Server 1 Bucket 2 secondary Bucket 1 *primary* Compare Persistent View On Restart
  • 56. 56© Copyright 2015 Pivotal. All rights reserved. Persistence – Shared Nothing Server 1 Bucket 2 secondary Bucket 1 *primary* Server 3Server 2 Bucket 3 *primary* Bucket 1 secondary Bucket 3 secondary Bucket 2 *primary* Bucket 2 secondary Bucket 1 secondary Recover Latest Data & Rebalance
  • 57. 57© Copyright 2015 Pivotal. All rights reserved. Modify k1->v5 Create k6->v6 Create k1->v1 Create k2->v2 Modify k1->v3 Create k4->v4 Modify k1->v5 Create k6->v6 Member 1 Persistence - Operation Logs Put k6->v6 Oplog2.crf Oplog1.crf Append to operation log
  • 58. 58© Copyright 2015 Pivotal. All rights reserved. Modify k1->v5 Create k6->v6 Create k1->v1 Modify k1->v3 Member 1 Persistence - Compaction Oplog2.crf Oplog1.crf Create k2->v2 Create k4->v4 Copy live data forward
  • 59. 59© Copyright 2015 Pivotal. All rights reserved. Modify k1->v5 Create k6->v6 Persistence - Compaction Create k2->v2 Create k4->v4 Oplog2.crf Member 1 Modify k4->v7Oplog3.crf Put k4->v7
  • 60. 60© Copyright 2015 Pivotal. All rights reserved. 60© Copyright 2013 Pivotal. All rights reserved. Design • Persistence • PDX Serialization
  • 61. 61© Copyright 2015 Pivotal. All rights reserved. Fixed or flexible schema? id name age pet_id { id : 1, name : “Fred”, age : 42, pet : { name : “Barney”, type : “dino” } } OR
  • 62. 62© Copyright 2015 Pivotal. All rights reserved. But how to serialize data? http://blog.pivotal.io/pivotal/products/data-serialization-how-to-run-multiple-big-data-apps-at-once-with-gemfire
  • 63. 63© Copyright 2015 Pivotal. All rights reserved. Portable Data eXchange C#, C++, Java, JSON | header | data | | pdx | length | dsid | typeId | fields | offsets |
  • 64. 64© Copyright 2015 Pivotal. All rights reserved. Efficient for queries SELECT p.name from /Person p WHERE p.pet.type = “dino” { id : 1, name : “Fred”, age : 42, pet : { name : “Barney”, type : “dino” } } single field deserialization
  • 65. 65© Copyright 2015 Pivotal. All rights reserved. Easy to use  Access from Java, C#, C++, JSON  Domain objects not required  Automatic type definition – No IDL compiler or schema required – No hand-coded read/write methods
  • 66. 66© Copyright 2015 Pivotal. All rights reserved. Distributed type registry Member A Member B Distributed Type Definitions Person p1 = … region.put(“Fred”, p1);
  • 67. 67© Copyright 2015 Pivotal. All rights reserved. Distributed type registry Member A Member B Distributed Type Definitions Person p1 = … region.put(“Fred”, p1); automatic definition
  • 68. 68© Copyright 2015 Pivotal. All rights reserved. Distributed type registry Member A Member B Distributed Type Definitions Person p1 = … region.put(“Fred”, p1); automatic definition
  • 69. 69© Copyright 2015 Pivotal. All rights reserved. Distributed type registry Member A Member B Distributed Type Definitions Person p1 = … region.put(“Fred”, p1); automatic definition replicate serialized data containing typeId
  • 70. 70© Copyright 2015 Pivotal. All rights reserved. Schema evolution PDX provides forwards and backwards compatibility, no code required Member A Member B Distributed Type Definitions v 1 v 2 Application #2 Application #1 v2 objects preserve data from missing fields v1 objects use default values to fill in new fields
  • 71. 71© Copyright 2015 Pivotal. All rights reserved. Questions?  http://geode.incubator.apache.org  dev@geode.incubator.apache.org  user@geode.incubator.apache.org  http://github.com/apache/incubator-geode
  • 72. 72© Copyright 2015 Pivotal. All rights reserved. 72© Copyright 2013 Pivotal. All rights reserved. Bonus Content
  • 73. 73© Copyright 2015 Pivotal. All rights reserved. 73© Copyright 2015 Pivotal. All rights reserved. Design • Persistence • Asynchronous Events • Client Subscriptions • PDX Serialization
  • 74. 74© Copyright 2015 Pivotal. All rights reserved. Asynchronous Events – Design Goals  High availability  Low latency, high throughput  Deliver events to a receiver without impacting the write path
  • 75. 75© Copyright 2015 Pivotal. All rights reserved. Member 3 Member 1 Serial Queues Member 2 LOL!!put Primary Queue Secondary Queue Enqueue sup LOL!! sup sup LOL!! Replicate
  • 76. 76© Copyright 2015 Pivotal. All rights reserved. Member 3 Member 1 Member 2 Serial Queues sup LOL!! sup LOL!! AsyncEventListener { processBatch() } LOL!! sup Dispatch Events from Primary
  • 77. 77© Copyright 2015 Pivotal. All rights reserved. put Secondary Queue (Partition 1) LOL!! sup sup LOL!! Primary Queue (Partition 2) Parallel Queues Primary Queue (Partition 1) LOL!! sup sup LOL!! Word Word
  • 78. 78© Copyright 2015 Pivotal. All rights reserved. LOL!! sup sup LOL!! Parallel Queues LOL!! sup sup LOL!! Word Word AsyncEventListener { processBatch() } AsyncEventListener { processBatch() } Parallel Dispatch
  • 79. 79© Copyright 2015 Pivotal. All rights reserved. 79© Copyright 2013 Pivotal. All rights reserved. Design • Persistence • Asynchronous Events • Client Subscriptions • PDX Serialization
  • 80. 80© Copyright 2015 Pivotal. All rights reserved. “There are only two hard things in Computer Science: cache invalidation, naming things, and off-by-one errors.” – the internet
  • 81. 81© Copyright 2015 Pivotal. All rights reserved. Client driver  Intelligent – understands data distribution for single hop network access  Caching – can be configured to locally cache data for even faster access  Events – registerInterest() in keys to receive push notifications
  • 82. 82© Copyright 2015 Pivotal. All rights reserved. Client subscriptions  Useful for refreshing client cache, pushing events (“topics”) to multiple clients  Highly available & scalable via in-memory replicated queues  Events are ordered, at-least-once delivery  Durable subscriptions, conflation optional
  • 83. 83© Copyright 2015 Pivotal. All rights reserved. Client subscriptions Member A Member B region.registerInterest(“Fred”); Client Client
  • 84. 84© Copyright 2015 Pivotal. All rights reserved. Client subscriptions Member A Member B Person p1 = … region.put(“Fred”, p1); region.registerInterest(“Fred”); Client Client
  • 85. 85© Copyright 2015 Pivotal. All rights reserved. Client subscriptions Member A Member B Person p1 = … region.put(“Fred”, p1); region.registerInterest(“Fred”); Client Client
  • 86. 86© Copyright 2015 Pivotal. All rights reserved. Client subscriptions Member A Member B Person p1 = … region.put(“Fred”, p1); region.registerInterest(“Fred”); Client Client
  • 87. 87© Copyright 2015 Pivotal. All rights reserved. Client subscriptions Member A Member B Person p1 = … region.put(“Fred”, p1); region.registerInterest(“Fred”); Client Client