5. Typical Web Architecture
Users • Need to get routed to same
machine (i.e. sticky sessions)
Web • Each machine round trips for
data
• Data that is cached is stored
Tier in the memory of one server
Data • CPU and disk can get
saturated due to traffic
Tier • Service access is slow
6. Web Explosion
Web
Site’s too
Where did slow!!
my
shopping
IIS/ASP.NET IIS/ASP.NET IIS/ASP.NET
cart go? Application Application Application
Servers are
crashing
Database
Services Database is
hot!!
are slow
9. Good but…
• Cache is scoped to machine / process
– Machines die
– Processes recycle
• Cache memory is limited
10. What if?
• You could have as much cache as you wanted?
• You could share a giant cache across
servers, services and even clients?
• What if this was something you could simply
add to the platform for 1free?
1Some features may require certain
editions of Windows Server
11. Windows Server AppFabric
AppFabric
CACHING WORKFLOW HOSTING
MONITORING SERVICE HOSTING
HIGH AVAILABILITY SCALE OUT MANAGEMENT
AppFabric Cache – Formerly known as Code Name “Velocity”
12. What is AppFabric Caching?
• An explicit, distributed, in-memory application cache for all
kinds of data
Caching clients
can be across
machines or
processes
Clients Access
the Cache as if it
Unified Cache View was a large
single cache
Cache Layer
distributes data
across the
various cache
nodes
13. AppFabric Cache
Users • No need for sticky sessions
Web • Cached data available to
all servers
Tier
Caching • Multiple machines means
scale and high-availability
Tier • Lots of cache memory
Data • Reduces load on database
• Duplicate Service Calls
Tier eliminated
14. Data Distribution - Partitioned Cache
ASP.Net App
Caching Client
ASP.Net App
Caching Client
… ASP.Net App
Caching Client
Web Tier
A B C D E F G H I
Cache Service
Cache Service
Cache Service
Cache
D I C G A F BH E
Tier
Scale on Data Size - More machines => More memory to cache
Scale on Cache Throughput - More machines => keys distributed across
more machines => better throughput
15. Scale Test Output
Load
Throughput
1 Cache Server
As load increases,
throughput fails
to scale
latency increases
Latency
Caching
Tier
16. Add Second Cache Server
Load
Load Max
Throughput Throughput
increases
Latency
decreases
Latency
Caching
Tier
22. Usage Pattern – Cache Aside
(Explicit Caching)
// Read from Cache
Toy toyObj = (Toy) Application
catalog.Get("toy-101"); Caching Access Layer
// If Not present in the cache
if (toyObj == null)
Caching Service
{
// Read from backend..
toyObj = ReadFromDatabase();
// Populate Cache
catalog.Put("toy-101", toyObj);
Database
return toyObj;
}
23. Administration
• PowerShell cmdlets are used to administer the
cache cluster
• Rich set of cmdlets for
– Cache cluster management
– Cache creation and monitoring
27. Security
• Domain Based Security Option
– Domain Account / Local Account based Authentication
– Only authorized servers can join the cluster
– Only authorized clients can connect to the cluster
grant-cacheallowedclientaccount RedDomainMachine1$
grant-cacheallowedclientaccount RedDomainJohn
• Transport Level Security
– Turn on/off Signing or Encryption
• Can turn off Cache Security
– Use Firewalls, IPSec, VLANs to protect cache
28. AppFabric Caching Logical
AppFabric AppFabric AppFabric
Hierarchy AppFabric
Caching Caching Caching Caching
Service Service Service Service
Named Cache : Product Catalog
Named Cache : Electronics Inventory
Regions Key Payload Tags
Region A 121 xxxx “Toy” “Child”
123 yyyy “Toy” “Chair”..
Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects
• Host • Cache Item
– Physical processes hosting AppFabric – Key, Payload (Object ), Tags, TTL,
Caching instance. Timestamps, Version
• Named Caches • Regions
– Can span across machines – Physically co-located Container of
– Defined in the configuration file Cache Items
– May be implicit or explicitly created
29. AppFabric Caching API
// Create instance of cachefactory (reads appconfig)
DataCacheFactory fac = new DataCacheFactory();
// Get a named cache from the factory
DataCache catalog = fac.GetCache("catalogcache");
// Simple Get/Put
catalog.Put("toy-101", new Toy("Puzzle", .,.));
// From the same or a different client
Toy toyObj = (Toy)catalog.Get("toy-101");
// Region based Get/Put
catalog.CreateRegion("toyRegion");
// Both toy and toyparts are put in the same region
catalog.Put("toy-101", new Toy( .,.), “toyRegion”);
Catalog.Put("toypart-100", new ToyParts(…), “toyRegion”);
Toy toyObj = (Toy)catalog.Get("toy-101“,"toyRegion");
30. Access APIs – Tagging Items
Tag hotItem = new Tag("hotItem");
catalog.Put("toy-101", new Toy("Puzzle"),
new Tag[]{hotItem}, “toyRegion”);
catalog.Put("toy-102", new Toy("Bridge"), “toyRegion”);
// From the same or a different client
List<KeyValuePair<string, object>> toys =
catalog.GetAnyMatchingTag("toyRegion", hotItem);
31. Types of Data
Reference Activity Resource
Primary Read Only Read-Write Not shared Read-Write, Shared
Catalog Data Shopping Cart Auction Data/Seat
Assignment
Grocery Shop
Web Tier
Shopping Cart
Grocery Inventory
Distributed Cache
Grocery Catalog
32. Reference Data – Performance
• Catalog data doesn’t change often
• Unnecessary network cost to access from different machines
• Solution – Local Cache
Application
Application Get(K2)
Get(K2)
Put(K2, v3)
AppFabric Caching Client AppFabric Caching Client
Local Cache K2, V2
Routing Table Routing Table
Cache1 Cache2 Cache3
Primary for K1,V1 Primary for K2,V2 Primary for K3,V3
K1, V1 K2, V2
V3 K3, V3
33. Reference Data – Bulk Get
• Bulk Fetch from region
– 200-300k ops per second
– Fewer network calls
Catalog.BulkGet(
new List<string>(){“toy-101”, “toy-102”} ,
“toyRegion”);
34. Activity Data – Session Integration
Load Balance Requests
No more sticky routing
<sessionState mode="Custom“ customProvider="SessionStoreProvider">
<providers> Drop in AppFabric
<add name="SessionStoreProvider"
type=“Microsoft.Data.Caching.DataCacheSessionStoreProvider,
Application Application …
Application
Caching
SessionStoreProvider
ClientLibrary“ Access Layer
Caching Caching Access Layer Caching Access Layer
cacheName="<YourNamedCache>"/>
</providers> Session State
</sessionState> stored in
AppFabric
Allows session state to be shared amongst multiple applications Caching
Scale your
Session Store
Caching Service
Caching Service
Cache Service
Dynamically
Highly
Available
35. Activity Data - Availability
Application
(K2, V2) Application
PUT Get(K2)
AppFabric Caching Client
Routing Table AppFabric Caching Client
Routing Table
Cache1 Cache2 Cache3
Primary for Primary for Primary for
K2, V2
Replication
K1, V1 K2, V2 Agent K3, V3
K2, V2
Secondary for Secondary for Secondary for
K2, V2 K3, V3 K1, V1
36. Resource Data - Optimistic Locking
• GetCacheItem returns a version object
• Every update to an object internally increments it's version
• Supply the version obtained along with the Put/Remove
• Put/Remove will succeed only if the passed in version matches the
version in the cache
Version Based Update
Time Client1 Client2 (Different Thread or process)
Two clients access the same
T0 CacheItem item = CacheItem item = item
catalog.GetCacheItem(“PlayerRegion”, catalog.GetCacheItem(“PlayerRegion”,
”Zune”); ”Zune”);
Both update the item
T1 ((ZuneObject)item.Object).inventory --; ((ZuneObject)item.Object).inventory--;
T2 catalog.Put(“PlayerRegion”, “Zune”, Second Client gets in first;
item.Object, item.Version); put succeeds because item
version matches; atomically
T3 catalog.Put(“PlayerRegion”, “Zune”, increments the version
item.Object, item.Version);
// Version mismatch First client tries put;
// Client must retry again Fails because the versions
don’t match
37. Resource Data - Pessimistic
Locking
Client1: Client2: Client3:
GetAndLock ("k1") GetAndLock ("k1") Get ("k1")
GetAndLock gets
lock handle
Regular Get
Other GetAndLock
succeeds
on same item fails
K1
• Take locks on non-existent keys
– Allows you to co-ordinate calls for data