3. Peak frequency, peak deviation
All the time
Once a day
Once a month
Once per season
Once per year
One time only
f(Frequency, Deviation)
50% over avg
2X avg
10X avg
1000X avg
∞ avg
4. Service zones
Private Public cloud
- Private trust - Elasticity
1-n
API-based integration
5. Coordinating service zones
Declarative abstraction layer home
Handle at perimeter:
- Secured communication
- Distributed policy enforcement
- Data Distribution public cloud
region 2
public cloud
region 1
6. Hybrid cloud infrastructure based coordination
API/Service Gateway at perimeter of each zone
Apply to traffic in both directions:
- Routing rules
- Translation rules
- Caching rules
- Access rules
- Security rules
8. SAML
SAML Based authentication, authorization
- User redirection
- API based authorization and attribute statements
Establish session
Authenticate, request attributes
Authenticate credentials
++SAML
against id provider
Get back SAML Authorize access to
service based on SAML
assertions associated with
session
9. OAuth across zones
Decouple AS and RS Each zone manages its own tokens
- authorization server on premise and - Still delegate authentication
resource server on cloud
- Scale best
- If opaque tokens and support for
revocation requires call back to issuing - No centralized token management,
side (less scaling) localized revocation
- Otherwise use self-signed tokens, no OAuth Authorization Server
revocation or pushed revocation, short
token lifespan
abc123
Identities
OAuth Resource Server
abc123
Which ID? Backends
Which scope?
Still valid?
10. Federate identity from public to private zone
OpenID Connect
- Standardizes how an OAuth handshake is used to delegate authentication
- Standardizes the API to discover an API authenticated with OAuth
- JWT instead of SAML
/userinfo
Authorization: Bearer [oauth access token]
• OAuth Authorization Server
• UserInfo endpoint
{
"user_id": "248289761001",
"name": "Jane Doe",
"given_name": "Jane",
"family_name": "Doe",
"preferred_username": "j.doe",
"email": "janedoe@example.com",
}
11. Scaling up on public cloud
AWS-based case study
- Front-end load distribution
- Back-end load distribution
- Distributed policy configuration
ELB
- Service endpoint discovery
- Perimeter gateway resize RDS
Perimeter gateway cluster
Auto Scale
Service endpoint instances
12. Ramp-up rate
Time to instantiate a new instance: 5 minutes
Time to instantiate 10 instances: 5+ minutes
Nr instances per increment controls how fast you meet increased demand
In some cases, you can’t react fast enough
Sudden demand increase example
t: tickets for U2 concert go on sale
t+3m: sold out
t t+3 minutes
13. Capacity planning and automatic scaling
Planned capacity adjustments
- Capacity planning: understand your traffic patterns, regular patterns and growth
over time
- Adjustment in capacity can be automated (e.g. autoscaling scheduled actions)
- Stay ‘ahead’ of demand
- Future demand cannot always be accurately predicted
Automatic scaling
- Temporary increase in capacity to address unexpected change in demand
- Best practice: limited by a ceiling
- Reaction time in minutes
- Notify provider so that unexpected peak be analyzed and further action determined
14. Perimeter gateway cluster on public cloud
Every node from cluster share same persistence layer instance
- Leverages RDS
Traffic is distributed across nodes
- ELB
ELB vip
…
RDS instance for this cluster
15. Scale-up perimeter gateways
Adding new gateway instances needs to be automated, no manual steps
New gateway instantiated, need to discover RDS target and credentials
- Cannot be part of AMI snapshot, nor passed as user data
Admin gateway instance holds this information for new gateway instantiated in such
fashion
AutoScale Group Scale controller
Master instance
Current cluster (discover RDS
API)
++node
Is this requester
really part of the
AS group?
RDS host?
Credentials?
AWS API
16. Configuration across regions
home
Automated change
provisioning and mapping
between environments
- Policy updates public cloud
Promote Localize and region 2
- New services UAT push to each rds
zone
public cloud
region 1
rds
17. Push+cache
home
++new or changed content
public cloud
region 2
Localize and
push to each
zone ++backend
++cache
public cloud
region 1
++backend
++cache
18. Backend load-balancing pool management
Backend endpoints scale using typical AutoScale Group strategy
Distribution from perimeter to backend is direct (no ELB)
Gateways discover backend instances by pulling
backends
distribute
Regular pull
through API
call …
Monitoring and
scaling
infrastructure
19. Backend load-balancing pool management
Pushed updates
- Gateway expose API to receive backend pool information
- Refresh pushed by same process which managed the pool in first place
backends
distribute
Push change
in backend
pool …
Monitoring and
scaling
infrastructure
20. Application-aware auto-scaling
Monitor over time Manage backend instances scale-up and
scale-down
- Concurrency
Interface with vCloud director API or
- Backend response time equivalent
- Nr queued
- Per service, per operation
- Application-level aware monitoring
++instance
cloud director
--instance
API traffic
API traffic
Backend/data
instances
21. Backend load-balancing pool management
Multi-pool coordination
- Perimeter gateway calculates effective service backend pool based on preferential
instances combined with varying availability
Primary pool Secondary pool Effective pool
10.7.2.91 10.7.2.94 10.7.2.91
10.7.2.92 10.7.2.95 10.7.2.95
10.7.2.92
10.7.2.93 10.7.2.96 10.7.2.93
22. Traffic compression across zones
Add header
- accept-encoding: gzip
Service zone A request Service zone B
response
Zip content
Unzip content
Add header:
- content-encoding: gzip
Compression applied at perimeter
Also pre-emptive compression on
requests with payloads