Addressing the scalability challenge of server sent events presentation

Scaling Server-Sent Events (AKA Long Polling)
Stephen Ludin
Chief Architect, Akamai Technologies

What We Saw

09:51:23.051736 IP client.62471 >server.80: Flags [S], seq 233319732, win 65535
09:51:23.056777 IP server.80 >client.62471: Flags [S.], seq 227753171, ack 233319733, win 5792
09:51:23.056906 IP client.62471 >server.80: Flags [.], ack 1, win 32976
09:51:23.057034 IP client.62471 >server.80: Flags [P.], seq 1:156, ack 1, win 32976
09:51:23.061841 IP server.80 > client.62471: Flags [.], ack 156, win 215

And it just sat there…

Velocity 2011 Powering a Better Internet ©2011 Akamai

Polling
How much longer now?
How much longer now? How much longer now?





Dad, when are the fireworks starting?


Long Polling


Long Polling – What is it?

A method for emulating „server push‟ and providing real time notifications
• Browser uses XMLHttpRequest to connect to origin and waits
• When there is data to send, the origin responds

Variants and frameworks:
• Long Polling
• Server-Sent Events
• HTTP Streaming
• Bayeux
• BOSH
• Comet


Usage is growing


What‟s Changing


Requests For Help


The Challenges of Long-Polling for the Origin

Trading off high request rate (polling) for massive concurrent connections

Scaling at the Origin
• Not everyone has event-driven Web servers (Jetty, lighttpd, nginx)
• Still a lot of older architectures out there

What is really desired is a “Server Push” model

But despite all that, we still like long-polling
• Provides a “Real Time Web” without polling
• Makes modern HTTP applications possible

So: Is there a way to offload the connection load and provide server push?


In Short…

Everyone wants to use long polling, but scaling is a challenge.

Addressing this scale problem will result in better origin performance.


“Normal” HTTP Request Flow with a CDN


Long Poll HTTP Request Flow with a CDN


How can a CDN help?

Offload Via Edge Caching or Computing?

Acceleration?

Application of business logic?

Security / Web Application Firewall?


Two Key Concepts

Half-Sync / Half-Async
• “Decouples synchronous I/O from asynchronous I/O in a system to simplify concurrent
programming effort” 1

Publish / Subscribe (Pub/Sub)
• The generic model behind most events

1 Douglas C. Schmidt and Charles D. Cranor, 1996, “Half-Sync/Half-Async: An Architectural Pattern for Efficient
and Well-Structured Concurrent I/O”


Requesting an Event (Subscribe)

“User A wants Event 1”

T

User Event Token
A 1 T1
B 2 T2
C 1 T3


Delivering the Event (Publish)

“I‟ve Got Mail!”
Event 2 Fired for User B!

T

T

User Event Token
A 1 T1
C
B 1
2 T3
T2
C 1 T3


Half Sync / Half-Async Benefits

Provides the ability to scale

Enables “true” Server Push

Retains “real time” notification

Makes load balancing at the origin easier

Makes infrastructure management at the origin easier


The Implementation

Token Construction
• Information needed to get back to the edge machine (IP)
• Customer specific code
• User information
• Subscription (Event) information
• Expiration


The Implementation

On the Client:
• Use HTML 5 Server-Sent Events
• Use old fashioned long-polling
• Essentially, do what you do today


The Implementation

On the Edge:
• Configure the surrogate to react appropriately


The Implementation

For example, on Akamai:
<match:uri.component value=“subscribe-event”>
<variable:extract from=“post” key=“id” name=“EVENT”/>
<variable:extract from=“cookie” key=“user” name=“USER”/>
<edgeservices:event.handle-subscription>
<token>
<key>ywewu238347i3u</key>
<nonce-source>PORT</nonce-source>
</token>
<user>$(USER)</user>
<event-id>$(EVENT)</event-id>
</edgeservices:event.handle-subscription>
</match:uri.component>


The Implementation

And go Forward with:
POST /subscribe-event HTTP/1.1
Host: mail.foo.com
X-Event-Id: 2
X-Event-User: B
X-Event-Token: of2948f394fornvo334o343o4oejo23jf2
X-Event-Signature: f1d2d2f924e986ac86fdf7b36c94bcdf32beec15
...


The Implementation

On The Origin - Subscription
• Receive the subscription request
• Respond with a “202” (eg.) in the positive

On The Origin – Event Firing
• When event fires, send the event data
• Sign token
• Application specific, recommend SSE
• Fire and forget, persist, or stream


The Implementation

POST /deliver-event HTTP/1.1
Host: event.foo.com
X-Event-Id: 2
X-Event-User: B
X-Event-Token: of2948f394fornvo334o343o4oejo23jf2
X-Event-Signature: e242ed3bffccdf271b7fbaf34ed72d089537b42f
Content-Length: 16

You’ve Got Mail!


Subscription Types

One Shot Event
• Force client reconnect (re-subscribe)


Subscription Types (cont)

Repeatable Event
• Origin → CDN: Multiple Requests
• CDN → Client: HTTP Streaming


Subscription Types (cont)

HTTP Streaming
• Similar to Multiple Events
• Potential for multiplexing


Security

Risk: Bogus Event Injection

SSL on all sides will help
• Origin to CDN MUST be authenticated

The token MUST be secure
• Necessitates a shared secret or more expensive asymmetrical operations
• Replay protection


Some Error Cases

Origin Rejects the subscription request
• An error is returned to the edge machine
• Edge machine delivers the error

Tokens are found to be invalid by some party
• Be paranoid
• Drop connections and force resubscription


Error Cases (cont)

Client drops and reconnects
• If detected by edge machine, unsubscribe event can be fired
• Origin should detect multiple subscriptions and resolve
• Optional: If client also has a token it can be used to reconnect to the original edge
machine via redirect or tunneling

Edge machine „disappears‟
• Devolves (hopefully) to a client drop and reconnect


Error Cases (cont)

Annoying routers dropping quiet connections
• Heartbeat events can help (Wait! Isn‟t that polling?)
• Fortunately a well understood problem


Mobile – Connectionless Push Friendly


What about WebSockets?

Not a good candidate (today)
• Bi-directional
• Opaque
Standard Acceleration techniques are ideal
Anticipating „standards‟ in the future


Use Cases

E-Mail
• Millions of users want to know when they get new mail. Now.


Use Cases

Social Networking
• What friends are online? What are they doing? I want to chat with them!


Use Cases

Stock Quotes
• BSC: 78.34

• BSC: 75.56

• BSC: 38.12

• BSC: 3.12

• BSC: Delisted


Use Cases

Cloud Printing
• Printer manufacturer sells 100 million Internet-enabled printers and wants to enable
cloud printing in a scalable and efficient manner.


Summary

Server-Sent Events is a great thing
• Introduces connection scaling problems
• Formalizes long-polling methodologies
• Useful whenever a user is expected to wait a „long time‟ for a reply

CDNs can help with the scaling problem
• Half-Sync / Half-Async
• Security Features
• Business Logic
• Acceleration

CDNs can provide a “server push” paradigm to the origin


Questions


Addressing the scalability challenge of server sent events presentation

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Addressing the scalability challenge of server sent events presentation

Ähnlich wie Addressing the scalability challenge of server sent events presentation (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Addressing the scalability challenge of server sent events presentation

Hinweis der Redaktion