FOWA London 2015
It's 2015 and we've all got real-time data coursing through our apps; the life-blood of their instantly updating, interactive and engaging user experiences. We're also all much more aware of development best practices and how tooling can assist this process. Many of these practices can also be applied when building realtime apps, but there are some tools and techniques that are more prevalent, and some that are unique, when working with real-time frameworks and data. In this talk I'll cover the tools, tips and techniques - from client to server - that I've found valuable when developing realtime apps.
22. Tip #2: You Have Real-Time Data
19 / 89
@leggetter
23. Tip #2: You Have Real-Time Data
Any event in your web stack...
Incoming Data
Data Changes
System Interactions
User Interactions
... represents real-time data.
19 / 89
@leggetter
30. Before we begin...
Tip #3: When designing & building real-time
apps, always think about how the
decisions you're making - anywhere in
your stack - will impact the client app.
25 / 89
@leggetter
38. 29 / 89
Example: The Twitter Streaming API
Waiting for data...
@leggetter
39. 30 / 89
Example: The Twitter Streaming API
Sometimes there's just too much...
@leggetter
40. Problem:
You can't always control what the data source sends and when it sends
it. Even it you can, it can be tedious to trigger test data.
With REST APIs we can generate fake data. With real-time... ?
31 / 89
@leggetter
41. Problem:
You can't always control what the data source sends and when it sends
it. Even it you can, it can be tedious to trigger test data.
With REST APIs we can generate fake data. With real-time... ?
Solution:
Capture & replay
31 / 89
@leggetter
55. You could...
But I want there to be better options
“log to stdout, use the logfmt format, know your
good old grep / sed / awk - @zimbatm
42 / 89
@leggetter
56. We need better HTTP
Streaming/WebSocket capture & replay
tooling to improve the developer
experience
43 / 89
@leggetter
57. It's all fine and well saying to capture &
replay the data. But...
You said, always think about the client!
What about the client?
44 / 89
@leggetter
58. Tip #6: The server is your real-time data
work-horse. It should still do the vast
majority of data processing and decision
making. Web, Mobile an IoT clients are
great, but they're not processing machines.
45 / 89
@leggetter
62. Don't send this When you only need this
Tip #7: Only send the data that is required
{
"screen_name": "leggetter",
"text": "I love to tweet",
"created_at": "Wed Sep 23 18:10:25 +0000 2015"
}
49 / 89
@leggetter
63. Send an initial image, then changes
Maintain an image of the current state of data
Only send data that changes
50 / 89
@leggetter
64. {
"home_team": "Liverpool",
"home_team_score": 0,
"away_team": "Arsenal",
"away_team_score": 0,
"event_time": "1443117125014"
}
Send an initial image, then changes
Maintain an image of the current state of data
Only send data that changes
50 / 89
@leggetter
65. {
"home_team": "Liverpool",
"home_team_score": 0,
"away_team": "Arsenal",
"away_team_score": 0,
"event_time": "1443117125014"
}
{
"home_team": "Liverpool",
**"home_team_score": 1,**
"away_team": "Arsenal",
"away_team_score": 0,
**"event_time": "1443117150287"**
}
Send an initial image, then changes
Maintain an image of the current state of data
Only send data that changes
50 / 89
@leggetter
66. {
"home_team": "Liverpool",
"home_team_score": 0,
"away_team": "Arsenal",
"away_team_score": 0,
"event_time": "1443117125014"
}
{
"home_team": "Liverpool",
"home_team_score": 1,
"away_team": "Arsenal",
"away_team_score": 0,
"event_time": "1443117150287"
}
Send an initial image, then changes
Maintain an image of the current state of data
Only send data that changes
▼
50 / 89
@leggetter
67. {
"home_team": "Liverpool",
"home_team_score": 0,
"away_team": "Arsenal",
"away_team_score": 0,
"event_time": "1443117125014"
}
{
"home_team": "Liverpool",
**"home_team_score": 1,**
"away_team": "Arsenal",
"away_team_score": 0,
**"event_time": "1443117150287"**
}
Send an initial image, then changes
Maintain an image of the current state of data
Only send data that changes
▼
{
"home_team_score": 1,
"event_time": "1443117150287"
}
50 / 89
@leggetter
68. 51 / 89
Example: Streaming Tweets to a UI
Non-Existent-UX
@leggetter
70. Control the frequency of updates
100ms is instantaneous to a human
52 / 89
@leggetter
71. Control the frequency of updates
100ms is instantaneous to a human
250ms is average human reaction time
52 / 89
@leggetter
72. Control the frequency of updates
100ms is instantaneous to a human
250ms is average human reaction time
Each update requires some processing
Data parsing
UI updates
...
52 / 89
@leggetter
73. Control the frequency of updates
100ms is instantaneous to a human
250ms is average human reaction time
Each update requires some processing
Data parsing
UI updates
...
Consider batching
52 / 89
@leggetter
75. We know what data we're sending to the
Real-Time Framework
54 / 89
@leggetter
76. Can we confirm what's going in?
And see what's going out?
55 / 89
@leggetter
77. Tip #8: You need to know what's going on in
your real-time framework
connections
subscriptions
messages
presence
other events/features
56 / 89
@leggetter
78. LOG ALL THE THINGS
Monitor them
Filter them
Build tools around them
57 / 89
@leggetter
89. Tip #9: Choose the right real-time framework for the
functionality you want to build and the client
devices you're targeting...
67 / 89
@leggetter
90. FOWA 2013 ForwardJS 2 - 2015
Tip #9: Choose the right real-time framework for the
functionality you want to build and the client
devices you're targeting...
67 / 89
@leggetter
91. Soon you won't be able to go to the
bathroom† without SSL.
† erm, I mean use HTTP/2, ServiceWorker, Fullscreen and many more web features
68 / 89
@leggetter
92. Soon you won't be able to go to the
bathroom† without SSL.
† erm, I mean use HTTP/2, ServiceWorker, Fullscreen and many more web features
Tip #10: Use SSL all the time - including in
development.
68 / 89
@leggetter
93. Soon you won't be able to go to the
bathroom† without SSL.
† erm, I mean use HTTP/2, ServiceWorker, Fullscreen and many more web features
Tip #10: Use SSL all the time - including in
development.
You need SSL for networks with proxies and
firewalls. In particular, mobile networks.
68 / 89
@leggetter
97. Because we've always considered the
client we're doing everything we can to
make things easy
72 / 89
@leggetter
98. Because we've always considered the
client we're doing everything we can to
make things easy
Processing - on the server
Payload - focused & minimal
Image + Changes
Formatted - further reduce processing
Update frequency - controlled / batched
72 / 89
@leggetter
99. Tip #11: Monitor Client Performance - if
you know how the client is coping, you can
make adjustments.
73 / 89
@leggetter
100. Latency
Timestamp published messages
Check time upon receipt
Consider:
Timezone differences
System clock offsets
If latency gets high, inform the user
74 / 89
@leggetter
101. Message Backlog
If updating the UI causes a
delay, a queue of messages
can start building up
setTimeout
requestAnimationFrame
75 / 89
@leggetter
102. Throttle Updates
Let the server know the client is struggling
RESET
If things get really bad
76 / 89
@leggetter
103. Using the Real-
Time Framework
library within
your App Logic
77 / 89
@leggetter
104. We've covered a lot
Data Sources - you can't control the data
Data considerations: processing, payload,
update frequency
Connectivity - Online/Offline, SSL, transports &
protocols
Monitoring client performance
78 / 89
@leggetter
105. Problem: All these things to consider and I
just want to build a real-time feature!
79 / 89
@leggetter
107. Tip #12: Use a service/library abstraction
or stub. Mock out data (messages) &
connectivity events.
81 / 89
@leggetter
108. Use a service/library abstraction
Angular/Ember/your own Services
var myModule = angular.module('myModule', []);
myModule.factory('pusher', function() {
var pusher = new Pusher('YOUR_APP_KEY');
return pusher;
});
82 / 89
@leggetter
109. Use a service/library abstraction
Angular/Ember/your own Services
var myModule = angular.module('myModule', []);
// myModule.factory('pusher', function() {
// var pusher = new Pusher('YOUR_APP_KEY');
// return pusher;
// });
myModule.factory('pusher', function() {
var pusher = new FakePusher();
return pusher;
});
83 / 89
@leggetter