2. Tech Talk - Frontend
Stephen Boak (steve@boundary.com)
Mark Mahoney (mark@boundary.com)
Matt King (m@boundary.com)

3. When we say real-time we mean it: JSON being
pushed to the browser once a second and
visualizations built to constantly move and adapt
to streaming data

4. Boundary | Architecture

5. Meters !
Lightweight and highly scalable, these sit on the cloud as well as
private data centers across virtual or physical servers.

6. Intercept Meter data via Transport
Collectors Layer Security (TLS) Authentication.

7. Collect all of the data at high resolution to gain rich insight into complex
Data Store environments & problems before they can impact critical business services.

8. Real-time continuous data streaming at
Streaming high resolution, low latency intervals.

9. Stream data to your dashboard with sub-second latency,
Streaming UI providing you with intuitive, powerful dashboard.

10. How We Get Data
• CometD server (streaker)!
• Pulls in data from multiple backend services and
streams it out to clients!
• Can also aggregate and filter data on demand

11. Subscriptions
1 2 3 4 5
Make a Get back a Subscribe to Receive Receive
subscription request subscription ID new query state-dump add/remove
(query and filters) (Query ID) (unique channel) (inserts & schema) messages
!!
!
!
!
!

12. Data Structure
• Schema and key
(which fields are concatenated for each record)
• State dump with schema for requested time window
(up to 100kb state dumps)
• Save bandwidth with N-tuples indexed by the schema
• Adds/removes update the state with keys (subset of schema)

13. JSON
Example port:protocol subscription State Object
{ {
channel:""/query/ac3941b8924a2f73/custom9 "1331234598000:80:6":"{
query92f536bbc93f159418b98f1a973b5dc4e78379 "epochMillis":=1331234598000,
filter979564d6c94c9394f479abae906ff95a3411e9 "portProtocol":="80:6",
aggregation9b5cc18d49098d949c29a08497037b08a30e6" "ingressPackets":"100,
data:"{ "ingressOctets":"8463,
insert:"[[1331234598000,"80:6,"100,"8463,"93," "egressPackets":"93,
9672],"[1331234925000,"8080:6,"242,"39657,"243," "egressOctets":"9672
40865],"…], },
keys:"[epochMillis,"portProtocol], "1331234925000:8080:6":"{
schema:"[epochMillis,"portProtocol,"ingressPackets," "epochMillis":=1331234925000,
ingressOctets,"egressPackets,"egressOctets], "portProtocol":="8080:6",
timestamp:"1331235119001 "ingressPackets":"242,
} "ingressOctets":"39657,
} "egressPackets":"243,
"egressOctets":"40865
}
…
}

14. DataSource.js
• Negotiates between multiple data
sources and subscribers on a page Subscriptions
• One data source for N subscribers Data
Sources
• Abstracts CometD subscription
process for JS development
Subscribers
• Smart enough to resubscribe and
notifies subscribers
(so that state dumps aren’t doubled up)

15. bndry.datasource repo
//"Receives"object"with"properties"for"state,"inserts"and"removes."
function"updateData(data)"{"
" console.log(data.added);"
}"
//"Create"a"handle"to"a"new"data"source"
var"volumeDataSource"="bndry.dataSource('volume_1s');"
//"Add"a"subscriber"and"start"getting"updates."
volumeDataSource.addSubscriber(updateData);

16. Demo Time

17. Data and Subcription Problems
• Monolithic, Multi-Purpose Queries
• Large State Dumps cause CometD timeout
(5-10MBs per second)
• No Resubscription
• Aggregated all data on front-end

18. Data and Subcription Solutions
• Stratified, Filterable Queries and server-side aggregation
• Top-N Limitations
(constrained output)
• Resubscription & filtering
• Web workers haven’t helped much
• Time smoothing (1-second ticks)

19. Visualization
DOM manipulation is expensive ...so is tweening
SVG Canvas

20. What’s Next
• WebSockets
• Historical Data and long term data storage
• HTML5 local storage (store data)
• Machine-learning for real-time network visualization

21. Thank You