In front-end software development it's still rare that data is collected on the client side besides some analytics data where developers usually don't have access to.
Imagine what you can do when you have front-end log data, you see how many ajax calls hitting your servers and you finally know whether the single page application is used like you expected or not. I will briefly talk about projects I was part of where we used these kind of data to improve our product and surprisingly reduced AWS costs by changing front-end code.
https://docs.google.com/presentation/d/1kGK8P7Ll2H4Z_1UUdBneAbNzUEDmpj8g2Mxj_Z-F5u8/pub?start=false&loop=false&delayms=3000
2. Abstract
In front-end software development it's still rare that data is collected on the client
side besides some analytics data where developers usually don't have access to.
Imagine what you can do when you have front-end log data, you see how many
ajax calls hitting your servers and you finally know whether the single page
application is used like you expected or not. I will briefly talk about projects I was
part of where we used these kind of data to improve our product and surprisingly
reduced AWS costs by changing front-end code.
3. Who am I?
Simon Dittlmann
Saving the world from
toil since 1998
Developer/Ops,
technology nerd
@SinnerSchrader
#IoT, #docker, #js, #frontend,
#infrastructure, #kubernetes
Contact
SimonDittlmann
github.com/Pindar
dittlmann.com
5. DevData: Why and How?
Everything fails all the time. Quality assurance testing is not enough. Let’s
formalize our developer assumptions, monitor them with data and improve.
Tools: so many. Google Analytics, Elastic-Stack, Datadog, Loggly etc.
6. Use cases
A. Logs: Client Side Exception tracking
B. Metrics: API performance, Call-Count, Page Performance
C. Application tracking: Find the important to prioritize
8. Logs: Client Side Exception tracking
Assumption: if there is no JS error in my developer console there aren’t any at all.
So often wrong. For example private browsing is totally different implemented in
various browser.
How to log client exceptions the easiest way without integrating yet another tool
and without hitting my own infrastructure? Use e.g., Google Analytics Exception
tracking.
12. Metrics: API performance, Call-Count, Page Performance
● Monitor how long an API call takes both on the server side and on the client
side
● Monitor the amount of calls throughout a day
● Program with edge cases in mind
13. Monitor Server Logs
● Check for latency, response time etc.
● Monitor which endpoints are requested the
most.
● Do load tests from time to time, e.g. with
$> ab -n 100 -c 10 -C
$cookie_string $domain
● Take advantage of state of the art tooling:
○ Elastic Stack (aka ELK): Kibana,
ElasticSearch, logstash
○ GCloud Stackdriver and BigQuery
○ AWS Cloudwatch logs
○ Software as a Service Solutions like loggly,
papertrail…
14. User Timings: Tracking an API call
var t0 = performance.now();
ga('send', 'timing', 'api1calls', 'start', Math.round(t0));
fetch('https://www.example.com/testApi1')
.then(function(response) { return response.json(); })
.then(function(data) {
var t1 = performance.now();
var timeItTook = t1 - t0;
console.log(timeItTook, JSON.stringify(data));
// Sends the timing to Google Analytics.
ga('send', 'timing', 'api1calls', 'load', Math.round(timeItTook));
return;
});
17. Do not accumulate ajax calls
var pause = false;
setInterval(() => { // the interval is usually shorter than the http timeout…
if (pause) return;
pause = true;
fetch('https://example.com/testApi').then((response) => { return response.json(); })
.then((data) => {
console.log('do something');
pause = false;
}).catch((error) => {
console.log('There has been a problem with your fetch operation: ' + error.message);
pause = false;
});
}, 5000);
18. Disable polling when page is not visible
// This may save you money in an autoscaling environment
// functionality uses vendor prefixes, see documentation for details.
document.addEventListener(visibilityChange, handleVisibilityChange, false);
function handleVisibilityChange() {
if (document[hidden]) {
clearInterval(timerId);
} else {
timerId = startPolling();
}
}
// https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API
20. Optimize the important
● Use analytics data to know what to optimize
● Collect the right data: to support that use html data-* attributes
● Rank your pages based on the value of the page
● Fix JS errors immediately
21. What else… some ideas.
● using conventional commit messages to determine hotspots in the code base:
e.g. many fixes in one file makes it to a hotspot
● one can use this hotspot information to automatically make code reviews
obligatory for risky changes
● … there is more to come