Web applications are massively shifting to the frontend, thanks to exciting new JavaScript / CSS technologies, expanding browser capabilities (visualizations, real-time apps, etc.) and faster perceived user experiences. However, client web applications can be a nightmare to maintain at scale, even for seasoned software architects and operations engineers. Deployment and production infrastructures are complex and rapidly changing. And, frontend JavaScript / CSS code ships to browsers worldwide, where errors and issues are notoriously difficult to systematically detect and diagnose. In this talk, we will tackle the wild west of the frontend with pragmatic steps and seasoned advice from helping organizations from startups to Fortune 500 companies create some of the largest frontend web applications on the Internet. In particular, we will examine the many hard lessons gleaned from leading frontend application development and education for a team of 50+ engineers rearchitecting a top-five e-commerce site. Some of the topics we will cover include: * Managing and building very large (500K+ line) frontend application / test code bases. * Surviving production traffic and errors on the frontend and handling spikes like Black Friday / Cyber Monday for one of the highest traffic e-commerce websites in existence. * How, where, and why your frontend application is likely to fail. * Monitoring, logging, and debugging frontend web applications out in the wild. * Automating checks, tests, and code introspection to protect your code in production. * Creating an effective, fast, and engineer-friendly development-test-deployment frontend pipeline. Whether your frontend application already supports millions of transactions a day or you are about to launch your first single-page-application, our aim is to prepare teams of all sizes for the most critical challenges and solutions facing modern frontend web applications.

1. @ryan_roemer | surge2015.formidablelabs.com
WRANGLING
LARGE SCALE
FRONTEND WEB APPLICATIONS

2. The web is
massively moving
to the frontend

3. Users want rich
and seamless
experiences

4. Product owners
want fast and
nimble apps

5. Browser apps are
now business
critical

6. And, yes, even for
the enterprise

7. ... which means

8. LOTS OF JAVASCRIPT
IN THE BROWSER
WRITTEN BY LARGE TEAMS

9. Let’s dig into some large
frontends at a high-traffic,
top-five e-commerce site

11. The Numbers
$10 billion FY 2014, $13 billion FY 2015 (est)
1.5 billion page views for Thanksgiving -
Cyber Monday 2014

12. The Code
50+ JS applications
650K lines, 2500 files of JavaScript source
More JavaScript lines & files than Java

13. The Technologies

14. The Dev Team
A 2+ year website rewrite
50+ core frontend engineers
14 vertical teams / "tracks"
... and many external teams / partners

15. A TOUR THROUGH
THE TRENCHES

17. My wrangling adventures
as the JavaScript lead for
the website & dev teams

18. A few battle-tested
tips from the field...

19. ... with a focus on
four personas

20. ARCHITECTS

GUIDES

GATEKEEPERS

LIFEGUARDS


21. 
ARCHITECTS

22. Plan & build a strong
foundation

23. Architects
A real build
Code organization

24. THE FUNDAMENTAL CHALLENGE:
JAVASCRIPT IS THE
WILD, WILD WEST

25. Architecture Challenges
Browser is a single execution environment
Code size / duplication is critical
Hard to manage, easy to do wrong
"Cowboy" legacy vs. large scale organization

26. Let's look at the
architectural complexities
of just one page...

27. 
THE HOMEPAGE

29. The Homepage
Multiple, independent JS apps
Code from 8 different teams
Exemplary "in transition" page, somewhere
between raw JavaScript & a SPA

30.  JavaScript
ON PAGE
Direct page scripts
10 remote scripts
18 inline scripts
LAZY LOADED
Injected script tags
6 app entry points

31.  On Page
2 application code
2 CDN infrastructure
2 internal ads
2 Google ads
1 fonts
1 IE-conditional polyfill
18 inline scripts

32.  Lazy (Code)
window._entry(function() {
require(["header/header"], function () {
require(["header/header-deferred"]);
});
// ...
require(["homepage/homepage"]);
// ...
});

33.  Lazy (App)
Asynchronously loaded
1 shared library
4 primary entry points
2 deferred entry points

34. THE TAKEAWAY?
IT'S COMPLICATED.

35. A "REAL" BUILD

36. Build Challenges
Modern JS apps are complicated
(compression, polyfills, transpiling, etc.)
Multiple JS apps on the same page is even
more complex
Supporting development & production / CDN

37. Use a Build Tool
Choose an paradigm: AMD, CommonJS
Choose a build tool / loader: RequireJS,
Browserify, Webpack
Take time to learn & evaluate the tradeoffs

38. Our Build Tools
Legacy: AMD + RequireJS
Modern: CommonJS + Webpack
Transition: AMD & CommonJS + Webpack

39. Keep prod & dev builds
blazingly fast

40. Make development
identical to production

41. Build Complexities
Sharing / caching common code (lib.js)
Varying repository structures
Sharing frontend & backend code
Combining application, OSS, & 3rd party code

42. CODE ORGANIZATION

43. Plan your repository
structure
1 → 14 → many repos

44. Have a bias for small &
flexible

45. 
GUIDES

46. Coordinate the chaos,
level up the development
teams

47. Guides & Guidance
The Meta team
Educate
Review everything

48. Guidance Challenges
Thinking of the project as a whole
Onboarding unfamiliar / junior developers
Helping teams be consistent /
complementary

49. THE META TEAM

50. Code from multiple teams
all combined on one page

51. These folks represent the
whole page

52. The Meta Team
Set conventions & best practices
Lead code reviews for consistency / DRY
Develop common utilities & the deployment
infrastructure

53. YOUR META TEAM TASK:
HAVE ONE.

54. Our Meta JS Team
1.5 dedicated developers
6 "loaned" track developers

55. Our Meta JS Duties
~1 day of code review / week
Chat channel participation
Cross-track JS initiatives (i18n, multi-tenancy,
PCI, etc.)

56. Meta JS Benefits
Project-wide consistency & support
Help tracks consider other teams
Represent track interests in the core
More engineers to support the whole project

57. EDUCATION

58. Continually write living
documentation,
close to the code

59. Take a "hands on approach"

60. Invest in rising developers &
spread knowledge back to
teams

61. CODE REVIEW

62. All code gets substantive &
meta review

63. Including all third party &
internal to the org vendor
code

64. 
GATEKEEPERS

65. Protect the frontend through
process & architecture

66. Gates & Gatekeepers
Automate quality
Minimize risks / exposure
Require performance

67. Gatekeeping
Challenges
Feature pressure
Poor quality code
Unknown included code

68. AUTOMATE QUALITY

69. Static checking
(eslint, jshint, jscs, etc.)

70. Clientside unit tests

71. Integration / E2E tests

72. Code coverage

73. Continuous integration (CI)

74. MINIMIZE RISK

75. Learn / identify your biggest
risk areas

76. Protect yourself wherever
possible

77. Architecture risks:
Injected HTML/CSS/JS

78. Code pattern risks:
Defer & pray
// I'm guessing...
var HOPEFULLY_ENOUGH_TIME = 2000;
// Wait until ready for next step.
setTimeout(function () {
theNextStep();
}, HOPEFULLY_ENOUGH_TIME);

79. Deployment / process risks:
"Hotfeatures"

80. REQUIRE PERFORMANCE

81. Frontend code must be small
& fast

82. Teams are "feature-driven"
unless performance is
enforced & required

83. Enforce with audits
"Web App Performance
Measurement, Monitoring and
Resiliency"
www.webpagetest.org

84. Enforce with process

85. Enforce with automation
"Automating Web
Performance Measurement"

86. Build your own tools where
necessary

87. 
LIFEGUARDS

88. Create escape hatches &
lifelines in production

89. Lifeguards & Life
Savers
Logging, Monitoring
Debugging helpers

90. Lifeguarding
Challenges
Code executes remotely on different
browsers
Frontend errors are costly & often invisible

91. LOGGING & MONITORING

92. Your code runs & fails
on a variety of browsers
out in the wild

93. Log & capture everything

94. Get errors & messages
to a remote store

95. And then aggregate,
report, & alert

96. Providers
Rollbar Loggly
Sentry Airbrake

97. DEBUGGING SUPPORT

98. Give developers life lines
when things go wrong

99. You ship this:
!function(){var e="Hello Surge!",a=$("<h1 />");
a.text(e),$("body").append(a)}();

100. Your devs want this:
(function () {
var message = "Hello Surge!";
var $heading = $("<h1 />");
$heading.text(message);
$("body").append($heading);
}());

101. Source Maps
/* ... LOTS MORE CODE ... */
[],window._entry=c}();
//# sourceMappingURL=http://dev.walmart.com:9873/js-dist-frontend

102. Publish in VPN on
deployment

103. Hidden from end users
Available to developers

104. All Together Now
A "real" build
Organized code
The Meta team
Education
Code Review
Automate quality
Minimize risks
Require performance
Logging, Monitoring
Debugging helpers

105. Some parting
thoughts on the
future

106. Embrace change

107. Reevaluate & refactor your
infrastructure & organization

108. Have a transition strategy

109. HAPPY
WRANGLING

110. surge2015.formidablelabs.com
THANKS!