CubeJS is a Node.js based application development platform at eBay. In this presentation, we will share our journey to build and launch this new platform at the eBay Marketplaces. We will discuss CubeJS’ overall architecture, versatile functionalities, robust operationalization, and streamlined development lifecycle process, as well as its customer adoption momentum in the company. We will also share a number of high-profile projects, powered on CubeJS, to be debuted by eBay Inc. ranging from Web to Mobile.

1. EBAY’S
CUBEJS:
NODE.JS ADOPTION JOURNEY
@

2. Patrick Steele-Idem
Senior Platform Architect @
@psteeleidem
psteeleidem@ebay.com
github.com/patrick-steele-idem

3. EBAY HAS EMBRACED NODE.JS…
SO HOW DID WE GET HERE?

4. BEFORE NODE.JS, JAVA WAS KING
BUT THINGS EVOLVED…
XML + XSLT  JAVA
PROPRIETARY  OPEN SOURCE
ONE LANGUAGE  POLYGLOT
ONE IDE  ANY EDITOR/IDE
HEAVY FRONT-END  LIGHTWEIGHT FRONT-END
CLEARCASE  GIT + GITHUB
SLOW DEVELOPMENT  RAPID DEVELOPMENT
JAVA APIS  SERVICE APIS

5. “A DECLARATIVE, DATA-RETRIEVAL AND
AGGREGATION GATEWAY FOR QUICKLY
CONSUMING HTTP APIS”
…BUILT USING NODE.JS
WHY NODE.JS?
• Non-blocking I/O
• Single-threaded

6. NODE.JS IS GREAT,
BUT CAN EBAY USE IT TO
BUILD FRONT-ENDS AS WELL?

7. WHY NODE.JS FOR THE FRONT-END?
DEVELOPER AGILITY, SCALABILITY AND
PERFORMANCE
―My advice: always bet on JavaScript‖
–Brendan Eich

8. WHAT IS REQUIRED TO SUPPORT
NODE.JS AT EBAY?
Tracking
Monitoring
Security
Encryption/Decryption
I18n Testing
Deployment
Personalization
Resource Optimization
Release Process
Look & Feel
CSRF
Tools
Scalability
SOAP
REST
Logging
Configuration
Authentication
App Metadata

9. POLYGLOT CHALLENGE:
HUGE INVESTMENT IN JAVA CODE
Possible Solutions
Pros
Cons
Port code to
JavaScript
• Works for Node.js
• Easy to debug
• Tedious
• Error prone
• What about Python, Ruby,
etc.?
Expose as a service • Polyglot friendly
•
•
•
•
Latency
Monitoring
Network Failures
Infrastructure
Port code to C/C++
• Fast
• Bindings available
• Difficult to debug
• Difficult to maintain
JVM ↔ JavaScript
Bridge
• Polyglot friendly
• Difficult to debug
• Slower startup
• Monitoring multiple VMs

10. 1ST CHOICE: EXPOSE AS A SERVICE
MAXIMIZE CODE REUSE
CALLS CAN BE MADE IN PARALLEL
2ND CHOICE: PORT CODE TO JAVASCRIPT
PERFORMANCE CRITICAL
BEST HANDLED LOCALLY

11. RUNTIME ARCHITECTURE
Application Layer
Infrastructure Layer
Resource
Server
Distributed
Logging
Tracking
cluster2
Client Layer
…
App Service Layer
Search
…

12. DEVELOPMENT AND DEPLOYMENT
INFRASTRUCTURE
Source Code
an
initiative
powered by:
npm-delegate
Private npm
Public npm

13. DESIGN GOAL:
MAXIMIZE DEVELOPER PRODUCTIVITY

14. DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY
Small, bite-sized and easy-to-digest
modules
Keep the documentation with the code.
README files for the win
Encourage contributions

15. DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY
Avoid “magic”
If developers cannot reason about a system
then they will not be happy or productive.
Do not be a restrictive framework
Developers love options and being able to
use the latest and greatest.

16. DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY
+ File Watching
+ Hot Reloading
+ Web Sockets
Live Coding
Instant Gratification
Web Development is Fun Again
CHECKOUT:
• hot-reload https://github.com/philidem/node-hot-reload
• socket.io http://socket.io/

17. DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY
NO BUILD STEP FOR RESOURCE
OPTIMIZATION!
Resource
bundling, minification, compilation, externalization—
all at runtime.
Compile Raptor
Templates, LESS, Dust, CoffeeScript, etc.
MORE INFO:
• RaptorJS Optimizer: http://raptorjs.org/optimizer/

18. DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY
Command Line Tools
Powered by Rápido
Features
• Module and application lifecycle
• Scaffolding
• IDE/editor independence
• PaaS integration
• Jenkins CI integration
• Command-line auto-completion
• Colorized output
Example:
ecube create app
ecube create page
ecube register
ecube create pool
ecube publish
ecube deploy
CHECKOUT:
Rápido: https://github.com/raptorjs/rapido

19. DESIGN GOAL:
HIGH PERFORMANCE

20. DESIGN GOAL: HIGH PERFORMANCE
PERFORMANCE IS GREAT*
• Node.js provides a very low-level programming platform
• V8 is a powerful and fast JavaScript engine
• Node.js can be extended with C/C++ libraries
• Clustering based on OS-level socket sharing
• High performance evented I/O at its core (libuv)
• Single-threaded so no synchronization issues or locks
* The not-so-great:
• Garbage collection is slow
• Memory usage can easily get out of control

21. DESIGN GOAL: HIGH PERFORMANCE
STREAMING, ASYNCHRONOUS
UI RENDERING
• Why?
•
•
•
•
Start rendering the template immediately
Give the browser a head start
Let the template drive the data
Minimize idle time on the server

22. DESIGN GOAL: HIGH PERFORMANCE
Traditional
ASYNCHRONOUS UI RENDERING
1. Make service calls
2. Wait for all data to come back
3. Prepare view model
Input
Controller
View Model
Template
HTML
Long Delay
CubeJS
1. Setup data providers
2. Go straight to the
template!
Data Providers
Input
Controller
View Model
No Delay
Template
HTML

23. DESIGN GOAL: HIGH PERFORMANCE
Raptor Templates
Extensible, high performance, asynchronous template
rendering engine for JavaScript that works on both the
server and the client
MORE
INFO:
Raptor Templates: http://raptorjs.org/raptor-templates/

24. DESIGN GOAL:
ROBUST

25. DESIGN GOAL: ROBUST
cluster2
•
•
•
•
•
•
•
Multi-process cluster management
Live production debugging
Monitoring
Idle timeouts
Pause and resume (out-of-traffic GC!)
Process warm-up (coming soon)
Shared cache (coming soon)
MORE INFO:
https://github.com/cubejs/cluster2

26. DESIGN GOAL: ROBUST
Monitoring Infrastructure
Adapted to eBay Ops
Monitoring Tools

27. DESIGN GOAL: ROBUST
Automated Testing
• Netmorphic
• Simulate network failures and slowness
• Mocha
• Asynchronous tests
• Code coverage reporting
• JSHint
• Flexible JavaScript static code analysis tool
• Casper + PhantomJS (in-progress)
• Headless browser testing
• Jenkins CI

28. DESIGN GOAL: ROBUST
Configuration Management
• Per-module configuration
• Real-time configuration updates
• Configuration repository based on MongoDB

29. ASYNCHRONOUS PROGRAMMING
USING PROMISES
All CubeJS modules have standardized on promises for
asynchronous programming.
Example:
Q()
.then(loadUserPreferences)
.then(performSearch)
.then(renderSearchResults)
.fail(function(error) {
res.end();
console.error("Something went wrong!")
})

30. ADOPTION:
PULL RATHER
THAN
PUSH
• 10+ customers before CubeJS goes GA
• Several live CubeJS applications running in
production
• Overwhelming attendance of CubeJS Tech Talks
• ―Word of Mouth‖ spreading within eBay
application teams
• No need to push

31. QUESTIONS?
psteeleidem@ebay.com
@psteeleidem
https://github.com/cubejs
http://raptorjs.org