Technical overview of the CartoDB platform, describing the most important pieces of an architecture that serves millions of maps every day.

1. CartoDB
Inside Out
Jorge Sanz · jorge@cartodb.com · @xurxosanz
SASIG · OSGeo.pt · Lisbon · 2015
A technical overview of a web mapping platform

2. Agenda
● Why
● CartoDB nowadays
● Architecture of the platform
○ Components
○ Setup
● How to serve dynamic data at scale
No maps in this talk sorry

3. https://congosto.cartodb.com/viz/e5da12e2-9fe7-11e4-bc43-0e853d047bba/embed_map

4. http://elpais.com/elpais/2015/08/20/media/1440070260_752772.html

5. Why
● @jatorre and @saleiva start Vizzuality in April 2008
● Company focused on visualization projects
● Every geospatial project had the same stack:
○ Database + Map Server + APIs + client code
● CartoDB: internal product to deploy faster this geo-stack

6. Mission
With CartoDB everybody can analyse,
visualise and share insights on location data.
We are democratising location intelligence

7. CartoDB nowadays
● A VC founded company of ~70 people (NY + Madrid)
● A leading mapping service with +130.000 users
● An Open Source product that can be deployed
on the cloud or on just one server (on-premises)
● Full stack solution from storage to client side libraries
● It’s a mapping editor supported by a platform (APIs)

8. CartoDB architecture

9. http://www.di.unito.it/~schifane/smellymaps/explorer.html

10. https://team.cartodb.com/u/piensaenpixel/viz/8a6530e0-5239-11e5-9cc4-0e0c41326911/public_map

11. The CartoDB
platform
● Spatial database for users
● NoSQL database for
configurations and cache
● Rendering, imports,
database and other services
● Cache and Content
Delivery Network
● JavaScript Library
● Map Editor
Platform
On-premises
or in the Cloud
Data services
Your data
Editor

12. The CartoDB platform
CartoDB.js Odyssey.jsTorque.js Leaflet /Gmaps
PostgreSQL
PostGIS
SQL API
Redis
Maps API
TorqueMapnik
Varnish Caching
Fastly CDN
CartoDBonAmazon
elasticRecipes
CartoDBon
Amazonelastic
Recipes
PostgreSQL
PostGIS
CartoDB Server API
Rest API Imports API
CartoDB Editor
Browser

13. Cloud setup
● 1 - load balancer (nginx)
● 1 - caching (varnish)
● N - UI servers (CartoDB UI)
● N - database servers (user data)
● 1 - Metadata server (CartoDB metadata)
● 1 - redis server (+slave)
● N - Maps API + API SQL servers

15. Postgres + Postgis
● Geodata, users and editor metadata storage
● Tiny Well Known Binary
● CartoDB - Postgres extension
Redis
● key-value highly performant storage
● configuration for APIs
● cache

16. Backend services
● Ruby on Rails
● Import API (ogr2ogr)
● Background management tasks
Invalidation service
● Node.js + HA Proxy + Consul
● Invalidates caches by request
● Multi-agent, replicated

17. SQL API
● node.js
● Exposes an entry point for running queries
● Acts as an export API
Maps API · Windshaft
● node.js + mapnik
● renders tiles from CartoCSS + SQL
● SQL API client

18. nginx
● web server
● load balancer
Varnish + Fastly/GCP
● web cache
● Content Delivery Network

19. CartoDB
Map Editor
the easiest web
mapping tool
ever

20. Map Editor
● Ruby on Rails + JavaScript
● Consumes the Imports, Maps, SQL API and its own API
● Highly focused on UX-UI
● Manage datasets (schema, privacy, etc)
● Manage maps (basemap, infowindows,
other visual elements, privacy, etc)
● Define layer data source with filters and SQL editor
● Define symbology through wizards and CartoCSS editor
● Publish maps as embeds, links or with CartoDB.js

21. CartoDB.js
● Based on jQuery + Backbone + Underscore
● Leverages Leaflet or Google Maps API
● Easy interaction with the SQL and Maps API
● Manages transparently utf-grids for interactivity
● Allows easy customizations on infowindows,
layer SQL and CartoCSS definitions, etc

22. Serving data at scale

23. https://scordivano.cartodb.com/viz/f55a3268-0c29-11e3-827e-5fd91394b353/embed_map

24. http://mappingkat.github.io/indian-removal/

25. Story of a tile: first request

26. Story of a tile: next request

27. Story of a tile: insert & next request

28. Cache
invalidation
Write queries fire a trigger
that updates
cdb_tablemetadata and
starts the invalidation
process for the tiles that use
that table

29. Summary
● CartoDB is an Open Source, full stack mapping product
● Runs on a single server or distributed
● Delivers to thousands of users the superpowers of Postgis
and Mapnik, not just pretty maps but also new insights
● Provides tools for users and developers
● It’s designed to scale, but we are always improving it

30. We are hiring!!
http://cartodb.com/jobs

31. CartoDB
Inside Out
Jorge Sanz · jorge@cartodb.com · @xurxosanz
SASIG · OSGeo.pt · Lisbon · 2015
http://bit.ly/cartodb-sasig2015
Muito obrigado! · Questions?

32. Links
● https://cartodb.com
● https://github.com/CartoDB/cartodb
● https://github.com/CartoDB/cartodb.js
● https://github.com/CartoDB/cartodb-postgresql
● https://github.com/CartoDB/Windshaft-cartodb
● https://github.com/CartoDB/CartoDB-SQL-API
● http://docs.cartodb.com
● http://cartodb.readthedocs.org