How PubSub helped build Vox
Media’s multi-component,
high-volume data applications
Moses Musaelian | Vox Media
Current 2022

Introduction
● ML Engineer at Vox Media for over a year
● Working in Revenue Data Team

Introduction - You visit a site…

Bid Enrichment Flow

Bid Enrichment Flow - Trace

Google Pub/Sub - “message passing system for distributed applications in GCP”

Use Case #1: ML Service Design : “Latency Sensitive” Approach
Expect 800 bids per second!
Time out at 250ms!
“Lighten the load” off of our
front-facing API and let the
other components subscribe to
our message service

Use Case #2: DataFlow Logging (Viewability Data Application)
API Layer
Memorystore
Pubsub
Dataflow
BigQuery
Request Response

Use Case #3: Real Time Metrics from Pub/Sub (GCP Cloud Monitoring API v3)
Requirement → Don't write data to a
single time series faster than one point
each 5 seconds

Concluding Thoughts
API Layer
→ Metric send
→ Logging send
→ Worker send
API Layer
→ PubSub Send