Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Scala @foursquare
     Jorge Ortiz
    @jorgeortiz85
foursquare runs on...

• Scala 2.8.0
• Lift 2.1
• MongoDB
• Twitter open source projects
• ~46k lines of Scala
foursquare server team


• 9 out of 11 had no major prior Scala experience
• Previously: Java, Python, C#
• Most exposed t...
foursquare loves Scala

• Pimped: Option, List, Map
• Type-safe equality: =?
• BackgroundOperation { ... }
• Throttle(“sec...
2.7.7 => 2.8.0
           (painless migration)

• Compile: 2.7.7 with -deprecation
• Branch
• Compile 2.8.0 without -depre...
foursquare on Lift

• REST API
• Out-of-the-box:
  • Rich Ajax/Comet support
  • ORMs for PostgresQL, MongoDB
  • Random: ...
Lift Deployment


• nginx
• HAProxy
• API machines - stateless
• WWW machines - stateful
GC issues


• Stateless API machines
  • Tens of millions of requests per day
  • Only a dozen or so servers
  • 12 GB hea...
GC issues
• GC tuning?
  • “There's no way to get around the fact that
    we're allocating faster than we can collect.”

...
GC issues

• Stateful WWW machines
  • Stateful Lift + Scala closures + Java GC
  • Lots of variables getting caught in Se...
MongoDB in Scala

• Some support built-in to Lift
• No expressive query language, so we wrote one:
 Venue where (_.mayor e...
Twitter open source


• Kestrel
• Ostrich
• GrabbyHands
• (probably more soon...)
We’re hiring
http://foursquare.com/jobs
Questions?
Scala at foursquare
Nächste SlideShare
Wird geladen in …5
×

Scala at foursquare

4.368 Aufrufe

Veröffentlicht am

Presentation at the New York Scala Meetup - 10/20/2010

Veröffentlicht in: Technologie, Bildung
  • Als Erste(r) kommentieren

Scala at foursquare

  1. 1. Scala @foursquare Jorge Ortiz @jorgeortiz85
  2. 2. foursquare runs on... • Scala 2.8.0 • Lift 2.1 • MongoDB • Twitter open source projects • ~46k lines of Scala
  3. 3. foursquare server team • 9 out of 11 had no major prior Scala experience • Previously: Java, Python, C# • Most exposed to: ML, Lisp
  4. 4. foursquare loves Scala • Pimped: Option, List, Map • Type-safe equality: =? • BackgroundOperation { ... } • Throttle(“secret feature!”) { ... } • Any Java library
  5. 5. 2.7.7 => 2.8.0 (painless migration) • Compile: 2.7.7 with -deprecation • Branch • Compile 2.8.0 without -deprecation • Compile 2.8.0 with -Xmigration • Compile 2.8.0 with -Xwarninit • Switch
  6. 6. foursquare on Lift • REST API • Out-of-the-box: • Rich Ajax/Comet support • ORMs for PostgresQL, MongoDB • Random: Email, Facebook, JSON, utils...
  7. 7. Lift Deployment • nginx • HAProxy • API machines - stateless • WWW machines - stateful
  8. 8. GC issues • Stateless API machines • Tens of millions of requests per day • Only a dozen or so servers • 12 GB heap => 15sec GCs *ouch*
  9. 9. GC issues • GC tuning? • “There's no way to get around the fact that we're allocating faster than we can collect.” • Allocating ~200 MB/s • Opportunity to reduce wasteful allocations • Better load-balancing strategy • Cap # of concurrent connections
  10. 10. GC issues • Stateful WWW machines • Stateful Lift + Scala closures + Java GC • Lots of variables getting caught in Session • val venues: RequestCache[List[Venue]] = RequestCache(Venue.findAll(...))
  11. 11. MongoDB in Scala • Some support built-in to Lift • No expressive query language, so we wrote one: Venue where (_.mayor eqs userid) modify (_.tags push “I’m mayor!”) updateMulti
  12. 12. Twitter open source • Kestrel • Ostrich • GrabbyHands • (probably more soon...)
  13. 13. We’re hiring http://foursquare.com/jobs
  14. 14. Questions?

×