Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie ScaleCamp 2009 - Last.fm vs Xbox
Ähnlich wie ScaleCamp 2009 - Last.fm vs Xbox (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
ScaleCamp 2009 - Last.fm vs Xbox
- 1. Scaling war stories • Last.fm vs Xbox360 • Quick and hacky profiling • Discuss Wednesday, 26 May 2010
- 2. What happens when scale is forced on you? • 24 Million Xbox360 users • Estimated ~5 times peak traffic increase • No control over source of client • External partner, with their own deadlines • Must be reliable at launch. Err? (The PR problem) Wednesday, 26 May 2010
- 3. A dose of reality • You won’t have the resources you’d like • People (skills, availability, • Budget (new servers would be nice) • Time (more of this would be nice too) Wednesday, 26 May 2010
- 4. B-b-but... • It’s not an ideal world • I’m not a systems developer (I spend half my time on the front end) • Quite hacky. Open to suggestions, alternatives, your experiences Wednesday, 26 May 2010
- 5. HTTP profiling • Had a development Xbox • Stuck a proxy box in the middle • Profiled traffic with Wireshark Wednesday, 26 May 2010
- 6. Wireshark Wednesday, 26 May 2010
- 7. Making a list... • API calls made • In order, as uses cases • Bad uses that can be factored out • What breaks when call X is disbabled • Proxy routed to a dev version of API Wednesday, 26 May 2010
- 8. Xbox Beta • Launched to 1,000 people (At midnight on a Friday). Then to a further 9,000 people • Gave a nice real usage sample • Great, the most common calls are... completely pointless. • frequent != expensive (necessarily) Wednesday, 26 May 2010
- 9. Custom Logging gen-time LogFormat "%h %l %u %t "%r" %>s %b "% {Referer}i" "%{User-Agent}i" %{gentime}n" <?php apache_note("gen-time", $x); Wednesday, 26 May 2010
- 10. Abusing bash Text awk 'BEGIN {FS="t"} {a[$2]++;b[$2]=b[$2]+$1} END {for (i in a) printf("%.2f %sn", b[i]/a[i]/ 1000000, i)} ' Wednesday, 26 May 2010
- 11. Stats Count Total Average 71638 trackgetInfo 15789 0.22 53941 artistgetImages 19647 0.36 15150 radiogetPlaylist 6962 0.46 7308 librarygetArtists 2402 0.33 5020 usergetRecentStations 1674 0.33 4979 adsgetvideos 1810 0.36 4205 radiotune 923 0.22 3155 tracklove 0 0.00 1507 artistgetinfo 440 0.29 1258 usergetRecommendedArtists 1457 1.16 1135 usergetInfo 171 0.15 1130 geogetTopArtists 575 0.51 1128 radiogamerstations 298 0.26 1102 taggettopartists 1488 1.35 1021 trackban 0 0.00 1006 usergetLovedTracks 271 0.27 Wednesday, 26 May 2010
- 12. zomg, graphs! Wednesday, 26 May 2010
- 13. Wednesday, 26 May 2010
- 14. Wednesday, 26 May 2010
- 15. Wednesday, 26 May 2010
- 16. Profiling with Xdebug • Profile PHP, stack traces, memory usage, etc • http://xdebug.org extension=xdebug.so xdebug.max_nesting_level=500 xdebug.default_enable=0 xdebug.profiler_enable_trigger=1 xdebug.profiler_append=1 xdebug.trace_output_name= %R.cache.out Wednesday, 26 May 2010
- 17. Making sense of it Wednesday, 26 May 2010
- 18. Contingencies • From invisible, to user affecting • Disable DB writes, speed up and prevent blocking on failure • Suppress badly architected Flash Player (and less evil taste-o-meter) • Kill % of method calls at perlbal Wednesday, 26 May 2010
- 19. Wednesday, 26 May 2010
- 20. Monitoring porn • Launch control with: 3 projectors, 5 x 24” monitors, a dozen dev/sysops with laptops • A moose, a cow and a fail train Wednesday, 26 May 2010