Cvcc performance tuning
•Als ODP, PDF herunterladen•
1 gefällt mir•952 views
Web Performance tuning presentation given at http://www.chippewavalleycodecamp.com/ Covers basic http flow, measuring performance, common changes to improve performance now, and several tools and techniques you can use now.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Cvcc performance tuning
Ähnlich wie Cvcc performance tuning (20)
Mehr von John McCaffrey
Mehr von John McCaffrey (7)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Cvcc performance tuning
- 2. John McCaffrey: Its all about me • Doing Java since 2000, Rails since 2007 • Presented at WindyCityRails – 2008: Advanced Firebug and JS unit testing – 2009: PDF Generation in Rails – 2010: Rails Performance Tuning • Addicted to Performance Tuning • railsperformance.blogspot.com • Feedback: http://spkr8.com/t/4961 @J_McCaffrey railsperformance@gmail.com (the slides will be available, with additional references)
- 3. What are we gonna cover? • Performance touches a lot of layers • We have a very diverse audience
- 4. Measuring Page Load Fixing page load problems Analyzing Server Performance Database issues Ruby issues Q&A Starting off on the right foot
- 5. It all started... def organization_branches get_organizations.flatten.uniq.select{|org| org.children.size > 0} end def organization_leaves get_organizations.flatten.uniq.select{|org| org.children.size == 0} end def load_organizations branches = current_user.organization_branches leaves = current_user.organization_leaves @branches = branches.to_json(:only => [:id, :name]) @leaves = leaves.to_json(:only => [:id, :name]) @organizations = @criteria.branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end
- 6. def organization_branches get_organizations.flatten.uniq.select{|org| org.children.size > 0} end def organization_leaves get_organizations.flatten.uniq.select{|org| org.children.size == 0} end def load_organizations branches = current_user.organization_branches leaves = current_user.organization_leaves @branches = branches.to_json(:only => [:id, :name]) @leaves = leaves.to_json(:only => [:id, :name]) @organizations = @criteria.branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end It all started... 2811 queries! Down to 17!!
- 7. And then... Invoicing issue: If a customer resubmits an order with 2 years, they should be charged $9 but were only being charged $4.50 How common are ROI calculations on software projects? 53626 * $4.50 = $241,317 (just in the last year)
- 8. Our theme of the day: Leverage to achieve the greatest value
- 9. Performance and Business metrics: Bing.com Slow site = revenue drop 1 sec = 2.8% 2 sec = 4.3%
- 10. Performance and Business metrics: Speed matters Amazon: 100 ms delay caused a 1% drop in revenue. Google: 400 ms delay caused a 0.59% decrease in search requests per user. Yahoo!: 400 ms delay caused a 5-9% decrease in traffic. Bing: 2 sec delay caused a 4.3% drop in revenue per user. Mozilla made their download page 2.2 seconds faster and saw an increase of 15.4% in downloads. Google Maps reduced the file volume by 30% and observed a 30% increase in map requests. Netflix enabled gzip on the server; pages became 13-25% faster and saved 50% of traffic volume!
- 11. Performance and Business metrics: Shopzilla.com Went from 6 sec down to 1.2sec http://www.phpied.com/the-performance-business-pitch/
- 12. Fred Wilson: Speed is the most important feature 10 Golden Principles of Successful Web Apps Fred Wilson, Venture Capitalist www.avc.com •Speed• Instant Utility • Software is media • Less is More • Make it programmable • Make it personal • Make it RESTful • Discoverability • Clean • Playful
- 13. OK, so where do we begin?
- 14. Give yourself a fighting chance, make a plan Alois Reitbauer, dynatrace Anti-patterns • Thinking Scalability something you just sprinkle on later • Guessing, not testing • Ad-hoc, unstructured • Waiting until it hurts, not proactively planning • Thinking that any early planning is Premature optimization Video on Parleys.com
- 15. Premature optimization "Premature optimization is the root of all evil" -Donald Knuth
- 16. Premature optimization Some people really like Donald Knuth
- 17. Premature optimization, with DJ DK like, really, really like him
- 18. Premature optimization "Premature optimization is the root of all evil' -Donald Knuth "Quoting Knuth, as a way to avoid planning for performance issues, is a cop-out" @J_McCaffrey Don't waste time on making optimization changes until you KNOW they are necessary
- 19. If you can not measure it, you can not improve it. — Lord Kelvin
- 20. Let's do this the right way Measure: • Understand the tools to use, at each layer, to measure and monitor performance Create repeatable tests: • Be able to invoke the system and easily test • A/B, before/after tests, long-term monitoring Isolate your changes: • One thing at a time That's it: Measure, Test, and Isolate your changes
- 21. Terminology Load: How much work is there? Utilization: How much of the resources are in use? Scalability: How well can we handle the load? Throughput: How many tasks can be done per unit of time? Concurrency: How many tasks can we do at once? Capacity: How big can our throughput go before things fall apart? Response time (avg) == Latency Requests per second == Throughput
- 22. Terminology Performance: • The resources and Time for a single request • Measured in Response Time Throughput: • The number requests that can be handled per/time • Measured in Requests per second Scalability: • Ability to grow and handle more requests, without significantly degrading the experience Performance != Scalability
- 23. Terminology Let's say: 1 worker that can compute a job in .5 sec Throughput = 2 jobs per sec Latency = .5 sec per job Performance != Scalability Adding more workers will not improve the latency But not having enough workers can increase the response time, due to Queuing and delay
- 24. There are performance opportunities at each layer
- 25. Waterfall: Twitter.com Waterfall Total time: 3815 ms Dynamic generation: 250 ms (~7%) Static: 3369 ms (~89%) * Will not add to 100% When the html is downloaded When the rest of it is loaded
- 26. Waterfall When the backend is actually doing work
- 27. Walmart: Waterfall Waterfall Total time: 5520 ms Dynamic: 388 ms (7%) Static: 4580 ms (~82%)
- 28. Firebug Netpanel Simple and repeatable way to measure page load Usually, less than 20% of the time is spent in fetching the HTML So how do we optimize the other 80%?
- 29. YSlow • Created by Steve Souders • Observes the netpanel data, applies rules and gives you a score. • Deep understanding of how the browser works • Its the best way to have a repeatable, measurement of your site's load performance http://developer.yahoo.com/yslow
- 30. YSlow • Minimize HTTP Requests • Use a Content Delivery Network • Add an Expires or a Cache-Control Header • Gzip Components • Put StyleSheets at the Top • Put Scripts at the Bottom • Avoid CSS Expressions • Make JavaScript and CSS External • Reduce DNS Lookups • Minify JavaScript and CSS • Avoid Redirects • Remove Duplicate Scripts • Configure ETags • Make AJAX Cacheable • Use GET for AJAX Requests • Reduce the Number of DOM Elements • No 404s • Reduce Cookie Size • Use Cookie-Free Domains for Components • Avoid Filters • Do Not Scale Images in HTML • Make favicon.ico Small and Cacheable
- 31. Yslow: Created by Steve Souders http://stevesouders.com/cuzillion/
- 32. Google Page speed • Similar to YSlow • includes paint events • separates out ads, trackers from your content • better recommendations and optimizations • 1 click to see optimized version of your files http://code.google.com/speed/page-speed
- 35. Make it repeatable http://www.showslow.com/ track your site, or your competitors will check every 24hrs for you • webpagetest.org • gtmetrix.org • zoompf.com • loadimpact.com • gomez.com
- 36. Summary • Page load is the place to start • Read everything in the Yslow and PageSpeed rules • Encourage other devs to play with YSlow, PageSpeed • Show Webpagetest.org, and zoompf.com to the boss Now that we have a way to measure page performance, we can think about making changes
- 37. Measuring Page Load Fixing page load problems Analyzing Server Performance Database issues Ruby issues Q&A Starting off on the right foot
- 38. Isn't everyone already doing this? Analysis of the Alexa top 1000 sites found: • 42% don't gzip • 44% have more than 2 css files • 56% serve css from a cookied domain • 62% don't minify • 31% have more than 100k size css Top 1000 retail sites 50% aren't doing both keep-alive and compression (the 2 easiest things!!)
- 39. Compress with Gzip • Can save you 50% to 80% of your bandwidth • Simplest performance improvement you can make • Beware differences in browsers • File types: html, js, css, xml, rss, txt, font, json Probably the simplest and smartest thing you can do for your app!
- 40. Combine and minify javascript and css Rails • javascript_include_tag :all, :cache => "all" • Asset_packager, Jammit Load common js libraries from google http://github.com/rpheath/google_ajax_libraries_api Jsmin or YUICompressor Closure compiler Let's check it out!
- 41. Combine and minify javascript and css Sprites and image optimization • spriteme.org • JqueryUI already gives you sprited images Image optimization • Smush.it
- 42. Expires and Browser Caching Setting a far future expires tells the browser not to ask for the file Expires:Mon, 02 Sep 2030 21:23:08 GMT /images/logo.png?1234567890 Using the query string lets us break the caching by having the browser ask for a new file There is a problem with using a URL query string this way http://blog.eliotsykes.com/2010/05/06/why-rails-asset-caching-is-broken/ /images/logo-fp-839b180ff39a24f8d6e0ee70e4c40fed.png
- 43. Rails Caching Take your dynamic content, and make it static • Page • Action • Fragment Full length tutorials • http://railslab.newrelic.com/ • http://railscasts.com/episodes?search=caching • Greg Pollack Aloha on Rails http://vimeo.com/10860860
- 44. Measuring Page Load Fixing page load problems Analyzing Server Performance Database issues Ruby issues Q&A Starting off on the right foot
- 45. The Stack You are here
- 46. Request-log-analyzer • http://github.com/wvanbergen/request-log-analyzer • Request distribution per hour • Most requested • HTTP methods • HTTP statuses returned • Rails action cache hits • Request duration • View rendering time • Database time • Process blockers • Failed requests
- 47. Rack::Bug
- 48. Rack::Bug
- 49. Commercial tools New Relic • Developer plugin is a good start • Free version doesn’t get you much (Bronze version with heroku) • Works for .NET • Scoutapp.com • Similar to new Relic • Large selection of community plugins
- 50. Testing tools Apache bench ab -n 10 -c 2 http://www.somewhere.com/ Httperf httperf --server localhost --port 3000 --uri / --num-conns 10000 Jmeter yes, its ugly, but its powerful
- 51. Measuring Page Load Fixing page load problems Analyzing Server Performance Database issues Q&A Starting off on the right foot
- 52. Database issues Common database issues • Bad queries • Not utilizing explain • Inadequate indexes • N+1 queries • Selecting more data than is needed • Inconsistent queries for the same data
- 53. Query_reviewer Common database issues • Bad queries • Not utilizing explain and slow query log • http://github.com/dsboulder/query_reviewer • Runs explain on all of your queries, outputs to div in page
- 54. Rails_indexes http://github.com/eladmeidar/rails_indexes Foreign key indexes • Columns that need to be sorted • Lookup fields • Columns that are used in a GROUP BY • Rake tasks to find missing indexes. New one I haven’t tried yet http://github.com/samdanavia/ambitious_query_indexer
- 55. Bullet plugin Bullet plugin • http://github.com/flyerhzm/bullet Help you reduce the number of queries with alerts (and growl).
- 56. Slim_scrooge Be more specific in your select • http://github.com/sdsykes/slim_scrooge • Instruments your code • Observes the usage pattern • Suggests/changes your select statement • When invoked within the exact same context
- 57. Ruby Issues Most are due to memory problems • Slow GC • Not release references • Capturing scope • Profiling will reveal what’s going on • http://guides.rubyonrails.org/performance_testing.html If you want to learn everything there is to know about profiling • Aman Gupta & Joe Damato • http://timetobleed.com/ • http://memprof.com/ • They’ve already identified and fixed several memory leaks
- 58. Ruby issues: Faster Ruby Libraries C Extension++ • XML parser http://nokogiri.org/ • JSON parser http://github.com/brianmario/yajl-ruby/ • CSV parser http://www.toastyapps.com/excelsior/ • HTTP client http://github.com/pauldix/typhoeus Date http://github.com/jeremyevans/home_run
- 59. Take home points Performance • Performance affects the bottom line • The biggest performance win is usually in improving the load time • Continue to monitor and test your apps performance over time • Gzip, combine and minify to get a big boost • Lack of indexes is likely to be one of your biggest backend issues • Try to stay up to date with Libraries and patches • A well tested codebase is easier to tune! • Upgrading to ruby 1.9 will give you a huge performance boost
- 60. Links: great info on performance Performance • Scaling Rails series http://railslab.newrelic.com • RailsCasts.com • Velocity Conf http://en.oreilly.com/velocity2010 • Google io conf http://code.google.com/events/io/2010/ • dynatrace site http://ajax.dynatrace.com/pages/ • http://www.igvita.com • http://www.mysqlperformanceblog.com
- 61. Wait, I made it through that? Designing for performance • Watch how they really use it • Multiple tabs? • Back and forth between list and detail view • Great presentation at WindyCityRails2009 by David Eisinger • Optimizing perceived performance • Make it feel fast
- 62. Wait, I made it through that? Architectural musings •Background long running tasks • Email, image processing, reports, feeds •Leverage Rack, Sinatra or Padrino •Parallel processing, Event Machine •Learn from heroku, scalable from the start •Revaluate your cloudy-ness • Great RailsConf presentation by James Golick
- 64. Beyond ySlow: improving page performance outside the initial load MySql and Postgres Database tuning Heroku: Building Scalable apps from the start Tuning performance for Mobile devices Performance Improvements coming in Rails 3, and 3.1 Speeding up your tests
- 65. Questions for you Architectural musings • Moving to or already on Ruby 1.9? • Serving your content from a separate domain • Cookieless?
Hinweis der Redaktion
- I really enjoy helping people get the most out of their apps add some color to these slides!
- https://spreadsheets.google.com/gform?key=0AhVz5gfAE-VYdHFfRHBDcFF5dktTX0pTeVR6am5DQkE&hl=en&pli=1&gridId=0#chart
- There is so much to cover, and I want to make sure that everyone gets something valuable out of this talk, whether you are just starting out, or you work for Groupon, serving mils of hits a day. My main objective is to help you get started. To know where to look, which tools to use, which techniques to investigate, and where to find the resources you need. I won't be able to teach you everything about performance, but I can at least get you excited, and point you in the right direction. If i don't teach you something new, or you don't get what you were looking for, please come up and talk to me. We can chat afterwards, if you have a specific issue, I'm sure I can help you out.
- how I got started customer complained report page was slow in prod first 2 methods are from the model, the 3rd is from the controller This code basically sets up 2 drop downs to pick which organization you want to run a report for. I knew it was slow, and I knew it was wrong, but I didn't know all the ways that it was wrong
- got my first taste of poorly performing Rails code I learned: test with real data review your queries let the database do what it do watch out for N+1 queries avoid unnecessary columns add indexes optimize the page lack of testing, leads to an odd pattern of reuse without refactoring or improving It was a rewarding experience knowing that a few basic changes were now going to improve the performance in several places in the app, and make a lot of people very happy.
- The next day they asked me if I could look into a discrepancy they had in the billing code. This only occurs if they have to resubmit due to their own delay, and then only if they have more than one year ordered. I fixed the issue, but then became ridiculously curious about how much this bug had cost. Its not just having 2yrs, its having more than one, and there are several orders with 3 or 4yrs. It was one of the first times I knew how much a bug cost, and could measure the return on the time it took to fix it. i then became obsessed with knowing how much a unit of development was going to be worth. It was a powerful feeling, to have real numbers. This is info that the rest of the business world is used to having, and how McDonald's knows that it can sell 59cent cheesburgers. What's the ROI? When will we break even?
- These 2 experiences, where I was able to observe massive improvements with a small but focused effort, encouraged me to seek out more opportunities to maximize return Leverage is an important theme here. When it comes to making the right performance decisions, we want to use the greatest leverage, and get big gains for small effort. Gzip is a great example of leverage Open source software Jake’s talk on metric_fu is a great example of leverage Nick’s talk on improving test performance is extremely valuable
- Improving performance is all about leverage, and its all about money The major sites know this, and you can see it in the work they do. Google just released instant search, because they know that faster search means more money google includes it in their adwords calculation velocity conference 2009 google did a similar experiment, adding a delay of only 400ms, and saw a .6% drop in searches, but most interestingly, even after they were done with the experiment,and removed the delay, those users still did significantly less searches,, showing that bad performance leaves a lasting impression Yahoo also added 400ms delay, and saw a 5-9% drop in full page traffic, meaning that as soon as the users started to feel a slow experience, they killed the page and went somewhere else (something to think about if you are the #3 player) http://www.phpied.com/the-performance-business-pitch/
- Improving performance is all about leverage, and its all about money The major sites know this, and you can see it in the work they do. Google just released instant search, because they know that faster search means more money google includes it in their adwords calculation velocity conference 2009 google did a similar experiment, adding a delay of only 400ms, and saw a .6% drop in searches, but most interestingly, even after they were done with the experiment,and removed the delay, those users still did significantly less searches,, showing that bad performance leaves a lasting impression Yahoo also added 400ms delay, and saw a 5-9% drop in full page traffic, meaning that as soon as the users started to feel a slow experience, they killed the page and went somewhere else (something to think about if you are the #3 player) http://www.phpied.com/the-performance-business-pitch/
- Shopzilla wasn't doing an experiment, they had a slow site and they needed to fix it big comparison site 100mil impressions aday 8000 requests per sec Improved response time from 6 down to 1.2 These slides from a deck that was put together for the purpose of explaining performance tuning to business users,
- http://thinkvitamin.com/business/fred-wilsons-10-golden-principles-of-successful-web-apps/ Said what they look for in the web apps they invest in "not just a feature, but the most critical feature" especially true of non-power users, users that have a choice As part of their due dillegence they investigate the speed of the app, and they continue to monitor it, like watching a stock ticker. They know that when the app gets slow, less users use it. Its a great video, you should check it out
- Any discussion of performance monitoring or measurement is met with sneers and the oft quoted 'Premature optimization is the root of all evil' http://wiki.parleys.com/display/PARLEYS/Home#talk=35061764;title=Common%20Performance%20Antipatterns
- I like Donald Knuth ka-nooth http://www-cs-faculty.stanford.edu/~uno/faq.html author of the are of software programming its true, and its worth reminding ourselves. We should forget about small efficiencies, say about 97% of the time: Premature optimization is the root of all evil
- http://laughingsquid.com/jacob-appelbaum-donald-knuth-demonstrate-the-recursive-homeboys-principle/
- http://laughingsquid.com/jacob-appelbaum-donald-knuth-demonstrate-the-recursive-homeboys-principle/
- We should forget about small efficiencies, say about 97% of the time: Premature optimization is the root of all evil some people quote it, and to them it means even thinking about performance is premature
- Let's make a distinction between premature optimization, and baseline measuring. we're gonna make sure we measure, and tackle the most critical problems Measuring your performance is not premature optimization Makes sense for capacity planning SLA Should line up with performance expectations of the project You need to establish the expectations, state your assumptions, and keep an eye on things
- why do people say that? scalability is a term that is thrown around a lot, its meaning becomes unclear, and people make bad decisions because of it Its possible to have an app with great performance, that can not scale And its possible to have an app that scales, but its performance is horrible
- We need to know what we are measuring Today we're going to focus on Response time and Requests per second
- why do people say that? scalability is a term that is thrown around a lot, its meaning becomes unclear, and people make bad decisions because of it Its possible to have an app with great performance, that can not scale And its possible to have an app that scales, but its performance is horrible
- why do people say that? scalability is a term that is thrown around a lot, its meaning becomes unclear, and people make bad decisions because of it 10 workers = 20req sec If latency goes up to a full sec... Its possible to have an app with great performance, that can not scale And its possible to have an app that scales, but its performance is horrible This kind of queuing and delay is what we want to eliminate, it represents time not working, just waiting
- Here's your typical stack at each one of these layers there are performance considerations Our goal will be to get each layer to do less work, and distribute the load from busy workers to not so busy workers Traditionally, performance tuning focuses on the backend, as it is logical that dynamic data takes more effort to serve up
- How do we know if our site is slow? What is it doing? waterfalls from zoompf
- How do we know if our site is slow? What is it doing?
- How do we know if our site is slow? What is it doing?
- steve souders
- steve souders http://stevesouders.com/cuzillion/
- steve souders
- now we have 2 tools that allow us to measure the speed of our app anyone on the team can do it. I encourage you to test out your apps, if you haven't already
- look up your competition, industry, etc You can ask showslow to check on a site for you everyday
- There is so much to cover, and I want to make sure that everyone gets something valuable out of this talk, whether you are just starting out, or you work for Groupon, serving mils of hits a day. My main objective is to help you get started. To know where to look, which tools to use, which techniques to investigate, and where to find the resources you need. I won't be able to teach you everything about performance, but I can at least get you excited, and point you in the right direction. If i don't teach you something new, or you don't get what you were looking for, please come up and talk to me. We can chat afterwards, if you have a specific issue, I'm sure I can help you out.
- www.stubornella.org
- netflix saw 43% decrease in bandwidth after they turned on compression
- google ajax library allows you to load local files, great when you are on the train to minimize delays due to network performance, and the browsers limitations on parallel downloads, we want to reduce http calls
- google ajax library allows you to load local files, great when you are on the train to minimize delays due to network performance, and the browsers limitations on parallel downloads, we want to reduce http calls
- There is so much to cover, and I want to make sure that everyone gets something valuable out of this talk, whether you are just starting out, or you work for Groupon, serving mils of hits a day. My main objective is to help you get started. To know where to look, which tools to use, which techniques to investigate, and where to find the resources you need. I won't be able to teach you everything about performance, but I can at least get you excited, and point you in the right direction. If i don't teach you something new, or you don't get what you were looking for, please come up and talk to me. We can chat afterwards, if you have a specific issue, I'm sure I can help you out.
- no instrumentation Rawk Pl_analyze Splunk
- There is so much to cover, and I want to make sure that everyone gets something valuable out of this talk, whether you are just starting out, or you work for Groupon, serving mils of hits a day. My main objective is to help you get started. To know where to look, which tools to use, which techniques to investigate, and where to find the resources you need. I won't be able to teach you everything about performance, but I can at least get you excited, and point you in the right direction. If i don't teach you something new, or you don't get what you were looking for, please come up and talk to me. We can chat afterwards, if you have a specific issue, I'm sure I can help you out.
- Images from ihower presentation
- Make sure you repeat their question
- what we couldn't cover this time, but might make for the latter half of a full day's tutorial...