This presentation was prepared for a Webcast where John Yerhot, Engine Yard US Support Lead, and Chris Kelly, Technical Evangelist at New Relic discussed how you can scale and improve the performance of your Ruby web apps. They shared detailed guidance on issues like:
Caching strategies
Slow database queries
Background processing
Profiling Ruby applications
Picking the right Ruby web server
Sharding data
Attendees will learn how to:
Gain visibility on site performance
Improve scalability and uptime
Find and fix key bottlenecks
See the on-demand replay:
http://pages.engineyard.com/6TipsforImprovingRubyApplicationPerformance.html
6. Database Performance
Lazy loading associated data can quickly
lead to an N+1 query problem.
ORMs (ActiveRecord, DataMapper, etc.) make it easy to
get our data but also make it easy to forget to optimize and
refactor.
N+1 problems are hard to spot in development since you
are working with limited data sets.
7. N+1 Query Creep
# app/models/customer.rb
class Customer < ActiveRecord::Base
has_many :addresses
end
# app/models/address.rb
class Address < ActiveRecord::Base
belongs_to :customer
end
# app/controllers/customers_controller.rb
class CustomersController < ApplicationController
def index
@customers = Customer.all
end
end
# app/views/customers/index.html.erb
<% @customers.each do |customer| %>
<%= content_tag :h1, customer.name %>
<% end %>
8. N+1 Query Creep
# app/views/customers/index.html.erb
<% @customers.each do |customer| %>
<%= content_tag :h1, customer.name %>
<%= content_tag :h2, customer.addresses.first.city %>
<% end %>
If @customers has 100 records, you'll have 101 queries:
SELECT "customers".* FROM "customers"
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 1
AND "addresses"."primary" = 't' LIMIT 1
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 2
AND "addresses"."primary" = 't' LIMIT 1
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 3
AND "addresses"."primary" = 't' LIMIT 1
...
...
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 100
AND "addresses"."primary" = 't' LIMIT 1
9. Eager Loading with .includes
# app/controllers/customers_controller.rb
class CustomersController < ApplicationController
def index
@customers = Customer.includes(:addresses).all
end
end
If @customers has 100 records, now we only have 2 queries:
SELECT "customers".* FROM "customers"
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" IN
(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21,
22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59,
60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78,
79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97,
98, 99, 100)
10. Finding N+1 in New Relic
New Relic > App Server > Web Transactions > Performance Breakdown
12. Adding an Index is Simple
# db/migrate/20120201040247_add_index_for_shop_id_on_orders.rb
class AddIndexForShopIdOnOrders < ActiveRecord::Migration
def change
add_index :orders, :shop_id
end
end
Index Protips:
• Searching an index on a table with 1,000 rows is 100x faster than searching a
table without an index.
• Put an index on any columns you will likely query against, it's better to have too
many than too few indexes.
• Adding an index to a table will lock the table!
14. Passenger 3
• Simple to operate
• Simple configuration
• Handles worker management
• Great for multi-application environments
• Great for low resource environments
• Attached to Nginx/Apache HTTPD
15. Passenger Request Queue
solo i-c3f2d8a2 ~ # passenger-status
----------- General information -----------
max = 3
count = 3
active = 0
inactive = 3
Waiting on global queue: 0
----------- Application groups -----------
/data/john_yerhot_org/current:
App root: /data/john_yerhot_org/current
* PID: 19802 Sessions: 0 Processed: 3 Uptime: 3h 10m 13s
/data/scalingrails/current:
App root: /data/scalingrails/current
* PID: 28726 Sessions: 0 Processed: 3 Uptime: 59m 22s
/data/sites/clmeisinger/current:
App root: /data/sites/clmeisinger/current
* PID: 22147 Sessions: 0 Processed: 70 Uptime: 10h 45m 57s
16. Unicorn
• Independent of front end web server
• More configuration options
• Master process will reap children on timeout
• Great for single application environments
• Allows for zero downtime deploys
17. Unicorn Request Queue?
Raindrops
solo i-5b74313d ~ # gem install raindrops
Fetching: raindrops-0.10.0.gem (100%)
Building native extensions. This could take a while...
Successfully installed raindrops-0.10.0
1 gem installed
solo i-5b74313d ~ # ruby -rubygems -e "require 'raindrops'; puts
Raindrops::Linux.unix_listener_stats(['/var/run/engineyard/
unicorn_appname.sock']).inspect"
{"/var/run/engineyard/unicorn_appname.sock"=>#<struct Raindrops::ListenStats
active=0, queued=0>}
20. Request Queuing in New Relic
• Time between first ActionContoller hit - X-Queue-Start = Time spent in queuing.
Internet => LB inserts X-Queue-Start => Nginx => Ruby Webserver => Rack =>
Application
Track Rack Middleware as well
def call(env)
env["HTTP_X_MIDDLEWARE_START"] = "t=#{(Time.now.to_f * 1000000).to_i}"
@app.call(env)
end
22. Cache Everything
Rails makes it
stupid easy to
cache everything.
Do it.
23. Static Files & Nginx
The best cache is a static file served by
Nginx.
# create it on #index, #show, etc..
caches_page :index
# expire it on #creates, #updates, #destory, etc...
expire_page :action => :index
24. A Note About Static Files:
Use the front end server.
upstream upstream_enki {
server unix:/var/run/engineyard/unicorn_enki.sock fail_timeout=0;
}
location ~ ^/(images|assets|javascripts|stylesheets)/ {
try_files $uri $uri/index.html /last_assets/$uri /last_assets/$uri.html
@app_enki;
expires 10y;
}
location / {
if (-f $document_root/system/maintenance.html) { return 503; }
try_files $uri $uri/index.html $uri.html @app_enki;
}
25. Memcached: The Standard
# config/initializers/memcached.rb
config.cache_store =:mem_cache_store,
"server-1:11211",
"server-2:11211",
"server-3:11211",
"server-4:11211"
26. Next Best: ActionCaching
Will still go through Rack/Rails, but the action gets
cached.
before_filter :make_sure_youre_ok
caches_action :all_the_things
def all_the_things
@all_things = Thing.all_in_a_complex_way
end
def expire
expire_action :action => :all_the_things
end
27. Fragment Caching
<% cache('my_cache_key') do %>
<%= render_large_tag_cloud %>
<% end %>
...
def update_large_tag_cloud
TagCloud.update
expire_fragment('my_cache_key')
end
28. Baremetal
Rails.cache.write("john", "yerhot")
Rails.cache.read("john")# => "yerhot"
# execute a block on miss and cache it.
Rails.cache.fetch("miss") do
"yerhot"
end
Rails.fetch("miss")# => "yerhot"
Rails.cache.exists("john") # => true
Rails.cache.delete("john") # => true
Rails.cache.exists("john") # => false
35. Rails 4
Background Processing baked in.
• Allow an application to switch job systems with minimal
code change due to common API
• Very basic queuing system built in
• Roll your own wrapper class that responds to push & pop
# application.rb
config.queue = QueueName
Rails.queue.push(Job.new)
36. Review
• You need to be monitoring your application.
• Performance has to be reviewed on a regular basis.
• Database indexes are cheap, make lots of them.
• Every application can take advantage of some level
of caching: page, action or fragment.
• Background any work that you can.
• Don't neglect front-end performance.
37. How to Install New Relic
New Relic Standard is Free at Engine Yard
1. If you’re an Engine Yard Customer, select your
plan in your Engine Yard Account Settings
2. Add newrelic_rpm to your Gemfile
3. Enable monitoring in the Engine Yard
Dashboard
Full Installation Details: http://ey.io/install-newrelic