This presentation was prepared for a Webcast where John Yerhot, Engine Yard US Support Lead, and Chris Kelly, Technical Evangelist at New Relic discussed how you can scale and improve the performance of your Ruby web apps. They shared detailed guidance on issues like: Caching strategies Slow database queries Background processing Profiling Ruby applications Picking the right Ruby web server Sharding data Attendees will learn how to: Gain visibility on site performance Improve scalability and uptime Find and fix key bottlenecks See the on-demand replay: http://pages.engineyard.com/6TipsforImprovingRubyApplicationPerformance.html

1. In Today’s Session You Will Learn how
to:
• Gain visibility on site performance
• Improve scalability and uptime
• Find and fix key bottlenecks

2. New Relic + Engine Yard
• Database
• Web Servers
• Caching
• Background Processing

3. Web Request Overview

4. Web Application Overview

5. DATA
BASE

6. Database Performance
Lazy loading associated data can quickly
lead to an N+1 query problem.
ORMs (ActiveRecord, DataMapper, etc.) make it easy to
get our data but also make it easy to forget to optimize and
refactor.
N+1 problems are hard to spot in development since you
are working with limited data sets.

7. N+1 Query Creep
# app/models/customer.rb
class Customer < ActiveRecord::Base
has_many :addresses
end
# app/models/address.rb
class Address < ActiveRecord::Base
belongs_to :customer
end
# app/controllers/customers_controller.rb
class CustomersController < ApplicationController
def index
@customers = Customer.all
end
end
# app/views/customers/index.html.erb
<% @customers.each do |customer| %>
<%= content_tag :h1, customer.name %>
<% end %>

8. N+1 Query Creep
# app/views/customers/index.html.erb
<% @customers.each do |customer| %>
<%= content_tag :h1, customer.name %>
<%= content_tag :h2, customer.addresses.first.city %>
<% end %>
If @customers has 100 records, you'll have 101 queries:
SELECT "customers".* FROM "customers"
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 1
AND "addresses"."primary" = 't' LIMIT 1
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 2
AND "addresses"."primary" = 't' LIMIT 1
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 3
AND "addresses"."primary" = 't' LIMIT 1
...
...
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 100
AND "addresses"."primary" = 't' LIMIT 1

9. Eager Loading with .includes
# app/controllers/customers_controller.rb
class CustomersController < ApplicationController
def index
@customers = Customer.includes(:addresses).all
end
end
If @customers has 100 records, now we only have 2 queries:
SELECT "customers".* FROM "customers"
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" IN
(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21,
22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59,
60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78,
79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97,
98, 99, 100)

10. Finding N+1 in New Relic
New Relic > App Server > Web Transactions > Performance Breakdown

11. Missing Indexes == Slow Queries

12. Adding an Index is Simple
# db/migrate/20120201040247_add_index_for_shop_id_on_orders.rb
class AddIndexForShopIdOnOrders < ActiveRecord::Migration
def change
add_index :orders, :shop_id
end
end
Index Protips:
• Searching an index on a table with 1,000 rows is 100x faster than searching a
table without an index.
• Put an index on any columns you will likely query against, it's better to have too
many than too few indexes.
• Adding an index to a table will lock the table!

13. RUBY
WEB
SERVERS

14. Passenger 3
• Simple to operate
• Simple configuration
• Handles worker management
• Great for multi-application environments
• Great for low resource environments
• Attached to Nginx/Apache HTTPD

15. Passenger Request Queue
solo i-c3f2d8a2 ~ # passenger-status
----------- General information -----------
max = 3
count = 3
active = 0
inactive = 3
Waiting on global queue: 0
----------- Application groups -----------
/data/john_yerhot_org/current:
App root: /data/john_yerhot_org/current
* PID: 19802 Sessions: 0 Processed: 3 Uptime: 3h 10m 13s
/data/scalingrails/current:
App root: /data/scalingrails/current
* PID: 28726 Sessions: 0 Processed: 3 Uptime: 59m 22s
/data/sites/clmeisinger/current:
App root: /data/sites/clmeisinger/current
* PID: 22147 Sessions: 0 Processed: 70 Uptime: 10h 45m 57s

16. Unicorn
• Independent of front end web server
• More configuration options
• Master process will reap children on timeout
• Great for single application environments
• Allows for zero downtime deploys

17. Unicorn Request Queue?
Raindrops
solo i-5b74313d ~ # gem install raindrops
Fetching: raindrops-0.10.0.gem (100%)
Building native extensions. This could take a while...
Successfully installed raindrops-0.10.0
1 gem installed
solo i-5b74313d ~ # ruby -rubygems -e "require 'raindrops'; puts
Raindrops::Linux.unix_listener_stats(['/var/run/engineyard/
unicorn_appname.sock']).inspect"
{"/var/run/engineyard/unicorn_appname.sock"=>#<struct Raindrops::ListenStats
active=0, queued=0>}

18. Request Queuing in New Relic

19. Request Queuing in New Relic
NOT COOL

20. Request Queuing in New Relic
• Time between first ActionContoller hit - X-Queue-Start = Time spent in queuing.
Internet => LB inserts X-Queue-Start => Nginx => Ruby Webserver => Rack =>
Application
Track Rack Middleware as well
def call(env)
env["HTTP_X_MIDDLEWARE_START"] = "t=#{(Time.now.to_f * 1000000).to_i}"
@app.call(env)
end

21. CACHING

22. Cache Everything
Rails makes it
stupid easy to
cache everything.
Do it.

23. Static Files & Nginx
The best cache is a static file served by
Nginx.
# create it on #index, #show, etc..
caches_page :index
# expire it on #creates, #updates, #destory, etc...
expire_page :action => :index

24. A Note About Static Files:
Use the front end server.
upstream upstream_enki {
server unix:/var/run/engineyard/unicorn_enki.sock fail_timeout=0;
}
location ~ ^/(images|assets|javascripts|stylesheets)/ {
try_files $uri $uri/index.html /last_assets/$uri /last_assets/$uri.html
@app_enki;
expires 10y;
}
location / {
if (-f $document_root/system/maintenance.html) { return 503; }
try_files $uri $uri/index.html $uri.html @app_enki;
}

25. Memcached: The Standard
# config/initializers/memcached.rb
config.cache_store =:mem_cache_store,
"server-1:11211",
"server-2:11211",
"server-3:11211",
"server-4:11211"

26. Next Best: ActionCaching
Will still go through Rack/Rails, but the action gets
cached.
before_filter :make_sure_youre_ok
caches_action :all_the_things
def all_the_things
@all_things = Thing.all_in_a_complex_way
end
def expire
expire_action :action => :all_the_things
end

27. Fragment Caching
<% cache('my_cache_key') do %>
<%= render_large_tag_cloud %>
<% end %>
...
def update_large_tag_cloud
TagCloud.update
expire_fragment('my_cache_key')
end

28. Baremetal
Rails.cache.write("john", "yerhot")
Rails.cache.read("john")# => "yerhot"
# execute a block on miss and cache it.
Rails.cache.fetch("miss") do
"yerhot"
end
Rails.fetch("miss")# => "yerhot"
Rails.cache.exists("john") # => true
Rails.cache.delete("john") # => true
Rails.cache.exists("john") # => false

29. Background
Processing

30. Why Background Processing?
• send email
• process images
• grab feeds and cache them
• complex computations/reports
• create/expire caches/pages (like Reddit)

31. Best Practice:
Use a utility server for background
jobs and cron.

32. Resque to the Rescue

33. Resque in New Relic

34. Delayed Job Too

35. Rails 4
Background Processing baked in.
• Allow an application to switch job systems with minimal
code change due to common API
• Very basic queuing system built in
• Roll your own wrapper class that responds to push & pop
# application.rb
config.queue = QueueName
Rails.queue.push(Job.new)

36. Review
• You need to be monitoring your application.
• Performance has to be reviewed on a regular basis.
• Database indexes are cheap, make lots of them.
• Every application can take advantage of some level
of caching: page, action or fragment.
• Background any work that you can.
• Don't neglect front-end performance.

37. How to Install New Relic
New Relic Standard is Free at Engine Yard
1. If you’re an Engine Yard Customer, select your
plan in your Engine Yard Account Settings
2. Add newrelic_rpm to your Gemfile
3. Enable monitoring in the Engine Yard
Dashboard
Full Installation Details: http://ey.io/install-newrelic

38. Questions?

39. Chris Kelly John Yerhot
Thanks for @amateurhuman
www.newrelic.com
@yerhot
www.engineyard.com
Watching!