9. Load Balancer
Reverse Proxy App Server
MySQL Proxy
Proxy as Middleware
middleware ftw!
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
10. 90% use case
%w*Transparent Intercepting Caching …+
There are many different types!
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
11. Transparent
HAProxy
App server A App server B
Transparent, Cut-Through Proxy
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
13. Proxy Proxy
App server A App server B App server C
Problem: Staging Environment
Production
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
14. Simulating traffic?
Proxy
Duplication
App server C
“Representative Load / Staging”
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
19. EventMachine: Speed + Convenience
building high performance network apps in Ruby
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
20. p quot;Startingquot;
while true do
EM.run do
timers
p quot;Running in EM reactorquot;
network_io
end
other_io
end
puts quot;Almost donequot;
EventMachine Reactor
concurrency without threads
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
21. p quot;Startingquot;
while true do
EM.run do
timers
p quot;Running in EM reactorquot;
network_io
end
other_io
end
puts quot;Almost donequot;
EventMachine Reactor
concurrency without threads
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
22. C++ core
Easy concurrency
without threading
EventMachine Reactor
concurrency without threads
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
23. http = EM::HttpRequest.new('http://site.com/').get
http.callback {
p http.response
}
# ... do other work, until callback fires.
Event = IO event + block or lambda call
EventMachine Reactor
concurrency without threads
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
24. http = EM::HttpRequest.new('http://site.com/').get
http.callback {
p http.response
}
# ... do other work, until callback fires.
Event = IO event + block or lambda call
EventMachine Reactor
concurrency without threads
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
25. EM.run do
EM.add_timer(1) { p quot;1 second laterquot; }
EM.add_periodic_timer(5) { p quot;every 5 secondsquot;}
EM.defer { long_running_task() }
end
class Server < EM::Connection
def receive_data(data)
send_data(quot;Pong; #{data}quot;)
end
def unbind
p [:connection_completed]
end
end
EM.run do
EM.start_server quot;0.0.0.0quot;, 3000, Server
end
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
26. EM.run do
EM.add_timer(1) { p quot;1 second laterquot; }
EM.add_periodic_timer(5) { p quot;every 5 secondsquot;}
EM.defer { long_running_task() }
end
class Server < EM::Connection
def receive_data(data)
send_data(quot;Pong; #{data}quot;) Connection Handler
end
def unbind
p [:connection_completed]
end
end
EM.run do Start Reactor
EM.start_server quot;0.0.0.0quot;, 3000, Server
end
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
27. http://bit.ly/aiderss-eventmachine
by Dan Sinclair (Twitter: @dj2sincl)
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
28. Proxies for Monitoring, Performance and Scale
welcome to the wonderful world of…
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
29. Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn|
conn.server :name, :host => quot;127.0.0.1quot;, :port => 81
conn.on_data do |data|
# ...
end
Relay Server
conn.on_response do |server, resp|
# ...
end
conn.on_finish do
# ...
end
end
EM-Proxy
www.github.com/igrigorik/em-proxy
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
30. Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn|
conn.server :name, :host => quot;127.0.0.1quot;, :port => 81
conn.on_data do |data|
# ...
end
Process incoming data
conn.on_response do |server, resp|
# ...
end
conn.on_finish do
# ...
end
end
EM-Proxy
www.github.com/igrigorik/em-proxy
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
31. Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn|
conn.server :name, :host => quot;127.0.0.1quot;, :port => 81
conn.on_data do |data|
# ...
end
Process response data
conn.on_response do |server, resp|
# ...
end
conn.on_finish do
# ...
end
end
EM-Proxy
www.github.com/igrigorik/em-proxy
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
32. Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn|
conn.server :name, :host => quot;127.0.0.1quot;, :port => 81
conn.on_data do |data|
# ...
end
conn.on_response do |server, resp|
# ...
end
Post-processing step
conn.on_finish do
# ...
end
end
EM-Proxy
www.github.com/igrigorik/em-proxy
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
33. %w[ <Transparent> Intercepting Caching … +
solution for every problem
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
34. Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn|
conn.server :srv, :host => quot;127.0.0.1quot;, :port => 81
# modify / process request stream
conn.on_data do |data|
p [:on_data, data]
data
end No data modifications
# modify / process response stream
conn.on_response do |server, resp|
p [:on_response, server, resp]
resp
end
end
Port-Forwarding
transparent proxy
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
35. Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn|
conn.server :srv, :host => quot;127.0.0.1quot;, :port => 81
conn.on_data do |data|
data
end
conn.on_response do |backend, resp| Alter response
resp.gsub(/hello/, 'good bye')
end
end
Port-Forwarding + Alter
transparent proxy
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
36. %w[ Transparent <Intercepting> Caching … +
solution for every problem
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
37. Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn|
@start = Time.now
@data = Hash.new(quot;quot;)
Prod + Test
conn.server :prod, :host => quot;127.0.0.1quot;, :port => 81
conn.server :test, :host => quot;127.0.0.1quot;, :port => 82
conn.on_data do |data|
data.gsub(/User-Agent: .*?rn/, 'User-Agent: em-proxyrn')
end
conn.on_response do |server, resp|
@data[server] += resp
resp if server == :prod
end
conn.on_finish do
p [:on_finish, Time.now - @start]
p @data
end
end
Duplex HTTP: Benchmarking
Intercepting proxy
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
38. Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn|
@start = Time.now
@data = Hash.new(quot;quot;)
conn.server :prod, :host => quot;127.0.0.1quot;, :port => 81
conn.server :test, :host => quot;127.0.0.1quot;, :port => 82
conn.on_data do |data|
data.gsub(/User-Agent: .*?rn/, 'User-Agent: em-proxyrn')
end
conn.on_response do |server, resp|
Respond from production
@data[server] += resp
resp if server == :prod
end
conn.on_finish do
p [:on_finish, Time.now - @start]
p @data
end
end
Duplex HTTP: Benchmarking
Intercepting proxy
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
39. Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn|
@start = Time.now
@data = Hash.new(quot;quot;)
conn.server :prod, :host => quot;127.0.0.1quot;, :port => 81
conn.server :test, :host => quot;127.0.0.1quot;, :port => 82
conn.on_data do |data|
data.gsub(/User-Agent: .*?rn/, 'User-Agent: em-proxyrn')
end
conn.on_response do |server, resp|
@data[server] += resp
resp if server == :prod
end
Run post-processing
conn.on_finish do
p [:on_finish, Time.now - @start]
p @data
end
end
Duplex HTTP: Benchmarking
Intercepting proxy
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
44. Hacking SMTP
for fun and profit
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
45. Proxy.start(:host => quot;0.0.0.0quot;, :port => 2524) do |conn|
conn.server :srv, :host => quot;127.0.0.1quot;, :port => 2525
# RCPT TO:<name@address.com>rn
RCPT_CMD = /RCPT TO:<(.*)?>rn/
Intercept Addressee
conn.on_data do |data|
if rcpt = data.match(RCPT_CMD)
if rcpt[1] != quot;ilya@igvita.comquot;
conn.send_data quot;550 No such user herenquot;
data = nil
end
end
data
end
conn.on_response do |backend, resp|
resp
end
Defeating SMTP Wildcards
end
Intercepting proxy
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
46. Proxy.start(:host => quot;0.0.0.0quot;, :port => 2524) do |conn|
conn.server :srv, :host => quot;127.0.0.1quot;, :port => 2525
# RCPT TO:<name@address.com>rn
RCPT_CMD = /RCPT TO:<(.*)?>rn/
conn.on_data do |data|
Allow: ilya@igvita.com
if rcpt = data.match(RCPT_CMD)
if rcpt[1] != quot;ilya@igvita.comquot;
conn.send_data quot;550 No such user herenquot;
data = nil 550 Error otherwise
end
end
data
end
conn.on_response do |backend, resp|
resp
end
Defeating SMTP Wildcards
end
Intercepting proxy
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
50. Proxy.start(:host => quot;0.0.0.0quot;, :port => 2524) do |conn|
conn.server :srv, :host => quot;127.0.0.1quot;, :port => 2525
RCPT_CMD = /RCPT TO:<(.*)?>rn/
FROM_CMD = /MAIL FROM:<(.*)?>rn/
MSG_CMD = /354 Start your message/
MSGEND_CMD = /^.rn/ Intercept commands
conn.on_data do |data|
#…
end
conn.on_response do |server, resp|
p [:resp, resp]
if resp.match(MSG_CMD)
@buffer = true
@msg = quot;quot;
end
resp
SMTP + SPAM Filtering
end
end
building a state-machine
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
51. Proxy.start(:host => quot;0.0.0.0quot;, :port => 2524) do |conn|
conn.server :srv, :host => quot;127.0.0.1quot;, :port => 2525
RCPT_CMD = /RCPT TO:<(.*)?>rn/
FROM_CMD = /MAIL FROM:<(.*)?>rn/
MSG_CMD = /354 Start your message/
MSGEND_CMD = /^.rn/
conn.on_data do |data|
#…
end
conn.on_response do |server, resp|
p [:resp, resp]
if resp.match(MSG_CMD) Flag & Buffer message
@buffer = true
@msg = quot;quot;
end
resp
SMTP + SPAM Filtering
end
end
building a state-machine
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
52. Save data
conn.on_data do |data|
@from = data.match(FROM_CMD)[1] if data.match(FROM_CMD)
@rcpt = data.match(RCPT_CMD)[1] if data.match(RCPT_CMD)
@done = true if data.match(MSGEND_CMD)
if @buffer
@msg += data
Buffer
data = nil
end
if @done
#…
end
data
SMTP + SPAM Filtering
end
building a state-machine
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
53. conn.on_data do |data|
@from = data.match(FROM_CMD)[1] if data.match(FROM_CMD)
@rcpt = data.match(RCPT_CMD)[1] if data.match(RCPT_CMD)
@done = true if data.match(MSGEND_CMD)
if @buffer
@msg += data
Flag end of message
data = nil
end
if @done
#… Process message
end
data
SMTP + SPAM Filtering
end
building a state-machine
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
58. @PostRank: Beanstalkd + Ruby Proxy
because RAM is still expensive
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
59. ~ 93 Bytes of overhead per job
~300 Bytes of data / job
x 80,000,000 jobs in memory
~ 30 GB of RAM = 2 X-Large EC2 instances
Oi, expensive!
Beanstalkd Math
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
60. Observations:
1. Each job is rescheduled several times
2. > 95% are scheduled for > 3 hours into the future
Memory is wasted…
3. Beanstalkd does not have overflow page-to-disk
We’ll add it ourselves!
Extending Beanstalkd
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
62. Proxy.start(:host => quot;0.0.0.0quot;, :port => 11300) do |conn|
conn.server :srv, :host => quot;127.0.0.1quot;, :port => 11301
PUT_CMD = /put (d+) (d+) (d+) (d+)rn/
conn.on_data do |data|
if put = data.match(PUT_CMD) Intercept PUT command
if put[2].to_i > 600
p [:put, :archive]
# INSERT INTO ....
conn.send_data quot;INSERTED 9999rnquot;
data = nil
end
end
data
end
conn.on_response do |backend, resp|
resp
end
end
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
63. Proxy.start(:host => quot;0.0.0.0quot;, :port => 11300) do |conn|
conn.server :srv, :host => quot;127.0.0.1quot;, :port => 11301
PUT_CMD = /put (d+) (d+) (d+) (d+)rn/
conn.on_data do |data|
if put = data.match(PUT_CMD)
if put[2].to_i > 600 If over 10 minutes…
p [:put, :archive]
# INSERT INTO ....
conn.send_data quot;INSERTED 9999rnquot;
data = nil
end
end Archive & Reply
data
end
conn.on_response do |backend, resp|
resp
end
end
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf
64. Overload the protocol
MySQL
EM-Proxy
PUT
RESERVE, PUT, …
put job, 900
Beanstalkd
@PostRank: “Chronos Scheduler”
Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf