21. ANATOMY OF A SSH TUNNEL
U
s
e
r
http://localhost:8080
Photos: http://eandt.theiet.org/magazine/2012/07/interview-andrew-blum.cfm, http://www.quickmeme.com/meme/35583x/
22. ANATOMY OF A SSH TUNNEL
U
s
e
r
http://localhost:8080
S
e
r
v
e
r
Photos: http://eandt.theiet.org/magazine/2012/07/interview-andrew-blum.cfm, http://www.quickmeme.com/meme/35583x/
23. ANATOMY OF A SSH TUNNEL
U
s
e
r
http://localhost:8080
I
n
t
e
r
n
e
t
S
e
r
v
e
r
Photos: http://eandt.theiet.org/magazine/2012/07/interview-andrew-blum.cfm, http://www.quickmeme.com/meme/35583x/
24. ANATOMY OF A SSH TUNNEL
U
s
e
r
http://localhost:8080
I
n
Ports t
605 e
8080 r
n
e
t
Ports
S
e
80 r
22 v
e
r
Photos: http://eandt.theiet.org/magazine/2012/07/interview-andrew-blum.cfm, http://www.quickmeme.com/meme/35583x/
25. ANATOMY OF A SSH TUNNEL
U
s
e
r
http://localhost:8080
I
n
Ports t
605 e
8080 r
n
e
t
Ports
S
e
80 r
22 v
e
r
Photos: http://eandt.theiet.org/magazine/2012/07/interview-andrew-blum.cfm, http://www.quickmeme.com/meme/35583x/
26. ANATOMY OF A SSH TUNNEL
U
s
e
r
http://localhost:8080
I
n
Ports t
605 e
8080 r
n
e
t
Ports
S
e
80 r
22 v
e
r
Photos: http://eandt.theiet.org/magazine/2012/07/interview-andrew-blum.cfm, http://www.quickmeme.com/meme/35583x/
27. SSH TUNNELS HAVE BAGGAGE
YOU STILL NEED...
•A server that’s somewhere on the internet
• Running some kind of *nix
• That you control
33. NEAT! HOW DOES IT WORK?
CLIENT SERVICE
1. Makes HTTP
POST to localtunnel
34. NEAT! HOW DOES IT WORK?
CLIENT SERVICE
1. Makes HTTP
POST to localtunnel
2. Responds with an open
hostname and port
35. NEAT! HOW DOES IT WORK?
CLIENT SERVICE
1. Makes HTTP
POST to localtunnel
2. Responds with an open
hostname and port
3. Opens a tunnel
using specified port
36. NEAT! HOW DOES IT WORK?
CLIENT SERVICE
1. Makes HTTP
POST to localtunnel
2. Responds with an open
hostname and port
3. Opens a tunnel
using specified port
4. Proxies traffic from
your assigned hostname
to your tunnel
40. TROUBLE IN PARADISE
• Random hostname, changes every time.
• Idle timeout
• Change network connections?
41. TROUBLE IN PARADISE
• Random hostname, changes every time.
• Idle timeout
• Change network connections?
• Put your laptop to sleep?
42. TROUBLE IN PARADISE
• Random hostname, changes every time.
• Idle timeout
• Change network connections?
• Put your laptop to sleep?
• Stop and start your server?
43. WHAT I REALLY WANT IS...
MACBOOK.MANUZAK.COM
ALL. THE. TIME.
47. WHAT IS CLOUDTUNNEL?
CLIENT SERVICE
•Keeps the connection
alive
•Restarts the connection
if needed
•Runs localtunnel as a
daemon
48. WHAT IS CLOUDTUNNEL?
CLIENT SERVICE
•Keeps the connection •Allows you to specify your
alive hostname
•Restarts the connection •Supports CNAMES!
if needed
•Redirects users to your
•Runs localtunnel as a current tunnel address
daemon
57. FORKING AND PIDS
• PID = Process ID
• Unique ID for each running process
• Allows direct access for management
• Why Process (fork)?
• Process.detach == Thread.new { Process.wait(pid) }
58. CLIENT - TUNNEL
def process_alive?(pid)
begin
Process.getpgid(pid)
true
rescue Errno::ESRCH
false
end
end
def ping_tunnel
url = "http://
#{@host}.#{TUNNEL_SERVICE_HOST_DOMAIN}"
uri = URI(url)
response = Net::HTTP.get_response(uri)
case response
when Net::HTTPSuccess
true
else
false
end
end
end
end
59. CLIENT - TUNNEL
def process_alive?(pid)
begin
Process.getpgid(pid)
true
rescue Errno::ESRCH
false
end
end
def ping_tunnel
url = "http://
#{@host}.#{TUNNEL_SERVICE_HOST_DOMAIN}"
uri = URI(url)
response = Net::HTTP.get_response(uri)
case response
when Net::HTTPSuccess
true
else
false
end
end
end
end
60. CLIENT - TUNNEL
def process_alive?(pid)
begin
Process.getpgid(pid)
true
rescue Errno::ESRCH
false
end
end
def ping_tunnel
url = "http://
#{@host}.#{TUNNEL_SERVICE_HOST_DOMAIN}"
uri = URI(url)
response = Net::HTTP.get_response(uri)
case response
when Net::HTTPSuccess
true
else
false
end
end
end
end
61. CLIENT - DAEMON
require 'cloud_tunnel'
SLEEP_INTERVAL = 180
loop do
@tunnel ||=nil
start_tunnel_and_register_pid unless @tunnel && @tunnel.connected?
sleep(SLEEP_INTERVAL)
end
def start_tunnel_and_register_pid
# Clean up if process was running previously
kill_background_process(@pid) unless @pid.nil?
# Open the tunnel
@tunnel = CloudTunnel::start
@pid = @tunnel.pid
# Ensure the background processes get killed on exit
at_exit do
kill_background_process(@pid)
end
end
62. CLIENT - DAEMON
require 'cloud_tunnel'
SLEEP_INTERVAL = 180
loop do
@tunnel ||=nil
start_tunnel_and_register_pid unless @tunnel && @tunnel.connected?
sleep(SLEEP_INTERVAL)
end
def start_tunnel_and_register_pid
# Clean up if process was running previously
kill_background_process(@pid) unless @pid.nil?
# Open the tunnel
@tunnel = CloudTunnel::start
@pid = @tunnel.pid
# Ensure the background processes get killed on exit
at_exit do
kill_background_process(@pid)
end
end
63. CLIENT - DAEMON
require 'cloud_tunnel'
SLEEP_INTERVAL = 180
loop do
@tunnel ||=nil
start_tunnel_and_register_pid unless @tunnel && @tunnel.connected?
sleep(SLEEP_INTERVAL)
end
def start_tunnel_and_register_pid
# Clean up if process was running previously
kill_background_process(@pid) unless @pid.nil?
# Open the tunnel
@tunnel = CloudTunnel::start
@pid = @tunnel.pid
# Ensure the background processes get killed on exit
at_exit do
kill_background_process(@pid)
end
end
64. CLIENT - REGISTRATION
module CloudTunnel
class RedirectClient
def create_or_update(source, destination)
uri = URI('http://ctr.herokuapp.com/route')
req = Net::HTTP::Post.new(uri.path)
req.set_form_data(:source => source, :destination => destination)
res = Net::HTTP.start(uri.hostname, uri.port) do |http|
http.request(req)
end
case res
when Net::HTTPSuccess
true
else
raise 'Could not update redirect server'
end
end
end
end
66. SERVICE - ROUTE
require 'sinatra'
require 'dm-core’
class Route
include DataMapper::Resource
property :id, Serial
property :source, String, :unique => true
property :destination, String
property :port, Integer
end
67. SERVICE - REDIRECT
get '/' do
route = Route.last(:source => parse_hostname(request.host))
if route
redirect_path = "#{route.destination}" + (route.port.nil? ?
'' : ":#{route.port}")
redirect redirect_path, 302
else
halt 404, 'Not found.'
end
end
68. SERVICE - REDIRECT
get '/' do
route = Route.last(:source => parse_hostname(request.host))
if route
redirect_path = "#{route.destination}" + (route.port.nil? ?
'' : ":#{route.port}")
redirect redirect_path, 302
else
halt 404, 'Not found.'
end
end
69. SERVICE - REDIRECT
get '/' do
route = Route.last(:source => parse_hostname(request.host))
if route
redirect_path = "#{route.destination}" + (route.port.nil? ?
'' : ":#{route.port}")
redirect redirect_path, 302
else
halt 404, 'Not found.'
end
end
- Let’s get the hard questions out of the way first.\n- For those of you who don’t know me, my name is Jonathan Manuzak\n- I’m the lead developer at a startup called CodeGuard, where we do website backup and monitoring\n\n
\n
Like most of you, I do most of my development on a laptop\nOk, maybe it looks more like this...\n
What’s more interesting is where I could be working from. I might be at the office, my house, a coffee shop, an airport or who knows where\n\n
- The one thing all of these situations have in common is that I’m behind a firewall or NAT device\n\nLet’s take a second to talk about these firewalls or nat devices\n
Firewalls are like diodes, they really prefer for things to go in one direction. In the case of firewalls, it’s out. You can connect to websites, email, maybe FTP.\n\n- Even if you could make these changes they can be a security risk\n- It’s really never accessible to the rest of the world\n
Firewalls are like diodes, they really prefer for things to go in one direction. In the case of firewalls, it’s out. You can connect to websites, email, maybe FTP.\n\n- Even if you could make these changes they can be a security risk\n- It’s really never accessible to the rest of the world\n
Firewalls are like diodes, they really prefer for things to go in one direction. In the case of firewalls, it’s out. You can connect to websites, email, maybe FTP.\n\n- Even if you could make these changes they can be a security risk\n- It’s really never accessible to the rest of the world\n
Firewalls are like diodes, they really prefer for things to go in one direction. In the case of firewalls, it’s out. You can connect to websites, email, maybe FTP.\n\n- Even if you could make these changes they can be a security risk\n- It’s really never accessible to the rest of the world\n
- But, the rest of the world has lots of cool stuff and people!\n\n\n
- You may want to share an app you’re working on with a client instead of sending screenshots\n\n
- Then there are services, full of webhooks and callbacks\n- OAuth Authentication providers\n- Payments\n- Communications\n\nThis stuff can be painful to develop if you have to keep deploying, testing, deploying, testing\n\n\n\n\n
- Then there are services, full of webhooks and callbacks\n- OAuth Authentication providers\n- Payments\n- Communications\n\nThis stuff can be painful to develop if you have to keep deploying, testing, deploying, testing\n\n\n\n\n
- Then there are services, full of webhooks and callbacks\n- OAuth Authentication providers\n- Payments\n- Communications\n\nWith all of these big companies -- facebook, twitter, google...\n\n
- With all of those companies on the previous page who claim to support developers, this should be a non-issue, right?\n
Sure, there are a few ways around it\n
Sure, there are a few ways around it\n
- Just fire up a ssh tunnel\n- Has anyone here used SSH tunnels before?\nDid you know that SSH was released in 1995 and by the end of that year it had over 20,000 users. Not bad growth for one year, right?\n
Assume you have a local webserver running on port 8080\nIt’s sunday afternoon, you’re at home, and you want to show someone this cool app you’re working on\nMaybe it’s your Grandma\n
... and she’s at her house in a different state\nLets also assume that you have a linode or slicehost...\n
...some kind server floating around...\n
... out there on the internet\nNow, how do we get grandma to see your local webserver?\nUsing some variation of that ugly command I showed earlier, we can create a tunnel from your server to your laptop \n\n
The first part of the tunnel is an SSH connection that’s made from your laptop out, through your firewall/router to the remote server\nThen the tunnel is created which essentially binds two ports on either side of the ssh connection\n\n
Thereby sending all traffic from port 80 on the server through the tunnel to port 8080 on your local machine\n\n
Then all grandma has to do is connect to the remote server in her browser and she’ll see what you see\n\n
SSH tunnels are easy if you have all of the right ingredients\n\n- Outside of your firewall\n- For ssh\n- ...because you’re probably going to need at root or sudo access to do this\n\nSo if you have all of that, it’s just a simple one-line command\n
I’m lazy, so the appeal of running my own server and having to run this command every time I want to use an ssh tunnel is not appealing.\n\n
\n
Localhost maintains the public server for you \nIt also provides a nice client to hide the ugly tunnel creation stuff\n\n
\n
\n
\n
\n
\n
Localtunnel is awesome, but it has a few issues that make it a pain to use more than occasionally\n\n- First, the hostname is random. It changes every time you connect.\n- Idle timeout\n\n- If you change network connections, restart your server, put your laptop to sleep or do anything else to interrupt that socket connection you’re going to have to restart localtunnel.\n
Localtunnel is awesome, but it has a few issues that make it a pain to use more than occasionally\n\n- First, the hostname is random. It changes every time you connect.\n- Idle timeout\n\n- If you change network connections, restart your server, put your laptop to sleep or do anything else to interrupt that socket connection you’re going to have to restart localtunnel.\n
Localtunnel is awesome, but it has a few issues that make it a pain to use more than occasionally\n\n- First, the hostname is random. It changes every time you connect.\n- Idle timeout\n\n- If you change network connections, restart your server, put your laptop to sleep or do anything else to interrupt that socket connection you’re going to have to restart localtunnel.\n
Localtunnel is awesome, but it has a few issues that make it a pain to use more than occasionally\n\n- First, the hostname is random. It changes every time you connect.\n- Idle timeout\n\n- If you change network connections, restart your server, put your laptop to sleep or do anything else to interrupt that socket connection you’re going to have to restart localtunnel.\n
Localtunnel is awesome, but it has a few issues that make it a pain to use more than occasionally\n\n- First, the hostname is random. It changes every time you connect.\n- Idle timeout\n\n- If you change network connections, restart your server, put your laptop to sleep or do anything else to interrupt that socket connection you’re going to have to restart localtunnel.\n
\n
- There are some alternatives to localtunnel\n- Both showoff io and pagekite allow custom cnames, but as far as I can tell still suffer from the other issues.\n- So, just like many engineers, rather than spending the princely sum of $4 or $5 per month, I decided to solve the solution myself using localtunnel.\n
Quick side note. This is what you see if you search for ‘cloud tunnel’ \n\nIt’s a type of arcus cloud, called a roll cloud. They’re relatively rare, and appear to be when an incoming cold sea breeze meets the warmer opposing landmass.\n
The service maintains a record of your localtunnel address and forwards users from a hostname you specify to your current localtunnel\n- Runs on Heroku\n
The service maintains a record of your localtunnel address and forwards users from a hostname you specify to your current localtunnel\n- Runs on Heroku\n
Lets take a closer look at what was going on during the demo\nIf we look at the processlist, we’d notice 3 cloud_tunnel processes\nThe monitor is created by the deamon to watch over, start, and stop the primary cloud_tunnel process\n- If you noticed during the first demo, the localtunnel client blocked until it was killed\n
Lets take a closer look at what was going on during the demo\nIf we look at the processlist, we’d notice 3 cloud_tunnel processes\nThe monitor is created by the deamon to watch over, start, and stop the primary cloud_tunnel process...\n\n
This so called primary tunnel process handles the creation and maintenance of the localtunnel client connection\n- If you noticed during the first demo, the localtunnel client blocked until it was killed\n- For compatibility, I wanted to re-use the localtunnel client instead of reimplementing it myself\nThis adds a bit of complexity, but with a new version of localtunnel on the horizon it will make forward compatibility easier\n\n
Finally, this is the localtunnel client which was forked from the primary cloud_tunnel process\n\n
This is the wrapper class that handles the LocalTunnel client. The meat of it is here...\n
After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
Process allows better management\nBut we’re kind of using threads. Thread waits on \n
This is the wrapper class that handles the LocalTunnel client. The meat of it is here...\n
This is the wrapper class that handles the LocalTunnel client. The meat of it is here...\n
This is the wrapper class that handles the LocalTunnel client. The meat of it is here...\n
This is the wrapper class that handles the LocalTunnel client. The meat of it is here...\n
Now that we detached that process, we have to keep an eye on it.\nThis is a script that starts and maintains that tunnel which and is invoked from the daemon\n
Now that we detached that process, we have to keep an eye on it.\nThis is a script that starts and maintains that tunnel which and is invoked from the daemon\n
Now that we detached that process, we have to keep an eye on it.\nThis is a script that starts and maintains that tunnel which and is invoked from the daemon\n
\n
\n
\n
\n
\n
\n
\n
\n
You can find everything I talked about here today on Github. I’d encourage you to give it a try and let me know what you think. Pull requests are welcome.\nQuestions?\n
