CloudStackユーザ会　advent calendar 2012年12月12日 フォントが綺麗にならない。文字をきちんと見たい方はPDF版をどうぞ。

1. CloudStack
仮想ルータの謎に迫る
KVM+NFS環境
⽇日本CloudStackユーザ会
@MayumiK0
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
1

2. さぁ受け取るといい。それが君の運命だ。
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
2

3. CloudStack構成例例
・典型的な構成例例
-‐‑‒Management Server
-‐‑‒NFS Server （Primary/Secondary領領域）
-‐‑‒Compute Node
Compute Compute
Management
NFS
Node
Node
Server
ここは仮想サーバでも可
Primary
(node04)
(node05)
Storage
Secondary
Storage
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
3

4. 仮想ルータの謎に迫る
・仮想ルータにログインしてみる
仮想ルータとCompute NodeはLink Local Networkで通信可能
仮想ルータが起動しているCompute Nodeにログインし
そこから仮想ルータのリンクローカルアドレスにsshする
Compute Compute
Management
NFS
Node
Node
Server
(node04)
インスタンス
(node05
仮想ルータ
Primary
Storage
Secondary
Storage
Link
Local
Network
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
4

5. 仮想ルータの謎に迫る
・LinkLocal確認
間違ってはいないね

6. 仮想ルータの謎に迫る
・ssh鍵認証でログイン
[root@node006
~]#
ssh
-­‐i
.ssh/id_rsa.cloud
169.254.3.116
-­‐p
3922
Linux
r-­‐5-­‐VM
2.6.32-­‐5-­‐686-­‐bigmem
#1
SMP
Mon
Jan
16
16:42:05
UTC
2012
i686
The
programs
included
with
the
Debian
GNU/Linux
system
are
ate
;
up]me
root@r-­‐5-­‐VM:~#
d free
so[ware;
the
exact
distribu]on
terms
for
each
program
are
described
in
the
TC
2012
Mon
Dec
10
15:54:59
U
individual
files
in
/usr/share/doc/*/copyright.
15:54:59
up
1
day,
1:01,
1
user,
load
average:
0.00,
0.00,
0.00
root@r-­‐5-­‐VM:~#
date;
ifconfig
-­‐a
Debian
GNU/Linux
comes
with
ABSOLUTELY
NO
WARRANTY,
to
the
e2012
Mon
Dec
10
15:55:08
UTC
xtent
permihed
by
applicable
law.
eth0
Link
encap:Ethernet
HWaddr
02:00:6b:3d:00:02
inet
addr:10.1.1.1
Bcast:10.1.1.255
Mask:255.255.255.0
Last
login:
Sun
Dec
9
14:20:04
2012
from
169.254.0.1
P
BROADCAST
RUNNING
MULTICAST
MTU:1500
Metric:1
U
Linux
r-­‐5-­‐VM
2.6.32-­‐5-­‐686-­‐bigmem
#1
SMP
Mon
J
an
1X
p16:42:05
Uerrors:0
dropped:0
overruns:0
frame:0
R 6
ackets:11592
TC
2012
i686
TX
packets:8741
errors:0
dropped:0
overruns:0
carrier:0
collisions:0
txqueuelen:1000
The
programs
included
with
the
Debian
GNU/Linux
s
ystem
are
free
so[ware;
bytes:2582211
(2.4
MiB)
RX
bytes:972709
(949.9
KiB)
TX
the
exact
distribu]on
terms
for
each
program
are
described
in
the
individual
files
in
/usr/share/doc/*/copyright.
eth1
Link
encap:Ethernet
HWaddr
0e:00:a9:fe:03:74
inet
addr:169.254.3.116
Bcast:169.254.255.255
Mask:255.255.0.0
UP
BROADCAST
RUNNING
MULTICAST
MTU:1500
Metric:1
Debian
GNU/Linux
comes
with
ABSOLUTELY
NO
W
ARRANTY,
to
the
rrors:0
dropped:0
overruns:0
frame:0
RX
packets:12285
e extent
permihed
by
applicable
law.
TX
packets:10166
errors:0
dropped:0
overruns:0
carrier:0
collisions:0
txqueuelen:1000
root@r-­‐5-­‐VM:~#
RX
bytes:1937229
(1.8
MiB)
TX
bytes:1915520
(1.8
MiB)
6

7. 仮想ルータの謎に迫る
・実は再起動するとLinkLocalが変わる
root@node006
~]#
ssh
-­‐i
.ssh/id_rsa.cloud
169.254.3.221
-­‐p
3922
Last
login:
Mon
Dec
10
16:00:04
2012
from
169.254.0.1
Linux
r-­‐5-­‐VM
2.6.32-­‐5-­‐686-­‐bigmem
#1
SMP
Mon
Jan
16
16:42:05
UTC
2012
i686
／）(
◕
‿‿
◕
)（＼
root@r-­‐5-­‐VM:~#
date;
up]me
Mon
Dec
10
16:18:29
UTC
2012
知らなければ知らないままで
16:18:29
up
1
min,
1
user,
load
average:
0.00,
0.00,
0.00
何の不都合もないからね
root@r-­‐5-­‐VM:~#
date
;ifconfig
-­‐a
Mon
Dec
10
16:18:34
UTC
2012
でいいのか？
eth0
Link
encap:Ethernet
HWaddr
02:00:6b:3d:00:02
inet
addr:10.1.1.1
Bcast:10.1.1.255
Mask:255.255.255.0
UP
BROADCAST
RUNNING
MULTICAST
MTU:1500
Metric:1
RX
packets:12
errors:0
dropped:0
overruns:0
frame:0
TX
packets:0
errors:0
dropped:0
overruns:0
carrier:0
collisions:0
txqueuelen:1000
RX
bytes:844
(844.0
B)
TX
bytes:0
(0.0
B)
eth1
Link
encap:Ethernet
HWaddr
0e:00:a9:fe:03:dd
inet
addr:169.254.3.221
Bcast:169.254.255.255
Mask:255.255.0.0
UP
BROADCAST
RUNNING
MULTICAST
MTU:1500
Metric:1
RX
packets:3373
errors:0
dropped:0
overruns:0
frame:0
TX
packets:3244
errors:0
dropped:0
overruns:0
carrier:0
collisions:0
txqueuelen:1000
RX
bytes:629043
(614.2
KiB)
TX
bytes:607306
(593.0
KiB)

8. 仮想ルータの謎に迫る
・テスト構成 Public IP : 202.228.225.32
Compute Compute
Management
NFS
Node
Node
Server
(node04)
(node05
Primary
Storage
インスタンス
仮想ルータ
test01：10.1.1.207
r-­‐5-­‐VM
Secondary
Storage
インスタンス
test02：10.1.1.131
仮想ルータが裏で
どんなコト（処理）を
しているか覗いてみましょう
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
8

9. 仮想ルータの謎に迫る
・起動時に⾏行行なっている処理理
Dec 10 16:16:46 r-‐‑‒5-‐‑‒VM cloud: Starting dnsmasq
Dec 10 16:16:46 r-‐‑‒5-‐‑‒VM cloud: Starting cloud-‐‑‒passwd-‐‑‒srvr
Dec 10 16:16:46 r-‐‑‒5-‐‑‒VM cloud: Starting ssh
仮想インスタンスが２台あり
Dec 10 16:16:46 r-‐‑‒5-‐‑‒VM cloud: Starting haproxy Firewallや負荷分散設定は
Dec 10 16:16:46 r-‐‑‒5-‐‑‒VM cloud: Starting apache2
Dec 10 16:16:46 r-‐‑‒5-‐‑‒VM cloud: Stopping cloud
何もされていない状態での起動
Dec 10 16:16:46 r-‐‑‒5-‐‑‒VM cloud: Stopping nfs-‐‑‒common
Dec 10 16:16:46 r-‐‑‒5-‐‑‒VM cloud: Stopping portmap
Dec 10 16:16:48 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh:Adding first ip 202.228.225.32/26 on interface eth2
Dec 10 16:16:48 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh:Added SourceNAT 202.228.225.32/26 on interface eth2
Dec 10 16:16:48 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh:Added first ip 202.228.225.32/26 on interface eth2
Dec 10 16:16:50 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh:Add routing 202.228.225.32/26 on interface eth2
Dec 10 16:16:51 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh:Add routing 202.228.225.32/26 rules added
Dec 10 16:16:51 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh: created VPN chain for 202.228.225.32
Dec 10 16:16:51 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh: created firewall chain for 202.228.225.32
Dec 10 16:16:51 r-‐‑‒5-‐‑‒VM cloud: edithosts: update 02:00:3e:53:00:01 10.1.1.207 test01 to hosts
Dec 10 16:16:51 r-‐‑‒5-‐‑‒VM cloud: /root/edithosts.sh: setting default router for 10.1.1.207 to 10.1.1.1
Dec 10 16:16:51 r-‐‑‒5-‐‑‒VM cloud: /root/edithosts.sh: setting dns server for 10.1.1.207 to 10.1.1.1
Dec 10 16:16:53 r-‐‑‒5-‐‑‒VM cloud: edithosts: update 02:00:79:6c:00:03 10.1.1.131 test02 to hosts
Dec 10 16:16:53 r-‐‑‒5-‐‑‒VM cloud: /root/edithosts.sh: setting default router for 10.1.1.131 to 10.1.1.1
Dec 10 16:16:53 r-‐‑‒5-‐‑‒VM cloud: /root/edithosts.sh: setting dns server for 10.1.1.131 to 10.1.1.1
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
9

10. 仮想ルータの謎に迫る
・dnsmasq:
DNSサーバのフォワーダとDHCPサーバをもつソフト
root@r-‐‑‒5-‐‑‒VM:~∼# ps afxwwww | grep dnsmasq
2079 ? S 0:00 /usr/sbin/dnsmasq -‐‑‒x /var/run/dnsmasq/dnsmasq.pid -‐‑‒u dnsmasq -‐‑‒7 /etc/dnsmasq.d,.dpkg-‐‑‒
dist,.dpkg-‐‑‒old,.dpkg-‐‑‒new
Dec 10 16:16:55 dnsmasq[2079]: started, version 2.55 cachesize 150
Dec 10 16:16:55 dnsmasq[2079]: compile time options: IPv6 GNU-‐‑‒getopt DBus I18N DHCP TFTP
Dec 10 16:16:55 dnsmasq-‐‑‒dhcp[2079]: DHCP, static leases only on 10.1.1.1, lease time 1h
Dec 10 16:16:55 dnsmasq[2079]: using local addresses only for domain cs2cloud.internal
意外な展開ではないよ
Dec 10 16:16:55 dnsmasq[2079]: reading /etc/dnsmasq-‐‑‒resolv.conf
Dec 10 16:16:55 dnsmasq[2079]: using nameserver 8.8.8.8#53
Dec 10 16:16:55 dnsmasq[2079]: using local addresses only for domain cs2cloud.internal
Dec 10 16:16:55 dnsmasq[2079]: read /etc/hosts -‐‑‒ 15 addresses
Dec 10 16:16:55 dnsmasq-‐‑‒dhcp[2079]: read /etc/dhcphosts.txt
Dec 10 16:16:55 dnsmasq-‐‑‒dhcp[2079]: read /etc/dhcpopts.txt
root@r-‐‑‒5-‐‑‒VM:/etc# cat /etc/dhcpopts.txt
10_̲1_̲1_̲207,3,10.1.1.1
10_̲1_̲1_̲207,6,10.1.1.1
10_̲1_̲1_̲131,3,10.1.1.1
10_̲1_̲1_̲131,6,10.1.1.1
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
10

11. 仮想ルータの謎に迫る
・haproxy:
L7ロードバランサ
root@r-‐‑‒5-‐‑‒VM:~∼# ps afxwwww | grep haproxy
1501 ? Ss 0:00 /usr/sbin/haproxy -‐‑‒f /etc/haproxy/haproxy.cfg -‐‑‒D -‐‑‒p /var/run/haproxy.pid
root@r-‐‑‒5-‐‑‒VM:~∼# cat /etc/haproxy/haproxy.cfg
global
願い事（設定）を決めるんだ
log 127.0.0.1:3914 local0 warning 早く！
maxconn 4096
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon
defaults
log global
mode tcp
option dontlognull
（中略略）
listen vmops 0.0.0.0:9
option transparent
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
11

12. 仮想ルータの謎に迫る
・仮想ルータで実⾏行行されているsh
root@r-5-VM:~#
pwd
/root
■firewall_rule.shの一部
ゴリゴリ
root@r-5-VM:~#
ls
-rwxr-xr-x
1
root
root
824
Oct
24
05:25
bumpup_priority.sh
root@r-5-VM:~#
cat
firewall_rule.sh
#!/usr/bin/env
bash
iptableに
-rwxr-xr-x
1
root
root
1462
Oct
24
05:25
clearUsageRules.sh
-rwxr-xr-x
1
root
root
3545
Oct
24
05:25
edithosts.sh
書いてる模様
fw_chain_for_ip
()
{
-rwxr-xr-x
1
root
root
6332
Oct
24
05:25
firewall_rule.sh
local
pubIp=$1
fw_remove_backup
$1
-rwxr-xr-x
1
root
root
12404
Oct
24
05:25
firewall.sh
sudo
iptables
-t
mangle
-E
FIREWALL_$pubIp
_FIREWALL_$pubIp
2>
/dev/
-rwxr-xr-x
1
root
root
2429
Oct
24
05:25
func.sh
null
-rw-r--r--
1
root
root
13600
Feb
6
2012
ipassoc.sh
sudo
iptables
-t
mangle
-N
FIREWALL_$pubIp
2>
/dev/null
#
drop
if
no
rules
match
(this
will
be
the
last
rule
in
the
chain)
-rwxr-xr-x
1
root
root
8239
Oct
24
05:25
loadbalancer.sh
sudo
iptables
-t
mangle
-A
FIREWALL_$pubIp
-j
DROP>
/dev/null
-rw-r--r--
1
root
root
3464
Feb
6
2012
netusage.sh
#
ensure
outgoing
connections
are
maintained
(first
rule
in
chain)
sudo
iptables
-t
mangle
-I
FIREWALL_$pubIp
-m
state
--state
-rwxr-xr-x
1
root
root
1667
Oct
24
05:25
reconfigLB.sh
RELATED,ESTABLISHED
-j
ACCEPT>
/dev/null
drwxr-xr-x
2
root
root
4096
Nov
25
09:28
redundant_router
#ensure
that
this
table
is
after
VPN
chain
sudo
iptables
-t
mangle
-I
PREROUTING
2
-d
$pubIp
-j
FIREWALL_$pubIp
-rwxr-xr-x
1
root
root
1441
Oct
24
05:25
savepassword.sh
success=$?
-rwxr-xr-x
1
root
root
2497
Oct
24
05:25
userdata.py
if
[
$success
-gt
0
]
-rwxr-xr-x
1
root
root
3235
Oct
24
05:25
userdata.sh
then
#
if
VPN
chain
is
not
present
for
various
reasons,
try
to
add
in
to
the
first
slot
*/
sudo
iptables
-t
mangle
-I
PREROUTING
-d
$pubIp
-j
FIREWALL_$pubIp
fi
}
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
12

13. 仮想ルータの謎に迫る
・新規インスタンス作成
root@r-‐‑‒5-‐‑‒VM:/var/log# cat dnsmasq.log
Dec 11 17:11:09 dnsmasq[8541]: started, version 2.55 cachesize 150
Dec 11 17:11:09 dnsmasq[8541]: compile time options: IPv6 GNU-‐‑‒
getopt DBus I18N DHCP TFTP
Dec 11 17:11:09 dnsmasq-‐‑‒dhcp[8541]: DHCP, static leases only on
10.1.1.1, lease time 1h
Dec 11 17:11:09 dnsmasq[8541]: using local addresses only for
domain cs2cloud.internal
Dec 11 17:11:09 dnsmasq[8541]: reading /etc/dnsmasq-‐‑‒resolv.conf
Dec 11 17:11:09 dnsmasq[8541]: using nameserver 8.8.8.8#53
Dec 11 17:11:09 dnsmasq[8541]: using local addresses only for
domain cs2cloud.internal
Dec 11 17:11:09 dnsmasq[8541]: read /etc/hosts -‐‑‒ 16 addresses
Dec 11 17:11:09 dnsmasq-‐‑‒dhcp[8541]: read /etc/dhcphosts.txt
Dec 11 17:11:09 dnsmasq-‐‑‒dhcp[8541]: read /etc/dhcpopts.txt
Dec 11 17:12:04 dnsmasq-‐‑‒dhcp[8541]: DHCPDISCOVER(eth0)
10.0.2.15 02:00:62:c8:00:04
dnsmasqが
Dec 11 17:12:04 dnsmasq-‐‑‒dhcp[8541]: DHCPOFFER(eth0) 10.1.1.100
02:00:62:c8:00:04
インスタンスにIPを
Dec 11 17:12:04 dnsmasq-‐‑‒dhcp[8541]: DHCPREQUEST(eth0)
払い出す
10.1.1.100 02:00:62:c8:00:04
Dec 11 17:12:04 dnsmasq-‐‑‒dhcp[8541]: DHCPACK(eth0)
10.1.1.100 02:00:62:c8:00:04 test03
13

14. 仮想ルータの謎に迫る
・Firewall設定
■/var/log/messages
設定スクリプト
ipassoc.sh Dec 11 17:18:54 r-‐‑‒5-‐‑‒VM cloud: FirewallRule public interfaces = eth2
firewall.sh Dec 11 17:18:54 r-‐‑‒5-‐‑‒VM cloud: firewall_̲rule.sh: enter apply firewall rules for
firewall_rule.sh public ip 202.228.225.32:tcp:10001:10003:0.0.0.0/0
Dec 11 17:18:54 r-‐‑‒5-‐‑‒VM cloud: firewall_̲rule.sh: exit apply firewall rules for public
ip 202.228.225.32
Dec 11 17:18:54 r-‐‑‒5-‐‑‒VM cloud: firewall_̲rule.sh: successful in applying fw rules for
Firewall設定
ip 202.228.225.32
Dec 11 17:18:54 r-‐‑‒5-‐‑‒VM cloud: firewall_̲rule.sh: deleting backup for ip:
202.228.225.32
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
14

15. 仮想ルータの謎に迫る
・ポートフォワーディング設定
■/var/log/messages
Dec 11 17:29:30 r-‐‑‒5-‐‑‒VM cloud: firewall.sh: creating port fwd entry for PAT: public
設定スクリプト ip=202.228.225.32 instance ip=10.1.1.207 proto=tcp port=10001:10001
ipassoc.sh dport=22-‐‑‒22 op=-‐‑‒A
firewall.sh Dec 11 17:29:30 r-‐‑‒5-‐‑‒VM cloud: firewall.sh: creating port fwd entry for PAT: public
ip=202.228.225.32 instance ip=10.1.1.207 proto=tcp port=10001:10001
dport=22-‐‑‒22 op=-‐‑‒D
Dec 11 17:29:30 r-‐‑‒5-‐‑‒VM cloud: firewall.sh: create HairPin entry : public
ip=202.228.225.32 instance ip=10.1.1.207 proto=tcp portRange=22-‐‑‒22 op=-‐‑‒D
ポートフォワーディ Dec 11 17:29:30 r-‐‑‒5-‐‑‒VM cloud: firewall.sh: done port fwd entry for PAT: public
ip=202.228.225.32 op=-‐‑‒D result=1
ング設定
Dec 11 17:29:30 r-‐‑‒5-‐‑‒VM cloud: firewall.sh: create HairPin entry : public
ip=202.228.225.32 instance ip=10.1.1.207 proto=tcp portRange=22-‐‑‒22 op=-‐‑‒A
Dec 11 17:29:30 r-‐‑‒5-‐‑‒VM cloud: firewall.sh: done port fwd entry for PAT: public
Copyright (C) 2012 Japan CloudStack User Group All Rights
ip=202.228.225.32 op=-‐‑‒A result=0
Reserved.
15

16. 仮想ルータの謎に迫る
・負荷分散設定 ■/var/log/messages
Dec 11 17:37:22 r-‐‑‒5-‐‑‒VM cloud: Loadbalancer public interfaces = eth2
Dec 11 17:37:24 r-‐‑‒5-‐‑‒VM cloud: New haproxy instance successfully
loaded, stopping previous one.
Dec 11 17:37:25 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh:Adding first ip
202.228.225.32/26 on interface eth2
Dec 11 17:37:25 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh:Added SourceNAT
202.228.225.32/26 on interface eth2
Dec 11 17:37:25 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh:Added first ip
202.228.225.32/26 on interface eth2
Dec 11 17:37:27 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh:Add routing 202.228.225.32/26
on interface eth2
Dec 11 17:37:27 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh: VPN chain for 202.228.225.32
already exists
root@r-‐‑‒5-‐‑‒VM:/var/log# cat /etc/haproxy/haproxy.cfg
Dec 11 17:37:27 r-‐‑‒5-‐‑‒VM cloud: ipassoc.sh: firewall chain for
global
202.228.225.32 already exists
log 127.0.0.1:3914 local0 warning
（中略略）
listen 202_̲228_̲225_̲32-‐‑‒80 202.228.225.32:80
balance roundrobin
server 202_̲228_̲225_̲32-‐‑‒80_̲0 10.1.1.207:80 check
haproxy.cfgに設定
server 202_̲228_̲225_̲32-‐‑‒80_̲1 10.1.1.131:80 check
mode http
option httpclose
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
16

17. 仮想ルータの謎に迫る
・負荷分散設定
root@r-‐‑‒5-‐‑‒VM:/var/log# cat haproxy.log
Dec 10 14:44:02 localhost haproxy[1486]: Pausing proxy cloud-‐‑‒default.
Dec 10 14:44:04 localhost haproxy[8711]: Server 202_̲228_̲225_̲32-‐‑‒80/202_̲228_̲225_̲32-‐‑‒80_̲0 is DOWN, reason:
Layer4 connection problem, info: "No route to host", check duration: 3ms.
Dec 10 14:44:04 localhost haproxy[8711]: proxy 202_̲228_̲225_̲32-‐‑‒80 has no server available!
Dec 10 14:44:19 localhost haproxy[8712]: Pausing proxy stats_̲on_̲public.
Dec 10 14:44:19 localhost haproxy[8712]: Pausing proxy 202_̲228_̲225_̲32-‐‑‒80.
Dec 10 14:44:21 localhost haproxy[9064]: Server 202_̲228_̲225_̲32-‐‑‒80/202_̲228_̲225_̲32-‐‑‒80_̲0 is DOWN, reason:
Layer4 connection problem, info: "No route to host", check duration: 0ms.
Dec 10 14:44:22 localhost haproxy[9065]: Server 202_̲228_̲225_̲32-‐‑‒80/202_̲228_̲225_̲32-‐‑‒80_̲1 is DOWN, reason:
Layer4 connection problem, info: "No route to host", check duration: 5ms.
Dec 10 14:44:22 localhost haproxy[9065]: proxy 202_̲228_̲225_̲32-‐‑‒80 has no server available!
Dec 10 15:58:10 localhost haproxy[1527]: Server 202_̲228_̲225_̲32-‐‑‒80/202_̲228_̲225_̲32-‐‑‒80_̲1 is DOWN, reason:
Layer4 connection problem, info: "No route to host", check duration: 5ms.
Dec 10 15:58:10 localhost haproxy[1527]: proxy 202_̲228_̲225_̲32-‐‑‒80 has no server available!
Dec 10 15:58:16 localhost haproxy[1527]: Pausing proxy stats_̲on_̲public.
Dec 10 15:58:16 localhost haproxy[1527]: Pausing proxy 202_̲228_̲225_̲32-‐‑‒80.
Dec 10 15:58:18 localhost haproxy[2432]: Server 202_̲228_̲225_̲32-‐‑‒80/202_̲228_̲225_̲32-‐‑‒80_̲0 is DOWN, reason:
ヘルスチェックの
Layer4 connection problem, info: "No route to host", check duration: 0ms.
Dec 10 15:58:19 localhost haproxy[2433]: Server 202_̲228_̲225_̲32-‐‑‒80/202_̲228_̲225_̲32-‐‑‒80_̲1 is DOWN, reason:
ログも出る
Layer4 connection problem, info: "No route to host", check duration: 0ms.
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
17

18. 仮想ルータの謎に迫る
・iptables
root@r-5-VM:/etc/init.d#
/etc/init.d/iptables-persistent
status
Filter
Rules:
--------------
Chain
INPUT
(policy
DROP
2503
packets,
101K
bytes)
pkts
bytes
target
prot
opt
in
out
source
destination
64324
6276K
NETWORK_STATS
all
--
any
any
anywhere
anywhere
0
0
ACCEPT
all
--
any
any
anywhere
vrrp.mcast.net
0
0
ACCEPT
all
--
any
any
anywhere
225.0.0.50
37401
3291K
ACCEPT
all
--
eth0
any
anywhere
anywhere
state
RELATED,ESTABLISHED
14833
2394K
ACCEPT
all
--
eth1
any
anywhere
anywhere
state
RELATED,ESTABLISHED
390
34943
ACCEPT
all
--
eth2
any
anywhere
anywhere
state
RELATED,ESTABLISHED
453
38052
ACCEPT
icmp
--
any
any
anywhere
anywhere
13
1401
ACCEPT
all
--
lo
any
anywhere
anywhere
2
656
ACCEPT
udp
--
eth0
any
anywhere
anywhere
udp
dpt:bootps
1961
133K
ACCEPT
udp
--
eth0
any
anywhere
anywhere
udp
dpt:domain
719
43140
ACCEPT
tcp
--
eth1
any
anywhere
anywhere
state
NEW
tcp
dpt:3922
0
0
ACCEPT
tcp
--
eth0
any
anywhere
anywhere
state
NEW
tcp
dpt:http-alt
0
0
ACCEPT
tcp
--
eth0
any
anywhere
anywhere
state
NEW
tcp
dpt:www
0
0
load_balancer_eth0
tcp
--
eth0
any
anywhere
anywhere
0
0
load_balancer_eth2
tcp
--
eth2
any
anywhere
anywhere
0
0
lb_stats
tcp
--
any
any
anywhere
anywhere
18

19. 仮想ルータの謎に迫る
・iptables
Chain
FORWARD
(policy
DROP
0
packets,
0
bytes)
pkts
bytes
target
prot
opt
in
out
source
destination
10587
7297K
NETWORK_STATS
all
--
any
any
anywhere
anywhere
0
0
ACCEPT
all
--
eth0
eth1
anywhere
anywhere
state
RELATED,ESTABLISHED
0
0
ACCEPT
all
--
eth0
eth0
anywhere
anywhere
state
NEW
0
0
ACCEPT
all
--
eth0
eth0
anywhere
anywhere
state
RELATED,ESTABLISHED
528
106K
ACCEPT
tcp
--
any
any
anywhere
test01
state
RELATED,ESTABLISHED
/*
202.228.225.32:10001:10001
*/
0
0
ACCEPT
tcp
--
any
any
anywhere
test01
tcp
dpt:ssh
state
NEW
/*
202.228.225.32:10001:10001
*/
2195
4043K
ACCEPT
all
--
eth2
eth0
anywhere
anywhere
state
RELATED,ESTABLISHED
2062
142K
ACCEPT
all
--
eth0
eth2
anywhere
anywhere
Chain
OUTPUT
(policy
ACCEPT
41154
packets,
2856K
bytes)
pkts
bytes
target
prot
opt
in
out
source
destination
54494
5162K
NETWORK_STATS
all
--
any
any
anywhere
anywhere
Chain
NETWORK_STATS
(3
references)
pkts
bytes
target
prot
opt
in
out
source
destination
4863
349K
all
--
eth0
eth2
anywhere
anywhere
5724
6948K
all
--
eth2
eth0
anywhere
anywhere
0
0
tcp
--
!eth0
eth2
anywhere
anywhere
0
0
tcp
--
eth2
!eth0
anywhere
anywhere
19

20. 仮想ルータの謎に迫る
・iptables
Chain
lb_stats
(1
references)
pkts
bytes
target
prot
opt
in
out
source
destination
0
0
ACCEPT
tcp
--
any
any
anywhere
202.228.225.32
state
NEW
tcp
dpt:tproxy
Chain
load_balancer_eth0
(1
references)
pkts
bytes
target
prot
opt
in
out
source
destination
0
0
ACCEPT
tcp
--
any
any
anywhere
202.228.225.32
tcp
dpt:www
Chain
load_balancer_eth2
(1
references)
pkts
bytes
target
prot
opt
in
out
source
destination
0
0
ACCEPT
tcp
--
any
any
anywhere
202.228.225.32
tcp
dpt:www
NAT
Rules:
-------------
Chain
PREROUTING
(policy
ACCEPT
41247
packets,
1685K
bytes)
pkts
bytes
target
prot
opt
in
out
source
destination
0
0
DNAT
tcp
--
eth2
any
anywhere
202.228.225.32
tcp
dpt:10001
to:10.1.1.207:22
0
0
DNAT
tcp
--
eth0
any
anywhere
202.228.225.32
tcp
dpt:10001
to:10.1.1.207:22
Chain
POSTROUTING
(policy
ACCEPT
37392
packets,
2244K
bytes)
pkts
bytes
target
prot
opt
in
out
source
destination
0
0
SNAT
tcp
--
any
eth0
10.1.1.0/24
test01
tcp
dpt:10001
to:10.1.1.1
581
35575
SNAT
all
--
any
eth2
anywhere
anywhere
to:202.228.225.32
Chain
OUTPUT
(policy
ACCEPT
37543
packets,
2253K
bytes)
pkts
bytes
target
prot
opt
in
out
source
destination
0
0
DNAT
tcp
--
any
any
anywhere
202.228.225.32
tcp
dpt:10001
to:10.1.1.207:22
20

21. 仮想ルータの謎に迫る
Mangle
Rules:
----------------
Chain
PREROUTING
(policy
ACCEPT
84426
packets,
5631K
bytes)
pkts
bytes
target
prot
opt
in
out
source
destination
6411
7002K
VPN_202.228.225.32
all
--
any
any
anywhere
202.228.225.32
81
4769
FIREWALL_202.228.225.32
all
--
any
any
anywhere
202.228.225.32
55712
5951K
CONNMARK
all
--
any
any
anywhere
anywhere
state
RELATED,ESTABLISHED
CONNMARK
restore
0
0
MARK
tcp
--
eth2
any
anywhere
202.228.225.32
tcp
dpt:10001
MARK
set
0x2
0
0
CONNMARK
tcp
--
eth2
any
anywhere
202.228.225.32
tcp
dpt:10001
state
NEW
CONNMARK
save
Chain
INPUT
(policy
ACCEPT
44607
packets,
3987K
bytes)
pkts
bytes
target
prot
opt
in
out
source
destination
Chain
FORWARD
(policy
ACCEPT
4785
packets,
4291K
bytes)
pkts
bytes
target
prot
opt
in
out
source
destination
Chain
OUTPUT
(policy
ACCEPT
41524
packets,
2927K
bytes)
pkts
bytes
target
prot
opt
in
out
source
destination
Chain
POSTROUTING
(policy
ACCEPT
46309
packets,
7218K
bytes)
pkts
bytes
target
prot
opt
in
out
source
destination
2
670
CHECKSUM
udp
--
any
any
anywhere
anywhere
udp
dpt:bootpc
CHECKSUM
fill
Chain
FIREWALL_202.228.225.32
(1
references)
pkts
bytes
target
prot
opt
in
out
source
destination
0
0
ACCEPT
all
--
any
any
anywhere
anywhere
state
RELATED,ESTABLISHED
0
0
RETURN
tcp
--
any
any
anywhere
anywhere
tcp
dpts:10001:10003
81
4769
DROP
all
--
any
any
anywhere
anywhere
Chain
VPN_202.228.225.32
(1
references)
pkts
bytes
target
prot
opt
in
out
source
destination
6123
6984K
ACCEPT
all
--
any
any
anywhere
anywhere
state
RELATED,ESTABLISHED
288
18062
RETURN
all
--
any
any
anywhere
anywhere
21

22. 仮想ルータの謎に迫る
わけがわからないよ
仮想ルータの謎に
⽣生々しく迫る予定でしたが
諸般の事情により
仮想ルータ内で実⾏行行されている処理理の
ほんのサワリだけでした
ごめんなさい
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
22

23. 仮想ルータの謎に迫る
ありがとうございました
See You Next Time !
Some Time Some Where
Copyright (C) 2012 Japan CloudStack User Group All Rights
Reserved.
23