1. Evolu&on
of
peering
and
Internet
Exchanges
in
Japan
[IDNOG
1]
Seiichi
Kawamura
BIGLOBE
Inc.
as2518.peeringdb.com
copyright
(c)
2014
BIGLOBE
Inc.
1

2. copyright
(c)
2014
BIGLOBE
Inc.
2
Congratula&ons
on
the
first
mee&ng!
from

3. In
2012
One
ques&on
was
asked
• This
was
a
long
forgoUen
ques&on.
(
in
our
community)
• It
used
to
be
easy.
– You
peered,
because
you
needed
routes.
• Now,
the
Internet
is
a
given.
– You
can
buy
full
routes
easily...
• A
stagnant
market…
– IX
ports
cost
20,000usd/month
– The
top
ISPs
never
peer
• So
why
peer?
copyright
(c)
2014
BIGLOBE
Inc.
3
What
does
peering
do
for
you?

4. copyright
(c)
2014
BIGLOBE
Inc.
4
photo
by
Megartworks
hUp://www.flickr.com/photos/mgeartworks/10183970615/
used
under
crea&ve
commons
license
2.0
hUps://crea&vecommons.org/licenses/by/2.0/

5. 2014
what
is
happening
now
(in
JP)
• An
evolving
ecosystem
– Rise
of
peering
communi&es
– Cloud,
Content,
mobile
driven
• Price
destruc&on
– 2
years
ago
10G
was
20,000usd
-­‐>
has
become
close
to
interna&onal
pricing
level
• More
technical
experiments
copyright
(c)
2014
BIGLOBE
Inc.
5
Finally
star&ng
to
catch
up
with
the
reali&es
of
the
Internet

6. but
first
a
bit
of
history
copyright
(c)
2014
BIGLOBE
Inc.
6

7. The
history
of
Internet
in
Japan
• hUps://www.nic.ad.jp/&meline/en/
copyright
(c)
2014
BIGLOBE
Inc.
7

8. copyright
(c)
2014
BIGLOBE
Inc.
8
BIGLOBE
March
1994
The
first
IX
in
Japan!
1997
JANOG

9. • Peering
between
licensed
telecoms
required
gov
permission
• No
domes&c
peering
– All
traffic
send
to
US
copyright
(c)
2014
BIGLOBE
Inc.
9
1)
Before
the
IX
2)
NSPIXP1
• Ini&ally
layer3
IX
based
on
peering
with
AS2500
(WIDE)
– AS2500
was
NOT
considered
a
telecom
and
could
peer
with
anyone
• Ini&al
traffic
about
192kbps
• Repeaters,
Cisco2501,
and
BGP3!!!
– IIJ,
Infoweb,
SPIN,
WIDE
3)
on
to
NSPIXP2
shim
to
a
Layer2
IX
• Relocate
to
a
real
DC
• Change
to
switching
hubs
warning!
history
more
than
20
years
ago

10. Internet
eXchanges
in
Japan
now
copyright
(c)
2014
BIGLOBE
Inc.
10
Major
city
exchange
Global
exchange
Equinix
Regional
exchange
Akita
IX,
Okinawa
IXand
others…
Inter-­‐region
(MPLS-­‐IX)
dis&x
(dissolved)

11. Distribu&on
throughout
the
country
copyright
(c)
2014
BIGLOBE
Inc.
11
Tokyo:
-­‐
JPNAP/JPNAP2
-­‐
BBIX
-­‐
JPIX
-­‐
Equinix
-­‐
dix-­‐ie
(nspixp2)
Osaka:
-­‐
JPNAP
-­‐
BBIX
-­‐
JPIX
-­‐
NSPIXP3
Niigata:
-­‐
Echigo-­‐IX
Akita:
-­‐
Akita-­‐IX
Nagoya:
-­‐
BBIX
-­‐
JPIX
Fukuoka:
-­‐
BBIX
Cable
Landing
Sta&ons
Okinawa:
-­‐
OIX

12. Traffic
in
Japan
copyright
(c)
2014
BIGLOBE
Inc.
12
Source:
hUp://www.soumu.go.jp/main_content/000244628.pdf
Download
traffic
at
2,275Gbps
(28%
from
overseas)

13. Major
Internet
Exchange
Architectures
copyright
(c)
2014
BIGLOBE
Inc.
13
op&cal
switch
L2
Switch
L2
Switch
L2
Switch
L2
Switch
redundant
rings
op&cal
switch
Fully
Redundant
MPLS
connected
Distributed
MPLS
IX
Core
other
IX
ISP
MPLS
Router
ISP
MPLS
Router
LSP
LSP
ISP
Router
ISP
Router

14. Changes
coming
from
Mobile
and
Cloud
area
• More
mobile
and
cloud
traffic
– Broadband
at
1.3x
growth
while
mobile
at
1.8x
growth
• Content
providers
have
different
requirements
compared
to
ISPs
– semi-­‐full
route
– simple
route
servers
– latency
aware
– mtu9000
– fast
detec&on
using
BFD
– DoS
protec&on
as
a
service
copyright
(c)
2014
BIGLOBE
Inc.
14

15. Evolu&on
of
peering
community
• Basically
we
had
none
– PAST:
People
disliked
talking
about
peering
at
JANOG
• Peering
relied
heavily
on
ISPs
copyright
(c)
2014
BIGLOBE
Inc.
15
• BoFs,
study
councils,
talks
at
NOGs,
and
joint
efforts
by
the
IXPs
and
the
community
– more
focus
on
content/dc
and
its
requirements

16. Star&ng
an
AS
1. get
an
AS
number
2. get
an
engineer
that
has
experience
with
BGP
3. buy
transit
from
an
upstream
ISP
4. connect
to
an
IX
5. peer
with
other
AS
6. maintain
the
peer
copyright
(c)
2014
BIGLOBE
Inc.
16
The
underlined
steps
have
always
been
the
problem,
and
this
was
because
we
did
not
have
a
community
to
help
out
new
comers

17. community
ac&vi&es
• Google
Groups
:
Peering
in
Japan
– local
Japanese
language
only
– discussion
on
latest
peering
issues
– no
IX
personnel
on
list
– host
Peering
BoFs
• Tutorials
– IX
companies
provide
low
cost
tutorials
(available
regularly)
– free
tutorials
at
JANOG
(not
always
available)
• CloudIX
Study
Council
– Group
of
members(BGP
operators)
in
BBIX
doing
technical
experiments
– sharing
skills
and
helping
out
each
other
to
peer
copyright
(c)
2014
BIGLOBE
Inc.
17

18. community
effort
example
:
PeeringDB
• What
is
PeeringDB?
– hUp://www.peeringdb.com/
– A
common
place
to
publish
informa&on
about
your
AS
(AS
number,
exchanges
and
facili&es
you
are
at,
your
peering
policy,
your
contact
info,
etc.)
– DO
NOT
use
it
to
collect
contacts
for
sales
or
spam
you
will
be
penalized
• This
REALLY
REALLY
helps
you
to
peer
and
maintain
your
network
• Good
presenta&on
here
– hUps://fileshare.tools.isoc.org/mwangi/public/AXIS/Technical
%20Aspects%20Workshop/Day-­‐5/Session-­‐3/PeeringDB-­‐and-­‐Role-­‐of-­‐
Peering-­‐Coordinator.pdf
copyright
(c)
2014
BIGLOBE
Inc.
18

19. copyright
(c)
2014
BIGLOBE
Inc.
19
4
years
ago
nobody
had
a
PeeringDB
record
in
Japan
community
efforts
to
promote
and
help
each
other
create
records
Now
everyone
that
I
peer
with
has
a
record
and
we
are
much
much
happier!

20. JANOG
where
we
all
come
together
copyright
(c)
2014
BIGLOBE
Inc.
20
• Founded
1997
• 6400
mailing
list
members
• Meetings
– Held twice a year, usually January and July.
– 2 days of plenary sessions
• mostly focus on technical and operational discussions
– Run by 12 Committee members + Meeting volunteers
– Presentations and discussions are in Japanese
• English acceptable but rough translation is recommended
• around 10 non-native attendees
per meeting
– ~800 attendees at Tokyo
– ~400 attendees at other Cities

21. Plenary
BoF
NOC
Sake

22. Recent
peering
trends
copyright
(c)
2014
BIGLOBE
Inc.
22

23. Emerging
Internet
Exchange
Architectures
copyright
(c)
2014
BIGLOBE
Inc.
23
SDN?
Simple
DIY?
OF
Switch
OF
Switch
ISP
Router
OF
Switch
OF
controller
ISP
Router
virtual
Router
L2
Switch
ISP
Router
ISP
Router
ISP
Router
facility
hosted
by
ISP:
at
your
own
risk
IX
WIDE
Project

24. PIX-­‐IE
(SDN
IX)
• What
it
aims
to
accomplish
– Using
OpenFlow
and
Cumulus
Linux
– Fine
grain
path
control
across
an
AS
boundary
– DDoS
protec&on
func&on
– Value
added
services
with
Func&on
Virtualiza&on
– Experiment
on
what
cannot
be
done
with
the
legacy
L2/L3
IX
technology
• Interested?
contact
[sekiya
at
wide.ad.jp]
copyright
(c)
2014
BIGLOBE
Inc.
24

25. Recent
topics
being
discussed
in
the
community
• decentraliza&on
from
Tokyo
• Full
service
IX
and
LCC
IX
• u&lizing
SDN
and
API
to
interconnect
clouds
• how
to
reach
MTU
9000
• more
open
peering
discussions
• which
DC
is
best
for
peering
• BCOP
(best
current
opera&onal
prac&ce)
on
peering
document
– hUp://www.internetsociety.org/deploy360/projects/
bcop/
copyright
(c)
2014
BIGLOBE
Inc.
25

26. Summary
:
The
lessons
that
we
learned
in
the
past
few
years
• An
ac&ve
community
is
key
driver
for
an
ac&ve
peering
landscape
• We’re
not
a
“customer”
of
an
IX,
but
rather
are
members
and
need
to
contribute
to
the
community
• There’s
s&ll
gaps
between
ISPs
and
content,
but
it’s
easier
than
before
to
talk
about
a
solu&on,
not
just
seUle
with
payment
• Being
more
open,
invi&ng
new
people,
will
keep
the
community
moving
and
it
will
lead
to
beUer
opera&on
prac&ces.
copyright
(c)
2014
BIGLOBE
Inc.
26

27. Acknowledgements
• Prof.
Akira
KATO
and
Prof.
Yuji
SEKIYA
from
WIDE
Project
• Shingo
KUDO
from
Sombank
• Tom
PASEKA
from
Cloudflare
copyright
(c)
2014
BIGLOBE
Inc.
27
The
following
persons
helped
review
the
content.
Thank
you!!!