Presentation by Sho Fujimura at APRICOT 2019 on Thursday, 28 February 2019.

1. Information Technology Center, Fukuoka University, Japan
Sho FUJIMURA
fujimura@fukuoka-u.ac.jp
NIPPON TELEGRAPH AND TELEPHONE WEST CORPORATION
Fuminori -Tany- Tanizaki
fuminori.tanizaki@west.ntt.co.jp
FUKUOKA UNIVERSITY PUBLIC
NTP SERVICE & BCP38

2. 2
Introducing Fukuoka University
Objectives
2 Fukuoka University NTP Service
2 Traffic Volumes and Causes
3 NTP SERVICE & BCP38
3 Packet Analysis and Observations
4 Conclusions
4 Reference Materials
Today’s Content
1
1

3. Fukuoka University introduction
n Private university
¡ 86th anniversary in May 2019
¡ Connected to internet in 1993
n Location: Fukuoka City, JAPAN
¡ The city we had APRICOT2015
n 9 faculties
(31 departments)
n 10 graduate courses
(33 specialties)
n Approximately 20,000
students
n Attached facilities
¡ Hospital: 3
¡ High school: 2
¡ Junior high school: 1
3
AS: 18148
Prefix: 133.100.0.0/16, 2405:be00::/32

4. Today’s Presentation(Objective)
n Proceeding with BCP38
(Best Current Practice 38)measures
4

5. Fukuoka University NTP Service
and Network Architecture
n Commenced
Operations Oct 1993
n Japan’s 1st open NTP
Server
¡ 133.100.9.2
¡ 133.100.11.8
n NTP Server load
distributed to 4 servers
n Multihomed internet
connection to OCN and
SINET
5
AS18148 … Fukuoka University
AS2907… Science Information NETwork SINET operated by National Institute of Informatics
AS4713 … Open Computer Network OCN operated by NTT Communications Corporation
Campus Network
BGP
router
BGP
router
FireWall FireWall
Router
(L3 switch)
Router
(L3 switch)
Each
building
L2 switch
Each
building
L2 switch
Edge
switch
Edge
switch
Edge
switch
AS18148
L2 switch
for NTP
L2 switch
for NTP
NTP Servers NTP Servers

6. What do these figures mean!?
270Mb/sec
350,000p/sec
6

7. 7
all traffic (bit/sec)
all traffic (packet/sec)
• Graph showing router traffic
and packet numbers
• Low night traffic at University at
night
• Therefore it can be deduced
that there is a high proportion
of NTP request packets

8. If this is so...
n “High traffic volumes are a problem.
So why not just shut down the NTP
Server?”
n “Because if we shut down the NTP
server the number of request packets
increase!”
8

9. Outline of Experiment
n To confirm that request packets increase when
the server disposes of NTP request packets
n Time of experiment 2018/07/21 - 2018/07/22
n Subject A specific AS (prefix no. 1361)
n Method
n Direct NTP Server prefixes to blackhole
n Deactivate all server blackhole settings
9

10. The Experimental Result
n Straight after enabling the black hold, request packets (green)
gradually began to increase
n The increase contiunued for 6 hours, then levelled off
n After disabling the black hole, the traffic immediately decreased.
n The range was over 160Mb/s 10

11. While investigating various
issues in preparation for
decommissioning the NTP
Server
We discovered another
troublesome issue!! 11

12. Request packets sent from 1.1.1.1
n On closer inspection, the request packets were
sent from 1.1.1.0/24 and 1.0.0.0/24
n Currently we are filtering them at the NTP
Server 12

13. What is 1.1.1.1?
n It is a public DNS Resolution Service operated
by Cloudflare
n Currently 1.0.0.0/24 and 1.1.1.0/24 are being
advertised as AS13335(Cloudflare)
13
https://1.1.1.1/ or https://one.one.one.one/

14. Where is it coming from?
n (Of course)it is not coming from
Cloudflare
14

15. Packet Analysis
n We collected and analyzed NTP request packets
n Collection period 2018/11/30 8:26 - 2018/12/6
0:00
n Packets collected 1,408,390
n Traffic volumes approx.2.8pps
15
12 . 04 23 .

16. From what address?
n 1.0.0.0/24
16
12.22%
10.57%
10.42%
10.40%
9.66%
5.99%
4.93%
4.65%
3.56%
2.70%

17. From what address?
n 1.1.1.0/24
17
19.73%
7.69%
4.90%
2.82%
2.63%
2.63%
2.56%
2.53%
2.52%
2.51%

18. What source port no.?
18
13 . 04 23 .
3 12
n Access from 2168 ports

19. Sample of NTP packets sent
19
source port is not
from inside 123 NAT
The time from when it was plugged
in was 7hr 53 min?

20. Sample of NTP packets sent
n It appears that one request is sent every 10
seconds until time synchronization is reached
¡ Synchronization not possible as IPv4 is incorrect
source port 1030 packet
source port 1025 packet

21. Presumed connection structure
and packet flow
21
ISP
Router etc.
Intranet
(With NAT)
IPv4
1.1.1.0/24
Router
NTP Server
Fukuoka U
Network A
cloudflare
1.1.1.1

22. What are these packets?
n 1.1.1.1 is used in
Captive Portal
in public Wi-Fi,
hotel routers,
University wireless
LAN etc.
¡ The setup by the
administrator of
hotel and cafe free
Wi-Fi forces
mandatory web
access
22
https://www.k-bit.de/wireless_lan/kb_easy-hotspot-userguide.pdf

23. Should a filter be created? (BCP38)
23
ISP
Customer side
router
Packets other than IP
source addresses
allocated to network
own network are
disposed
Packets other than IP
source addresses
allocated to network
customers are
disposed
In this case (1.1.1.1)
it is extremely difficult to
filter

24. The future of Fukuoka-U NTP Service
n We plan to collect all of these NTP
Server directed packets, including BGP
routed packets sent to the NTP Server,
collect them in a designated router and
null them
n We plan to analyze the dispose packets
with netflow/sflow
24

25. Proposed new network architecture
25
AS2907 AS4713
Fukuoka University/AS18148 (133.100.0.0/16)
Campus Network
NTP Server
#1,#2
NTP Server
#3,#4
AS18148
133.100.9.2/24
133.100.11.0/24
NTP BGP Router
BGP Router #1 BGP Router #2
SINET Fukuoka DC

26. Conclusion
n We should establish a filter based on
BCP38
¡ Let's not send out disguised packets and
private address block packets
26

27. References
n BCP38
¡ http://www.bcp38.info/
¡ https://tools.ietf.org/html/bcp38
n Fukuoka University Public NTP Service
Deployment Use case (APRICOT 2017)
¡ https://2017.apricot.net/program/schedule/
#/day/8/apops-1
27

28. Thank you for your kind attention