With constantly evolving threats across the Internet, Harvard University deployed a security network platform to mitigate cyber threats, current and future, to protect institutional and research data. By using multiple geographic locations, best of breed equipment, and network automation, Harvard provides visibility, availability, and multilayer protections for their cloud network. This talk discusses the benefits, considerations, and lessons learned from using their security network platform at the edge of the cloud. Learn how Harvard designed and deployed the platform, utilizing serverless architecture to orchestrate the solution from within to protect their most sensitive data and afford students, faculty, and staff the flexibility of cloud computing.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Leo Zhadanovsky
Principal Solutions Architect
Amazon Web Services
November 29, 2016
How Harvard University Improves Scalable Cloud Network
Security, Visibility, and Automation
SAC326
Thomas Vachon
Manager of Cloud Architecture
Harvard University

2. What to expect from the session
Learn how Harvard designed and deployed the platform,
utilizing serverless architecture to orchestrate the solution
from within to protect their most sensitive data and afford
students, faculty, and staff the flexibility of cloud computing.

3. Connecting your on-premises
networks to Amazon VPCs

4. How to connect to your VPC
• Bastion host
• Site-to-site VPN
• AWS Direct Connect
virtual private
cloud
corporate data
center

5. How to connect to your VPC
• Bastion host
• Needs Elastic IP address
• Adds extra hop
• Single point of failure
• Simple
virtual private cloudcorporate data center
Bastion HostServer

6. How to connect to your VPC
• Site-to-site VPN
• AWS: Virtual private gateway
• On-premises: Customer gateway
• IKE, IPSec v2, BGP (optional but preferred)
• Can run into bandwidth limit with on-premises VPN devices
virtual private cloud
customer
gateway
VPN
gateway
VPN
connection
corporate data center

7. How to connect to your VPC
• AWS Direct Connect
• Dedicated, fiber connection between AWS and on-premises
• Available in 1 Gbps, 10 Gbps
• Many PoPs around the world
• Public and private VIFs available
• Transit over AWS backbone for US regions
• Routing priority
Virtual private cloud
customer
gateway
VPN
gateway
Corporate data center
AWS Direct
Connect

8. Network security options

9. Controlling network access in a VPC
• Security groups
• Network ACLs
• Routing tables
• Internet gateway
• NAT gateway
• S3 private endpoint
Internet
gateway
Route table
Security
group
VPC subnet

10. Network visibility
• AWS CloudTrail
• VPC Flow Logs
• Amazon S3 bucket logs
• Elastic Load Balancing logs
• AWS Config Flow logs
AWS
CloudTrail
AWS
Config

11. IDS/IPS
• Agent-based solutions
• Available in AWS Marketplace
• Examples: Trend Micro Deep Security, Alert Logic Threat
Manager
• Costs usually scale by number of hosts
• Inline solutions
• Available in AWS Marketplace
• Examples: Cisco, Brocade, Fortinet, Palo Alto
• Single point of failure

12. IDS/IPS
• Egress through Direct Connect
• Use on-premises IDS/IPS devices
• There should be redundant Direct Connects
• Ideally, also diverse paths
• On-premises network becomes single point of failure for AWS
Internet connectivity
• Makes DNS more interesting

13. Harvard Cloud Shield

14. What is Cloud Shield?
• Network security platform
• Traffic aggregation and
inspection points
• Redundant and
geographically diverse
points of presence

15. Goals and alternatives

16. Solution overview: Design goals
• Provide highly available network access to the cloud
• Provide visibility of traffic into, out of, and between
applications
• Provide next-gen firewall protections such as IPS and
antivirus
• Provide simpler configuration through inline filtering

17. Security agents
• Easier configuration
• No additional overhead costs
• More expensive for customers
• Reactive response
Solution overview: Other options
Inline virtual firewalls
• Proactive response
• Cheaper for customers
• Very high overhead costs
• Complex VPC routing

18. Technical design overview

19. Network connectivity

20. Connectivity (2015)

21. Connectivity 2016 proposed

22. Connectivity 2016 actual

23. Network connectivity: Overview
• Four connections to AWS over Direct Connect
• Two private links between Harvard’s campus and
Virginia network point of presence
• No common spans or buildings between any links

24. Network design

30. Routing in detail

31. Routing in detail: Direct Connect
config router bfd
config neighbor
edit 10.254.1.4
set interface ”vlan10"
edit "10.254.1.4"
set advertisement-interval 1
set activate6 disable
set bfd enable
set prefix-list-in "vpc-cidr-network"
set remote-as 7224
set route-map-out "prepend-ASN"
set send-community6 disable
end

32. Routing in detail: Upstream router
template peer-policy cs-aws-peering
default-originate
advertisement-interval 0
send-community exit-peer-policy
template peer-session cs-aws-peering
timers 10 30
fall-over bfd
exit-peer-session
neighbor 10.254.1.2 remote-as 64816
neighbor 10.254.1.2 inherit peer-session cs-aws-peering
neighbor 10.254.1.2 description EBGP to atsdev1
address-family ipv4
aggregate-address 198.54.100.0 255.255.255.0 summary-only

33. Routing in detail: Key route filtering
config router prefix-list
edit “pub-nets”
set prefix 198.54.100.0 255.255.255.0
set le 32
end
edit "vpc-cidr-network”
set prefix 10.0.0.0 255.255.240.0
unset ge
unset le
end

34. Network orchestration

35. Network orchestration: Overview
• Developed a server-less architecture for a manager of
managers
• Built on Python and overlays 5 different network
management products or networking devices
• Utilize a schema-less managed NoSQL database to
pass state between different components

37. Lessons learned

38. Lessons learned: Business
• Ensure network security is
in place first
• Align with your technology
providers and vendors
• Have key business
sponsors
• Constant communication is
essential

39. Lessons learned: Network design
• Stateful failover isn't
practical
• Failing over sites
periodically is a must
• Network interoperability is a
myth

40. Lessons learned: Routing
• iBGP and eBGP function
differently
• Graceful restart is not
always ideal
• Use BFD on every network
hop
• Terminate public peering at
each network PoP

41. Lessons learned: Connectivity
• Path selection is critical and
hard
• The price of a service does
not imply quality of a
service
• Use multiple Direct Connect
endpoints

42. Lessons learned: Orchestration
• Not all APIs are created
equal (or exist)
• Network vendors are not
software engineers
• Ensure all values are
externally configurable

43. Thank you!

44. Remember to complete
your evaluations!