NBCUniversal, a worldwide mass media corporation, was looking for a more affordable and easier way to manage their database solution that hosts their extensive online digital assets. With Datavail’s assistance, NBCUniversal, made the move from MongoDB 3.6 to MongoDB Atlas on AWS.

1. Why NBC Universal Migrated to MongoDB Atlas
Charleste King,
MongoDB Practice Lead, Datavail
Trinh Nguyen,
Sr. Director of Technology, NBCUniversal

2. MongoDB Practice Lead, Datavail
Charleste King
§ Over 20 years in the IT industry
§ Expertise in software development to data
analysis, architecture, and administration
§ Industry experience includes aerospace,
agriculture, medicine, education, and more.
§ Specializes in developing solutions for unique
client problems ranging from multi-level
upgrades to forecasting problem areas to
tuning and performance monitoring
§ Has a Master of Science from Georgia Institute
of Technology

3. Datavail Overview
Databases, Analytics, and
Application Development
We are data specialists
15+ years delivering
data services
Projects, Managed
Services and staffing
500+ customers
7 years average retention
§ All major platforms including
MongoDB and MongoDB Atlas
§ AWS, Azure and Google Cloud
§ Comprehensive development &
operational services
§ US & Global delivery models
§ Health checks & performance
tuning
§ Upgrades
Database Operations

4. Sr. Director of Technology,
NBCUniversal
Trinh Nguyen
§ 12 years with E!’s Interactive Technology team
§ Oversees three teams: Software engineers, Web
Developers, & DevOps
§ Expertise in Lambda, EC2, ECS, Redis, GraphQL, Chef
to build Spring-based applications as well as React
apps with NodeJS.
§ Works primarily in Java/Javascript, using MongoDB
and MySQL for data storage
§ Migrated E! to the cloud completely in March 2017 -
first Cable brand at NBCU to move its entire
infrastructure to the cloud
§ Has a dog named Professor, a son named Maverick,
and a wife who let him come to this conference
§ Bachelor’s in Computer Engineering from the
University of California, Irvine

5. www.nbcuniversal.com
E! is the only global, multi-platform brand
for all things pop culture. The network is
currently available to 91 million cable and
satellite subscribers in the U.S and 161
countries globally.
E! is a network of NBCUniversal Entertainment & Lifestyle
Group, a division of NBCUniversal, one of the world's
leading media and entertainment companies
NBC Universal
E! Entertainment

6. Agenda
• NBC Universal – Business drivers & technical needs
• Top pain points driving move to MongoDB Atlas
• Technical needs to be overcome before migration
• Migration
• Post migration observations and benefits
• Lessons learned
• Q&A

7. The Journey to Atlas

8. NBCUniversal Business Drivers For Atlas Migration
§ Ability to build digital products and features quickly
§ Highly available and secure database
§ Streamline operational costs – Licensing and Infrastructure
§ Self Service provisioning and management
§ Intuitive metrics and visualizations

9. Technical Needs for E! Digital to Migrate to Atlas
Cost and Ease of Management
§ Ongoing/Repeated tasks:
§ Daily health checks
§ Storage issues – constantly checking storage
§ Manual deletion of snapshots
§ Upgrading/Patching
§ Performance tuning

10. E! Digital – Prior Architecture

11. NBCUniversal
Top Pain Points Driving Move to MongoDB Atlas

12. Top Pain Points Driving Move to MongoDB Atlas
Prior Challenge
• Provisioning Instances
• Access to Ops Manager
• Downtime / Patching
• Storage / Data Retention
• Alerts Lagged in Getting to
Right Individuals and Teams
Benefits of Atlas Migration
• Self Service
• Access Atlas Portal from Anywhere
• Auto patching and one click
upgrades w/o downtime
• Storage Auto Expand and
Automated Backups
• Streamlined and Targeted Alerts

13. NBCUniversal
Technical Challenges
Engineering Prerequisites

14. In our network we have a proxy appliance, which all
connections to the internet must route through.
Normally this is fine with web browsing, however when
trying to connect from our desktops to mongo atlas
instances connection is denied. Is there a way to setup
connectivity to include the proxy settings?
Arbi Begian,
DevOps Lead

15. First Challenge
Developers were not able to
connect from our corporate office to
Mongo Atlas since our corporate
network egress is restricted to go
out through a proxy firewall device,
which blocks all ports except for
HTTP and HTTPS.

16. Connectivity to Atlas is established using the
MongoDB wire protocol, which is not HTTP-based.
Unless the proxy appliance that you mention has
explicit support for the wire protocol or port
forwarding, then it is unlikely that the appliance will
function in the manner you intend.
Mongo Support

17. How E! Online Overcame Connectivity Restriction
To overcome connectivity limitation from our corporate network to hosted Mongo Atlas we
had to reconfigure our network and routing setup:
§ Instead of using corporate internet egress to connect to Atlas we decided to route through our
AWS VPC.
§ By VPC peering the two networks, E! was able to establish communication to MongoDB clusters
within the network .
§ To abide with our AWS environment security we had to peer according to each of our specific
VPCs (Dev/Shared/Staging/Prod).
§ We reconfigured our internal routing to pass the MongoDB communication internally rather than
going through the public interface.

18. /etc/hosts
#Using Dev or Shared Open VPN
172.22.64.XXX dev-pub-shard-00-XX-abcde.mongodb.net
172.22.64.XX dev-pub-shard-00-XX-abcde.mongodb.net
172.22.64.XX dev-pub-shard-00-XX-abcde.mongodb.net
#MongoDB Atlas Properties
### Dev env Atlas ###
atlasConnectionUrl=mongodb://webclient:USER@dev-pub-shard-00-XX-
abcde.mongodb.net:27017,
dev-pub-shard-00-XX-abcde.mongodb.net:27017,
dev-pub-shard-00-XX-
abcde.mongodb.net:27017/{database}?authSource=admin&retryWrites=true
&ssl=true&replicaSet=dev-pub-shard-0
atlasBudConnectionUrl=mongodb://
atlasEnable=true
Developer Setup

19. VPC Peering Developers’ communication issues
were solved since we did not have to
go through the proxy device.
By going this route it also made our
connection to the Atlas cluster much
faster, even faster than accessing our
EC2 instances that were running
MongoDB Enterprise.
Unexpected Performance
Gains

20. Setting up Your Subnets
§ When we setup our subnets we tried to keep the address spaces similar
§ Dev-Test subnet before: 172.21.68.0
§ Atlas Dev-Test Cluster after: 172.22.68.0
§ During Cluster Setup we did not use default subnet, this has to be explicitly defined if you
want to use your own
§ Easy/Helps know your database IPs when scanning, setting up firewall rules, inspecting
logs, etc.

21. Route 53
§ We ran into problems configuring SRV within
Route53
§ SRV service call in Atlas already has logic to
figure out which is node is primary
§ Keeping the DNS in sync wasn’t worth it just for
cleaner/shorter names when new nodes were
brought up for maintenance or under load
(cluster nodes increasing/decreasing)
§ In our case we weren’t able to leverage Route53
to abstract the cluster’s configuration
parameters to be stored in DNS but many of you
will be able to leverage SRV and TXT records as
a solution
Solution and Learnings
§ Long Mongo hostnames
§ Setting up Simple to read CNAMES
§ Namespace “mongodb.net”
Challenge

22. Updating Application Drivers
§ Almost all of our applications have a centralized back-end service called “commons client
interface” (CCI).
§ Previously CCI was configured with Spring Data MongoDB Core » 2.0.7.RELEASE, packaged
with mongodb-driver-async v3.5.0.
§ Upgraded to 3.6.1 and leveraged the retryWrites=true parameter.
§ Cutover timing.

23. Migration

24. Why we used
Mongomirror § Luckily, MongoDB provides
mongomirror which works in this
situation
Solution
If you have VPC Peering with
MongoDB 3.6, Live Migration
requires extra network configuration
that we wanted to avoid.
Challenge

25. How We Leveraged mongomirror to Help with the
Migration
§ Helps to perform live migration without downtime.
§ mongomirror is a utility for migrating data from an existing MongoDB replica
set to a MongoDB Atlas replica set
§ Does not require to shut down existing replica sets or applications.
§ Drawback: it does not import the user/role data, thus we had do manually
create the users in MongoDB Atlas after the migration of the data.

26. 7 Steps to Use Mongomirror
1. Download mongomirror on secondary node
2. Configuration changes and prerequisites in place for mongomirror to work
3. Setup MongoDB User in the AWS and run getParameter command
4. Setup MongoDB User in the target MongoDB Atlas Cluster with Atlas admin role.
5. Add IP(s) to whitelist
6. Mongomirror sync and tailing of OpLog
7. Switch application and stop mongomirror

27. use admin
db.createUser(
{
user: "AtlasMigrationUser",
pwd: "passwordxxxxx",
roles: [ "clusterMonitor", "readAnyDatabase" ]
}
)
We used a command similar to the following:

28. mongomirror --host "rs-pub-
dev/mongo.dev.eol.inbcu.com:27017" --username
"datavail_user" --password "passwordxxxxx" --
authenticationDatabase "admin" --destination "dev-
pub-shard-0/dev-pub-shard-00-00-
uwwl2.mongodb.net:27017,dev-pub-shard-00-01-
uwwl2.mongodb.net:27017,dev-pub-shard-00-02-
uwwl2.mongodb.net:27017" --destinationUsername
"admin" --destinationPassword "passwordxxxxxxxx”
Step 6
6. Started mongomirror process. We executed using a statement that looks like the following:

29. Show sample screen shot of progress bar

30. Post Migration
Observation and
Benefits

31. Benefits of Mongo
Hosted Cloud
Eliminated: EC2 instances
reserved to run OpsManager
and its replica set.
Additional Gains

32. Benefits of Mongo
Hosted Cloud
Productivity: no double VPN
needed to access Atlas since
it is publicly accessible.
Additional Gains

33. Benefits of Mongo
Hosted Cloud
Secure: 2FA required to
access Atlas.
Additional Gains

34. Benefits of Mongo
Hosted Cloud
Self Service: Developers can
spin up clusters for
project/prototype-based
microservice work.
Additional Gains

35. Benefits of Mongo
Hosted Cloud
Roles & Permissions are
Transparent and don’t
necessarily need a DBA for
setup.
Additional Gains

36. Benefits of Mongo
Hosted Cloud
Maintenance patching
requires no downtime and
easy one-click.
Additional Gains

37. Lessons Learned
§ If you’re part of a large corporation
that uses a proxy… be ready for:
§ Know about the port restrictions
§ Or have a plan on who can accommodate
your testing/migration and how
§ If you are running in the cloud, VPC
peering helps, but test routing
§ Allocate time in your migration plan to
update your applications’ drivers
§ Talk openly about your gains to the
organization:
§ Rapid Prototyping
§ Decreased Downtime
§ Accessibility
§ Alerting
§ Storage/Backup
§ Autopatching

38. Q&A