Speaker: Gaurav Goyal, Sr IT Architect, Cisco Systems Inc
Speaker: Dharmesh Panchmatia, Director, Cisco Systems Inc
Level: 200 (Intermediate)
Track: RDBMS to MongoDB
Cisco’s e-commerce platform is a suite of 35 different applications and 300+ services that powers product configuration, pricing, quoting, and order booking across all Cisco product lines including hardware, software, services and subscriptions. It’s a B2B platform used by Cisco Sales Team, Partners and Direct Customers, serving 140,000 unique users across the globe, handling 4 million transactions per day. The Benefits of migrating to MongoDB were as follows: 1) 5x performance improvement, 2) Fault tolerant architecture, 3) Continuous deployments and upgrades with zero downtime, 4) Faster application development.
What You Will Learn:
- How to transform your e-commerce platform to enable cloud native architecture.
- Bulk data migration in real time between relational databases & MongoDB.
- Best practices for brownfield migration for mission critical systems.
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Transforming a Large Mission-Critical E-Commerce Platform from a Relational Architecture to MongoDB
1. Dharmesh Panchmatia, Director IT, Gaurav Goyal, Architect IT
Transforming a Large Mission-
Critical E-Commerce Platform from a
Relational Architecture to MongoDB
2. 1
How to transform your e-commerce platform to enable
cloud native architecture
2
Best practices for migrating an e-commerce application
platform from relational to MongoDB
3
Planning considerations for data migration from
relational to MongoDB
Learning Objectives
6. Pain Points
10-12 hour
downtimes for
code
Lack of agility due
to monolithic code
base
6+ month
effort for major
software
upgrades
Multiple platform
outages due to
infrastructure, storage
and network failures
1 32 4
8. 1 2 3 4
Zero downtime for
software upgrades
and code
deployments
Increased Agility -
move towards API
based Architecture
5X performance
improvement &
scalability
Fault tolerance for
high application
availability
Objectives
9. Ref Data
REFERENCE DATA SOURCE
DMPRD - RDBMS
Logging
Order Capture
DC1 - Tomcat
Order Capture
Addresses Items
Preferences
Roles
Contacts Logging
DC1 & DC2
Transaction
Data
Transaction
sync
Downstream
Publish
X-Functional
Services (73)
DC3DC2DC1
TRANSACTION DATA STORE
P S S S S
N1 N2 N3 N4 N5
DC2 - Tomcat
1 2
3
4
5
Journey to Cloud Native
10. Changed 110 backend integrations to Rest API calls
X-Functional
Dependencies
7 months without additional head count
Delivery
TimeLine
Delivering 26 Biz capabilities (10H, 7 M, 9 L) along with this changeBrownfield
Rewrote 120 K lines of back-end code;
Touched 600k lines of existing java code
Code
Risk Mitigation
• Backward Compatibility
• Extensive shadow testing for B2B Orders
• Phase rollout
Migration by the Numbers
11. • Migrate all reference data to Elastic Search
• Build APIs to leverage reference data
3. Reference Data
• MongoDB primary database for transactions
• Rewrite SQL code in middle tier (~800,000 lines of code )
4. Transaction Data
• Feature flag based approach to switch to RDBMS in case
of issues
• Phase rollout to mitigate functional parity risk
6. Backward Compatibility
1.Zero Downtime Deployment
• Leverage Blue-Green deployment to deploy front end /
middle tier code
• Use Consul as Service Registry
2. Decouple dependency
on other systems
• Migrate from DB access to APIs
• Influence other systems to provide fault tolerant APIs
• Leverage Kafka for publishing data to downstream systems5. Downstream Publish
Cloud Native Design Cookbook
12. Completed three
quarters without
issues
Software
Upgrades in
minutes without
downtime
Zero Downtime
Response Times
reduced to sub-
sec from 3-5 sec
earlier
5x Performance
Improvement
Eliminated
~120,000 lines
of BE code
Architecture
Optimization
Benefits & Post Go-Live Status
7+ infrastructure
outages since go live,
no business impact
Multiple upgrades
without any downtime
Fault Tolerance
14. Data Model
Error Order
Order
Ext
Financial
Summary
Shipping
Group
Line Contact Payment TaxDtls
Smart
Account
Line Trx
Info
Line
Details
Line
Pricing
Price
Adj
Line
TFS
Details
Child Tables
Order
Child Tables
RDBMS
Tables: 14
Indexes: 60
Sequences: 16
Mongo DB
Tables: 1
Indexes: 7
Sequences: 6
16. Denormalize schema
Optimize Docment Size
Keep # of indexes to less than 20
Minimize # of sequences
Cache sequences in the middle tier
Schema Definition
Complete data migration before query
optimization
Use projections
Update only changed apptributes
Use WriteConcern: 2 / majoirty,
ReadPreference: Primary Preferred
Avoid complex queries in MongoDB
Query Optimization
Best Practices
17. Authentication & Authorization a must
Change passwords every 6 months
Encrypt sensitive data
Define Read/Write and Read Only user
roles
Leverage 3rd data center for all adhoc
queries
Security
Setup alerts in Ops Manager
Use Ops Manager for backups &
upgrades
Print queries taking more than Xms.
Ops Manager
Best Practices
18. Ops Manager is must for
mission critical apps.
Keep an eye on Headroom
during bulk data migration.
Lessons Learned
SAN/SSD is a must
Subscribe to MongoDB bug
advisory & have frequent
upgrades
Most of the performance issues arises
because of missing or wrong indexes.
Create indexes in
background after bulk
data migration
MongoDB provides ACID
properties at document level
v/s transaction.
WriteConcern: majority for
critical transactions
Lessons
Learned
20. DATA
MIGRATION
TO DB
DATA
MIGRATION
TO MONGO
ON USERS
REQUEST
SYNC
MAPPING EVENT
ONE TIME REAL TIME SYNC SCHEDULED
POLLING
CONTINUOUS
SYNC
RESTRICTED
MODE
RDBMS
BACKEND
BASED
WEB SERVICE
BASED
POLLING
BASED
MONGO
DB
POLL ON
INTERVAL
BASIS
DATA
MIGRATION
TO DB
R2M M2R
R2M M2R
R2M
R2M M2R
R2M: RDBMS to MongoDB
M2R: MongoDB to RDBMS
22. 1
How to transform your e-commerce platform to enable
cloud native architecture
2
Best practices for migrating an e-commerce application
platform from relational to MongoDB
3
Planning considerations for data migration from
relational to MongoDB
Learning Objectives
Created 34 new services and moved 20+ reference objects to Elastic Search
DNS – Domain Name Service
ES- Elastic Search
Do not store null values or default values in the document
For documents greaten than 16MB, have another overflow document in the same collection
Use projections to select subset of a document (Minimize Traffic)
Update only changed apptributes instead of updating the complete document
Use findAndModify() to lock documents and $addToSet() to add additional lines to an array
Setup alerts in Ops Manager for Node down, Primary re-election, Connections count