%in ivory park+277-882-255-28 abortion pills for sale in ivory park
OpenStack Trove in Production at eBay - Trove Day 2014
1. OpenStack Trove Day
August 19, 2014
Session Trove in Production Name-at click eBay
to change
Auston McReynolds, Software Engineer, eBay
Mat Lowery, Software Engineer, eBay
2. The eBay Inc. Portfolio
PLATFORM
8/19/14 tesora.com 2
4. eBay Private Cloud: Like a Public Cloud
• One cloud on shared infra
• Like a public cloud for all eBay Inc properties
• Multi-tenant, isolated
• Different tenants with different needs
• Frictionless, on-demand access
• Reduce time to market
• Enable innovation
8/19/14 tesora.com 4
5. Architecture: Region vs. AZ
Region B
Swift
Keystone Nov
a Neutron Cinder
Glance Dashboard MySQL Misc
Dashboard behind global
load balancer VIP
Region A
Swift
AZ Services Region Svcs
Keystone Nov
a Neutron Cinder
Glance Dashboard MySQL Misc
AZ Services Region Svcs
Region C
Swift
AZ Services Region Svcs
Keystone Nov
a Neutron Cinder
Glance Dashboard MySQL Misc
8/19/14 tesora.com 5
6. Customizations
• VPCs
• Enforce obligations, restrictions, and capabilities
• Associated with Keystone projects
• OpenStack resources mapped to VPC (via metadata)
• Image metadata
• Host aggregate metadata
• Enforcement via custom middleware or scheduler
• Keystone
• LDAP
• Auto-tenancy
8/19/14 tesora.com 6
7. Customizations
• Horizon
• Most use the custom Horizon dashboard vs. API
• Connect To (uses image metadata)
• Copy datastore root password
• Disable UI functionality based on AZ capabilities or
Datastore
• Multi-AZ tenant switcher glues the clouds together
8/19/14 tesora.com 7
8. Packaging and Deployment
• Deployment unit: virtualenv tarball
• virtualenv --no-site-packages –relocatable
• Simplifies rollback
• Not perfect: doesn’t handle OS libs
• Rollout automation by Puppet, Foreman
• Auditing of changes
• Host overrides for testing in isolation
8/19/14 tesora.com 8
9. Monitoring
• Metrics
• Periodic gathering based on service DBs
• Share with end users
• Alerts to engineers
• SLAs
• Ensure being met via sampling; simulates cloud user
• Nova boot
• Swift upload and download
• Capture logs on failures
• Alerts to engineers
• Graphite for graphing and Zabbix for alerting
• Logstash/Elasticsearch/Kibana stack
8/19/14 tesora.com 9
11. Trove at eBay – The Beginning
• Mid-2013
• 1 developer
• Stack on Folsom
• Not integrated, still Red Dwarf
• MySQL only
• Single region
• No backups
8/19/14 tesora.com 11
12. Trove at eBay – Now
• Team (all ATCs)
• Icehouse
• Multiple Datastores
• MySQL 5.5, 5.6
• MongoDB 2.4
• Cassandra 2.0
• Couchbase 2.2
• Redis 2.8
8/19/14 tesora.com 12
13. Trove at eBay – Now - Continued
• Multi-region + HA, with limited VPC availabilty
• Backups for MySQL, MongoDB, & Couchbase
• Contributions to Trove:
• Couchbase Impl + Backups
• Multi-Datastore Horizon
• Clusters (in-flight)
• + many more (incl. 400+ code-reviews)
8/19/14 tesora.com 13
14. Deployment Details
• HA
• Stable* release cuts
• Puppet + Foreman for Control Plane
• Fabric for Guests
• Shared infra, sans RabbitMQ
• RabbitMQ HA Mirrored
8/19/14 tesora.com 14