Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Intro to DC/OS Platform
Ähnlich wie Intro to DC/OS Platform (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Intro to DC/OS Platform
- 1. © 2017 Mesosphere, Inc. All Rights Reserved. 1 Introduction to DC/OS
- 2. © 2017 Mesosphere, Inc. All Rights Reserved. 2 Motivation Solve common problems in cluster computing ● Infrastructure agnostic apps ● Resource management ● Task scheduling ● Container orchestration ● Self-healing infrastructure ● Logging and metrics ● Network management
- 3. © 2016 Mesosphere, Inc. All Rights Reserved. 3 Apache Mesos: The datacenter kernel
- 4. © 2017 Mesosphere, Inc. All Rights Reserved. 4 Apache Mesos TWITTER TECH TALK The grad students working on Mesos give a tech talk at Twitter. March 2010 APACHE INCUBATION Mesos enters the Apache Incubator. Spring 2009 CS262B Ben Hindman, Andy Konwinski and Matei Zaharia create “Nexus” as their CS262B class project. MESOS PUBLISHED Mesos: A Platform for Fine- Grained Resource Sharing in the Data Center is published as a technical report. September 2010 December 2010
- 5. © 2015 Mesosphere, Inc. All Rights Reserved. 5 Resource offers ● Mesos can’t run applications on its own. ● A Mesos framework is a distributed system that has a scheduler. ● Schedulers like Marathon keeps your application running. A bit like a distributed “init.d”. ● Mesos mechanics are fair and HA
- 6. © 2015 Mesosphere, Inc. All Rights Reserved. 6 High utilization time
- 7. 7 DC/OS Architecture Product Overview Virtual MachinesPhysical Servers Private Cloud Public Cloud Security & Governance Container Orchestration Monitoring & Operations User Interface & Command Line HDFS Jenkins Marathon Cassandra Kubernetes Spark Docker Rocket MongoDB +30 more... Existing Infrastructure Mesosphere DCOS Services & Containers
- 8. © 2017 Mesosphere, Inc. All Rights Reserved. 8 Master ● Busy loop all services until their dependencies are ready ● Exhibitor starts, builds zookeeper config ● Once exhibitor has a full exhibitor configuration, Mesos Master detects that and finds it ● Marathon sees Mesos is around and starts up ● Mesos-DNS finds leader.mesos from zk, populates records Agents (Slaves) ● Ping Mesos-DNS watching for leader.mesos and start the agent DC/OS Node Boot
- 9. © 2017 Mesosphere, Inc. All Rights Reserved. Source: https://github.com/dcos/dcos Written in: Python, template language. ● Packaging the internals of DC/OS ● Config generation / de-normalization before cluster launch ● Generates AWS, Azure templates ● Generates On-Prem bash installer 9 DC/OS Installer
- 10. © 2017 Mesosphere, Inc. All Rights Reserved. 10 GUI, CLI or API ● Universe ● Services ● Jobs User- Facing Component s
- 11. © 2017 Mesosphere, Inc. All Rights Reserved. Demo
- 12. © 2017 Mesosphere, Inc. All Rights Reserved. Source: https://github.com/dcos/dcos-cli Written in: Python ● dcos package install kafka ● dcos node log ● dcos kafka broker add 1 Each framework can extend with new subcommands (kafka, cassandra, etc) 12 DC/OS CLI
- 13. © 2017 Mesosphere, Inc. All Rights Reserved. Source: https://github.com/mesosphere/universe Written in: Javascript, Mustache.js Contains Marathon App definitions for launching things on DC/OS Services written in several languages: ● Java ● Scala ● Go 13 Universe
- 14. © 2017 Mesosphere, Inc. All Rights Reserved. 14 1. CLI fetches package templates from universe 2. Substitutes options in config.json into the marathon.json 3. Posts the marathon.json to Marathon 4. pip installs the CLI if the service has one 5. Marathon app starts up on the cluster 6. Framework scheduler finds zookeeper at master.mesos / leader.mesos 7. Framework registers with mesos, sets Framework URL, Name 8. Framework launches tasks via Marathon 9. AdminRouter forwards requests to Framework URL from /service/<name> dcos package install hdfs
Hinweis der Redaktion
- Stay calm Intro yourself
- Why DC/OS, why mesos? Abstraction of machines , not resources Task scheduling and health
- Let me introduce Apache Mesos and the concept of a datacenter kernel
- Spring 2009: 262B project between BenH, Andy and Matei 2009: Nexus project started at UC Berkeley’s RAD Lab (later renamed to Mesos) March 2010: Grad students give talk at Twitter, Twitter starts exploring deployment September 2010: Mesos technical report published (http://mesos.berkeley.edu/mesos_tech_report.pdf) December 2010: Mesos enters Apache Incubator (http://incubator.apache.org/projects/mesos.html)
- What are the main concepts important to understand in Mesos. Mesos provides a level of indirection by being responsible of the allocation and reallocation of resources. Weighted DRF (dominant resource fairness) and allocator module. All of this messaging can seem a little over the top but let’s consider what happens if we have many more schedulers that want to do work and many agents to do that work. Now the master is essentially acting as a messaging hub - except that it has control over which schedulers get offered which resources and tries to allocate them fairly. Marathon acts as the cluster “init” system All additional services we launch via Marathon
- Clustering your hardware this way allows you to share idle resources amongst applications and, more often than not, leads to some amount of cost saving through hardware reduction.
- Mesos by itself is an abstraction of resources, by adding other tools on it we could use it as a paas We started with mesos + marathon, very quickly more components started getting built such as logging,metrics, UI,universe, security for this to be enterprise production ready. We packed all of these components and built dc/os
- Config parameters for cluster setup, networking options,
- Service discovery
- Demo of the DC/OS GUI
- Admin router is basically nginx