PaaS Comparison 2014
comparing and contrasting leading PaaSes on multiple dimensions
and feature sets to see where CF lead...
Agenda
• Background
• Limitations and threat to validity
• Methodology - briefly
• Results - along comparison main axes
‣ ...
Background
• Based IBM Research study (circa 2010)
• Comparing & contrasting 6 leading PaaSes
‣ Microsoft’s Azure
‣ Google...
Limitations and validity threat
• Challenges in comparing PaaSes
‣ what is claimed may not equal day-to-day reality!
‣ no ...
Overview of comparison
• Found 50 features of comparisons
• Eight groups
‣ (1) workloads, (2) tooling!
‣ (3) integration s...
workloads & tooling
integration services & SLAs
datastores & programming models
management
misc.
Quality of Experience
Scenario: Run (deploy, scale, update, troubleshoot)
a java web application with a database
1. OpenSh...
AWS Beanstalk
Does well:
• Super well integrated into all AWS Services and feels like eases
IaaS friction
• OOTB Monitorin...
Google AppEngine
Does well:
• Super well integrated into many Google Services like Cloud
SQL and other APIs
• Events and r...
Azure Websites
Does well:
• Many options for frameworks
• Well integrated web UX
• Impressive list of SCM integration opti...
OpenShift
Does well:
• Excellent container ssh and port forwarding UX
• SCM and CI directly integrated as an option (did n...
Heroku
Does well:
• Best-in-class polish all around on UX
• Run command, pipelines, stats in logs, marketplace
• Holds req...
Cloud Foundry
Does well:
• Java build tools best in class with maven, gradle, Eclipse and
many buildpacks to choose from
•...
Insights and lessons learned
1. Difficult to extract comparison axes across PaaSes
2. Many comparison points are supported...
Potential next steps
• Socialize, update comparison matrix
• QoE test expanded to thorough scenario
• Could we develop a P...
Thank you
(Q&As)
workloads & tooling
• Workload classes representing cloud apps
• Five types of workloads
1. Analytics - Hadoop-style workl...
integration services & SLAs
• Integration services
‣ includes BSS-types of services
‣ also includes other types of cloud s...
datastores & programming models
• Datastores
‣ relational models support
‣ so-called No-SQL databases, e.g.,
Cassandra
‣ C...
misc.
• Misc. (other)
‣ pattern (or template) -based deployment
‣ support for old-style (mature enterprise) platforms
‣ su...
Nächste SlideShare
Wird geladen in …5
×

Cloud Foundry Compared With Other PaaSes (Cloud Foundry Summit 2014)

34.590 Aufrufe

Veröffentlicht am

Business Track presented by Michael Maximilien, Chief Architect PaaS Innovation at IBM & James Bayer, Director of Product Management, Cloud Foundry at Pivotal.

Veröffentlicht in: Technologie

Cloud Foundry Compared With Other PaaSes (Cloud Foundry Summit 2014)

  1. 1. PaaS Comparison 2014 comparing and contrasting leading PaaSes on multiple dimensions and feature sets to see where CF leads and lags dr.max @maximilien maxim@us.ibm.com IBM Cloud Labs ! and ! James Bayer jbayer@gopivotal.com Pivotal v0.8.0 10 June 2014 credit: http://www.teacherspayteachers.com/
  2. 2. Agenda • Background • Limitations and threat to validity • Methodology - briefly • Results - along comparison main axes ‣ Workloads, Tooling, Integration services ‣ SLAs, Datastores, Programming Models, Management, and Misc. • Insights • QoE - quality of experience test • Potential next steps
  3. 3. Background • Based IBM Research study (circa 2010) • Comparing & contrasting 6 leading PaaSes ‣ Microsoft’s Azure ‣ Google App Engine (GAE), GAE4Business ‣ Amazon AWS+, i.e, EC2, Beanstalk, + services ‣ Heroku from SalesForce.com ‣ OpenShift OSS PaaS from RedHat ‣ CloudFoundry OSS PaaS • Started at IBM but revised with Pivotal in ‘14
  4. 4. Limitations and validity threat • Challenges in comparing PaaSes ‣ what is claimed may not equal day-to-day reality! ‣ no standard measuring stick for most features! ‣ “signal vs. noise” problem measuring some features • Limitation of approach ‣ non-scientific, i.e., subjective rather objective data! ‣ potentially biased (limited numbers of evaluators)! • Value is providing a framework for own analysis • Goal is measure PaaS QoE (quality of experience)
  5. 5. Overview of comparison • Found 50 features of comparisons • Eight groups ‣ (1) workloads, (2) tooling! ‣ (3) integration services, (4) SLAs! ‣ (5) datastores, (6) programming models! ‣ (7) management, and (8) miscellaneous • Question: to what extent does the PaaS support each of the features in group? 0: none, 1: basic, 2: strong, and 3: leading
  6. 6. workloads & tooling
  7. 7. integration services & SLAs
  8. 8. datastores & programming models
  9. 9. management
  10. 10. misc.
  11. 11. Quality of Experience Scenario: Run (deploy, scale, update, troubleshoot) a java web application with a database 1. OpenShift (OpenShift Online) 2. Heroku 3. Amazon Beanstalk 4. Google AppEngine 5. Microsoft Azure 6. Cloud Foundry (Pivotal Web Services)
  12. 12. AWS Beanstalk Does well: • Super well integrated into all AWS Services and feels like eases IaaS friction • OOTB Monitoring is best-in-class ! Not so well: • Responsiveness - VM orchestration takes a really long time • Treats frameworks differently • Must understand many AWS concepts to use effectively • Extensibility – only fixed number of options
  13. 13. Google AppEngine Does well: • Super well integrated into many Google Services like Cloud SQL and other APIs • Events and runtime info well displayed ! Not so well: • Java support limited (limitations on what you can do with threads, requests, SQL driver, services integration) • Forces you to learn too much about Google Cloud • Limited ability to customize container • No debug/shell access in standard AppEngine
  14. 14. Azure Websites Does well: • Many options for frameworks • Well integrated web UX • Impressive list of SCM integration options ! Not so well: • Windows default option instead of Linux • Very much an IaaS / VM feel ! Did not finish yet
  15. 15. OpenShift Does well: • Excellent container ssh and port forwarding UX • SCM and CI directly integrated as an option (did not try Jenkins cartridge but it is available) Not so well: • No auto-detection, must explicitly choose a cartridge • Out of date dependencies like JDK, Tomcat version, etc • No Status, Health Management • Scaling and managing many app instances limited • Host network connections visible with $netstat -a • WAR file support requires SCP
  16. 16. Heroku Does well: • Best-in-class polish all around on UX • Run command, pipelines, stats in logs, marketplace • Holds request at router when updating, unpausing apps ! Not so well: • Java seems to be a 2nd or 3rd class citizen • Must use git, does not support war file directly • Teams and managing multiple apps
  17. 17. Cloud Foundry Does well: • Java build tools best in class with maven, gradle, Eclipse and many buildpacks to choose from • Scales and responds quickly • Logs (both app and audit events) • Super easy Services integration Not so well: • Shell access without workarounds • SCM / CI integrations without workarounds • Provide app metrics • Build in zero downtime updates
  18. 18. Insights and lessons learned 1. Difficult to extract comparison axes across PaaSes 2. Many comparison points are supported in all 3. AWS+ seem to be most mature PaaS (advertised?) 4. Might need to define a benchmark for PaaS a. should contain common apps b. should allow PaaS apps to be tested openly 5. Make spreadsheet and full presentation OSS 6. Crowdsource ratings to remove some subjectivity? 7. Involve members of each other PaaS community 8. Iterate, iterate, iterate
  19. 19. Potential next steps • Socialize, update comparison matrix • QoE test expanded to thorough scenario • Could we develop a PaaS benchmark? ‣ Sample apps that can be ported for each PaaS ‣ Develop data collection tools and spreadsheets ‣ Collect data for each PaaS for benchmarking ‣ Making sense of the results data • Iterate benchmark results for new PaaS releases
  20. 20. Thank you (Q&As)
  21. 21. workloads & tooling • Workload classes representing cloud apps • Five types of workloads 1. Analytics - Hadoop-style workload 2. Transaction - DB-driven (with TX) workload 3. Media - workloads with large static files (read) 4. Web2 - workloads with read/write intensive characteristics but no transactions and not sensitive to read-coherency 5. Mobile - workloads for mobile apps and backend • Tooling support includes IDE, CLI, APIs, git
  22. 22. integration services & SLAs • Integration services ‣ includes BSS-types of services ‣ also includes other types of cloud services • Microsoft seems to dominate this space • Service level agreements ‣ most IaaS and PaaS providers advertise SLAs ‣ most advertised SLAs are coarse grained ‣ no monetary guarantees advertised ‣ most give more time as compensation ‣ no aggregate stats yet
  23. 23. datastores & programming models • Datastores ‣ relational models support ‣ so-called No-SQL databases, e.g., Cassandra ‣ Column-store vs. Row-store • Programming models ‣ support for various scripting languages ‣ support for different frameworks ‣ customized frameworks for cloud ‣ shared-runtimes?
  24. 24. misc. • Misc. (other) ‣ pattern (or template) -based deployment ‣ support for old-style (mature enterprise) platforms ‣ support for multiple target IaaS clouds

×