The document outlines 11 criteria to consider when evaluating Infrastructure as a Service cloud computing providers:
1) References from past customers about reliability, quality of service, and support.
2) Details of the Service Level Agreement and compensation for downtime.
3) Documentation, ease of setup, and first impressions.
4) Performance metrics like storage input/output and network bandwidth.
2. Evaluating Cloud Computing IaaS Providers: Criteria to Consider Page 2
Contents
Introduction.........................................................................................................................................................................................3
1. References, References, References.......................................................................................................................................3
2. Service Level Agreement (SLA)................................................................................................................................................4
3. Documentation, Provisioning and Account Set-up..............................................................................................................4
4. Performance................................................................................................................................................................................5
5. User Interface..............................................................................................................................................................................6
6. Network Infrastructure and Data Center Location...............................................................................................................6
7 Security/Data Segregation........................................................................................................................................................7
8. Billing Methodology and Pricing..............................................................................................................................................8
9. Customer Service and Tech Support......................................................................................................................................8
10. Total Cost of Ownership............................................................................................................................................................9
11. Financial Security......................................................................................................................................................................10
About ProfitBricks............................................................................................................................................................................10
3. Evaluating Cloud Computing IaaS Providers: Criteria to Consider Page 3
11 criteria to consider when appraising
prospective providers
Cloud computing offers significant benefits to businesses both large and small — allowing them to have their appli-
cations developed, tested and in production faster, improving manageability and with less maintenance, enabling IT
to more rapidly adjust resources to meet fluctuating and unpredictable business demand.
The selection of a Cloud computing Infrastructure-as-a-Service (IaaS) provider is so important, it can determine
the success or failure of any IT infrastructure deployment. Given the differences between Cloud and on-premise
deployments, it is important to partner with the Cloud computing services partner with the right experience. Let’s
examine 11 important criteria to consider when appraising prospective IaaS providers:
1. References, References, References
The reputation of the Cloud computing IaaS provider is the final evaluation topic. There is significant competition
in the Cloud computing industry and it is vital for providers to have and maintain excellent references for long-term
viability. Be sure to ask prospective providers for references that can specifically highlight service reliability, quality
of service, quality of support, and network performance. Review their website or ask them for Case Studies and
analyst and media coverage examples.
• Ask your team: Who are you working with in the evaluation process, and do you trust them?
• Has vetting this provider been an easy process, or does it make you question the level of service and
support that lies ahead?
4. Evaluating Cloud Computing IaaS Providers: Criteria to Consider Page 4
2. Service Level Agreement (SLA)
A service level agreement (SLA) is not a guarantee. Rather, it is a commitment from the provider that if things go
wrong, and they probably will at some point if your architecture runs there long enough, they will respond in a
timely manner and credit you a percentage of your bill to compensate for the downtime.
3. Documentation, Provisioning and Account Set-up
They say that nothing beats a good first impression. Everything has to be right the first time or business
opportunities will be lost.
With Cloud computing IaaS, first impressions are your initial views of website content, dealings with the
sales team, and the sign-up process as a whole. Have a look at the provider’s support documentation, FAQs,
API Documentation.
• In the event of an outage or problem, how are responsibilities shared between your company and the
IaaS provider?
• What is the provider’s level of transparency, and do they proactively inform consumers of SLA
noncompliance and breaches?
• In the event of a disaster, does the language limit the provider’s liability at the expense of your
company’s data?
• What types of restitution is offered by the provider in the face of disaster?
• How many minutes of down time is 99.5% versus the 99.95% ofered by another provider on
an annual basis?
5. Evaluating Cloud Computing IaaS Providers: Criteria to Consider Page 5
4. Performance
Cloud providers deliver different compute and network performance leading to differing levels of application and
database performance results and cloud platform architecture
Most Cloud providers can offer computing resources of varying dimensions — from the smallest single-core
instances to the largest multi-core mammoth-memory instances. What’s not so clear, however, is that storage IO
can really vary from one cloud provider to another. And storage IO, not CPU, is often the key in determining between
your application excelling or under performing in the cloud.
Also, many providers’ customers suffer from “noisy neighbors” and variable compute/network performance as a
result of flat networks and shared cores.
• How many IOPS and how much network bandwidth does your business currently use?
• What is the average latency on storage operations?
• Have you seen a report of their UnixBench or iPerf benchmark scores compared to other providers?
• Do potential cloud providers offer flexible, easy-to-create server options to answer fluctuating capacity
needs on demand?
• Is the offering and pricing plan simple, clear and easily understood?
• Did you find the information you were looking for in a timely manner or is it hidden under multiple
website levels?
• Did you speak with sales people and, if so, did they take the time to listen to your needs and offer
suggestions?
• Will it be easy to set-up your account? Estimate how long it will take to set-up a server and have it
running.
6. Evaluating Cloud Computing IaaS Providers: Criteria to Consider Page 6
5. User Interface
The best way to plan and architect an IT environment is to use a large, whiteboard or diagramming software, like
Microsoft Visio, to outline, adjust and improve the complete structure, until your team sees a clear version of the
overall picture. Most Cloud computing IaaS providers require you to code against an API or convert these white-
board sketches into tables of virtual servers and storage, before linking these tables together in a cumbersome
manner. With each subsequent time you make a change, users are required to draw up a new structure, which will
then need to be transferred, again!
ProfitBricks offers a ‘virtual whiteboard’, the Data Center Designer (DCD), which can do much more than a tradition-
al whiteboard. Using the DCD, you design your Virtual Data Center with servers, storage, load balancers, firewalls
and associated networking and IP address assignments. Once satisfied with the structure, you activate your
newly-designed data center with a simple mouse-click, all without any complicated forms and tables.
User Interface (UI) has become an increasingly critical element in the evaluation of cloud computing solu-
tions. As time and resources grow scarce, a good UI can help to mitigate the steep learning curve that IT
departments face in their organizational migration to a virtualized infrastructure.
6. Network Infrastructure and Data Center Location
Just as it would be in your own facility, the physical network infrastructure of a cloud hosting provider is a key
factor in performance. It is important to gain insight into the ‘bones’ of the organization to ensure it can support
your company’s specific infrastructure needs. Ask about their network infrastructure, performance, and physical
location(s). Physical location is very important for a few reasons – statute restrictions may regulate your use of
customer data, jurisdictional requirements will determine the laws that apply in the event of a dispute, and environ-
mental factors should be assessed, including natural risks such as hurricanes, earthquakes, etc.
7. Evaluating Cloud Computing IaaS Providers: Criteria to Consider Page 7
7. Security/Data Segregation
Cloud security is a “hot topic” with traditional concepts playing a major role in the discussion. The majority of these
debates fail to recognize that the actual physical presence of any device or server on the Internet is no longer an in-
herent threat. A server physically located in a secure room within your office building is just as vulnerable to attack
by hackers and saboteurs as any Internet-connected PC. Physical walls, doors, and locks, no matter how strong,
will no longer prevent your vital information from being compromised. At most, all that is prevented is the server
itself from being stolen.
The safety of your data can only be guaranteed if the server is protected against Internet-based attacks. Your best
defense is a professionally maintained system with firewalls and 24/7 monitoring by well-trained, experienced
system administrators. In-house data centers do not typically employ dedicated security engineers, thus providing
hackers with greater opportunities to compromise systems or cause even greater harm. Many cloud hosting pro-
viders will offer advice and assistance to help you design and maintain a highly secure environment.
• Can you engineer redundancies into your cloud infrastructure via its management platforms, or will
the provider work with you to create a 100% available solution? Ensure that the level of support offered
meets your organization’s needs.
Your software developers will be directly impacted by their provider’s Cloud infrastructure, as certain IaaS
offerings may require them to rethink how applications are written and deployed.
• Do potential providers provide technical specifications about their infrastructure during the vetting
phases of Cloud selection?
• How does the physical security of your Cloud provider’s data center compare to security of your own
IT environment?
• Do you feel confident with the security at the data center, or does the provider leave you with doubts?
8. Evaluating Cloud Computing IaaS Providers: Criteria to Consider Page 8
8. Billing Methodology and Pricing
One of the primary driver behind most business moves to the Cloud is to save money. Almost all Cloud providers
offer a utility-based pricing model, similar to electric and water usage, where you use the resources you need and
then simply pay for the resources consumed. But that can be lost when you are forced to buy packages that have
pre-set CPU cores and RAMs tied to each other – often forcing you to over-buy to meet resource needs. Cloud
customers need to be aware of four primary resources - CPU, RAM, data storage and bandwidth. The majority of
IaaS providers bundle these resources together into preset server sizes with monthly “subscription packages or
plans” with peak hourly surcharges. By billing in simple and transparent units for each resource separately users
purchase only what they need. While some cloud providers might offer inexpensive computing or storage resourc-
es, that storage may be temporary, necessitating additional fees for block storage.
• Learn more about how to compare Cloud IaaS pricing by visiting
http://info.profitbricks.com/Compare-Cloud-Computing-Pricing.html.
9. Customer Service and Tech Support
Although many providers claim to offer “extraordinary” levels of support, cloud computing providers have not
been known to offer superlative customer service. The early cloud adopters, generally speaking, were software
developers or very technical teams who didn’t require a lot of assistance. Today, as cloud makes inroads into orga-
nizations of all types and sizes, IT leaders need to know they can contact someone at their selected provider if, and
when, problems arise. It is important to find out if your prospective providers offer free technical support or only
various tiers of paid assistance – either as a subscription or pay-as-you-go fees. You might even want to make a
call to support and check their authentication policy and ask a question or two. Is it a call center
or a technically astute person you speak to?
9. Evaluating Cloud Computing IaaS Providers: Criteria to Consider Page 9
10. Total Cost of Ownership
Many Cloud computing providers hide the total cost associated with the subscription-based pricing, choosing
instead to focus on their “advertised” price. You must be aware of hidden costs and insist on transparency at the
beginning of the relationship.
Some of the most popular hidden fees include:
• Subscription charges – A monthly, pre-paid “package” for a specific level of resources.
• Software licensing fees – operating system, database software images are often left out of discussions
• Burst resource pricing above subscription level — Increased cost of additional bandwidth, CPU cores, memory,
or storage
• I/O server-to-disk back-end network activity
• Amplified levels and types of customer support – Phone and email support are often not included. Some pro-
viders even charge more for the escalation of a support issue.
• Data migration costs – what are the initial costs of modifying your application to fit the specific design require-
ments of a cloud provider, and what is the total cost of transferring your data to the cloud?
Rather than just looking at the lowest price offered by each provider, consider your specific resource
requirements.
• Your IT Budgets: What resources are behind your organization’s transition to a cloud-based
infrastructure?
• How does your budget align with the reality of achieving your current IT goals?
• Competitive Pricing: How is the solution priced?
• Does it vary by compute capacity, network traffic, storage, additional services, flat fees and overages, etc.?
• What levels of support does your cloud deployment require?
• Do you require a hands-off solution or more personal attention and support?
• Will you or your team manage your cloud internally, or do you need managed cloud services?
• What access to support and customer service is available in each scenario?