1. Testing a Service
A High Level Overview

2. What do we mean with a service?
• What do we really mean with a service?
• Client-server application?
– Do we call something a service just because we have a back end solution to our application that has to be maintained conintuously?
• A normal application?
– Is this a product, even though updates are regularly available?
• A web application and back end?
• Is it something more than just the application?
– Do we mean something more than just the application (back end or not)? Are we delivering an application to the customer, or our we
delivering a service that happens to include an application of some sort?

3. Difference between Goods and Services[9]
•There are five essential differences between Goods and Services [9]
1. The first is that a service is an intangible process that cannot be weighed or measured, whereas a good is a tangible
output of a process that has physical dimensions
2. The second is that a service requires some degree of interaction with the customer for it to be a service. The
interaction may be brief, but it must exist for the service to be complete.
3. The third is that services, with the big exception of hard technologies such as ATMs and information technologies such
as answering machines and automated Internet exchanges, are inherently heterogeneous—they vary from day to day
and even hour by hour as a function of the attitudes of the customer and the servers.
4. The fourth is that services as a process are perishable and time dependent, and unlike goods, they can't be stored
5. And fifth, the specifications of a service are defined and evaluated as a package of features that affect the five senses.
• These features are
• Supporting facility (location, decoration, layout, architectural appropriateness, supporting equipment).
• Facilitating goods (variety, consistency, quantity of the physical goods that go with the service; for
example, the food items that accompany a meal service).
• Explicit services (training of service personnel, consistency of service performance, availability and
access to the service, and comprehensiveness of the service).
• Implicit services (attitude of the servers, atmosphere, waiting time, status, privacy and security, and
convenience).

4. The Goods-Services Continuum [9]
• When is something goods and when is something services? The answer is not black or white, but more of a gray scale
• Pure goods industries have become low-margin commodity businesses, and in order to differentiate, they are often adding some services. Some examples
are providing help with logistical aspects of stocking items, maintaining extensive information databases, and providing consulting advice. [9]
• Core goods providers already provide a significant service component as part of their businesses. For example, automobile manufacturers provide
extensive spare parts distribution services to support repair centers at dealers. [9]
• Core service providers must integrate tangible goods. For example, your cable television company must provide cable hookup and repair services and also
high-definition cable boxes. [9]
•Pure services, such as may be offered by a financial consulting firm, may need little in the way of facilitating goods, but what they do use—such as
textbooks, professional references, and spreadsheets—are critical to their performance. [9]

5. Testing a “Service”
Service Quality
Front End Communication Back End
Application User QoS (Qualtiy of
Back End Logic
Experience Service)
Back End
Application Logic
Capabilities
Application
Capabilities

6. Front End [3]
The front end has to be tested just like any other application, but with the client-service relationship taken into consideration
during testing
Functionality - A set of attributes that bear on the existence of a set of functions and their specified properties. The functions are those that satisfy stated or implied needs.
Suitability
Accuracy
Interoperability
Security
Functionality Compliance
Reliability - A set of attributes that bear on the capability of software to maintain its level of performance under stated conditions for a stated period of time.
Maturity
Fault Tolerance
Recoverability
Reliability Compliance
Usability - A set of attributes that bear on the effort needed for use, and on the individual assessment of such use, by a stated or implied set of users.
Understandability
Learnability
Operability
Attractiveness
Usability Compliance
Efficiency - A set of attributes that bear on the relationship between the level of performance of the software and the amount of resources used, under stated conditions.
Time Behaviour
Resource Utilisation
Efficiency Compliance
Maintainability - A set of attributes that bear on the effort needed to make specified modifications.
Analyzability
Changeability
Stability
Testability
Maintainability Compliance
Portability - A set of attributes that bear on the ability of software to be transferred from one environment to another.
Adaptability
Installability
Co-Existence
Replaceability
Portability Compliance

7. Communication [2]
Problems that occur with the service because of lacking network resources has to be handled by the service in a way that gives a good user
experience
Due to varying load from other users sharing the same network resources, the bit rate (the maximum throughput) that can be provided to a certain data stream may be too low for realtime
multimedia services if all data streams get the same scheduling priority.
• Dropped packets
– The routers might fail to deliver (drop) some packets if their data is corrupted or they arrive when their buffers are already full. The receiving application may ask for this information to be
retransmitted, possibly causing severe delays in the overall transmission.
• Errors
– Sometimes packets are corrupted due to bit errors caused by noise and interference, especially in wireless communications and long copper wires. The receiver has to detect this and, just as if the
packet was dropped, may ask for this information to be retransmitted.
• Latency
– It might take a long time for each packet to reach its destination, because it gets held up in long queues, or takes a less direct route to avoid congestion. This is different from throughput, as the delay
can build up over time, even if the throughput is almost normal. In some cases, excessive latency can render an application such as VoIP or online gaming unusable.
• Jitter
– Packets from the source will reach the destination with different delays. A packet's delay varies with its position in the queues of the routers along the path between source and destination and this
position can vary unpredictably. This variation in delay is known as jitter and can seriously affect the quality of streaming audio and/or video.
• Out-of-order delivery
– When a collection of related packets is routed through a network, different packets may take different routes, each resulting in a different delay. The result is that the packets arrive in a different order
than they were sent. This problem requires special additional protocols responsible for rearranging out-of-order packets to an isochronous state once they reach their destination. This is especially
important for video and VoIP streams where quality is dramatically affected by both latency and lack of sequence.

8. Back End
Dependant on the Back End Solution - To test the server
based applications, you need to perform typical tests like
Volume Testing, Stress Testing, Performance
Testing, Recovery Testing, Back up and Restore
Testing, Security Testing etc. [4][5][6]

9. Service Quality [10]
• The measurement of subjective aspects of customer service depends on the conformity of the expected
benefit with the perceived result. This in turns depends upon the customer's expectation in terms of
service, they might receive and the service provider's ability and talent to present this expected service.
Successful Companies add benefits to their offering that not only satisfy the customers but also surprise
and delight them. Delighting customers is a matter of exceeding their expectations.
• Pre-defined objective criteria may be unattainable in practice, in which case, the best possible achievable
result becomes the ideal. The objective ideal may still be poor, in subjective terms.
• Service quality can be related to service potential (for example, worker's qualifications); service process
(for example, the quickness of service) and service result (customer satisfaction).

10. Service Quality Attributes [1]

11. Service Level Agreement [7]
• Service level agreements can contain numerous service performance metrics with corresponding
service level objectives
Metrics commonly agreed to in these cases include:
• MTTR (Mean Time To Recover): Time taken to recover after an outage of service.
• Uptime is also a common metric, often used for data services such as shared hosting, virtual
private servers and dedicated servers. Common agreements include percentage of network
uptime, power uptime, number of scheduled maintenance windows, etc.

12. Quality of Experience [8]
• Quality of experience (QoE), some times also known as quality of user experience, is a subjective
measure of a customer's experiences with a service (web browsing, phone call, TV broadcast, call to a Call
Center). Quality of Experience systems will try to measure metrics that customer will directly perceive as a
quality parameter (in example : time for a new channel to be played when changing channel on TV). It
looks at a vendor's or purveyor's offering from the standpoint of the customer or end user, and
asks, "What mix of goods, services, and support, do you think will provide you with the perception that
the total product is providing you with the experience you desired and/or expected?" It then asks, "Is this
what the vendor/purveyor has actually provided?" If not, "What changes need to be made to enhance
your total experience?"

13. Scope Split
• Who takes responsibility for what?
– Which organization tests what part of the complete?
• What does the Application Developers test?
– Testing in a mirror of the live environment, or some more basic environment?
– Only test the application in a staged environment?
• What does the back end solution owners test?
– Testing back end through a real applications, or through APIs, or test application, or a combination?
• Is someone else testing the E2E solution?
– Why is this not done by either the back end solution owners or the application developers?
– Is the service location/network dependant?
– Any legal requirements or certifications?

14. Reference
[1]Service Quality Models: A Review
http://www.ftsm.ukm.my/aishah/paper%20pdf_1st%20retail/Nitin%20Seth%20Service%20quality%20models%20A%20review.pdf
[2] Quality of Service
http://en.wikipedia.org/wiki/Quality_of_service
[3] ISO 9129
http://en.wikipedia.org/wiki/ISO/IEC_9126
[4] Introduction to Relational Database Testing
http://www.agiledata.org/essays/whatToTest.html
[5] Testing Client-Server Applications
http://www.softwaretestingstuff.com/2008/04/testing-client-server-applications.html
[6] Back End Test
http://www.scribd.com/doc/21011832/Back-End-Test
[7] Service Level Agreement
http://en.wikipedia.org/wiki/Service-level_agreement
[8] Quality of Experience
http://en.wikipedia.org/wiki/Quality_of_experience
[9] Difference between Services and Goods
http://highered.mcgraw-hill.com/sites/0073403296/student_view0/ebook/chapter1/chbody1/differences_between_services_and_goods.html
[10] Wikipedia – Service Quality
http://en.wikipedia.org/wiki/Service_quality