SlideShare ist ein Scribd-Unternehmen logo
1 von 54
Downloaden Sie, um offline zu lesen
Your Monthly Magazine
on
Software Testing
www.TestingCircus.com
with
Smita Mishra
Testing Circus
Volume 5 - Edition 3 - March 2014
Article submission guidelines –
● Subject of article can be based on any area of Software Testing. If you want to publish your article on
theme based subject please read our announcement of monthly theme published in our site. Article
can be submitted without any theme based subject.
● There is no minimum and maximum length of article. If you feel the article is lengthy, please divide
the article into logically separated parts so that we can print them in a monthly series.
● Give a meaningful title to the article. If you want a sub-title as well , then add that in a different line.
● Add images/pictures if necessary. If you are using any image/picture which is not yours own work,
please include the source. Take care of copyrighted materials. Images need to sent separately with the
article.
● Send us the article in MS word (doc/docx) format only. Pdf files are not accepted.
● Write a short write up on the author(s). Usually 7/8 liners in 3rd person descriptive language.
● Include photograph of author(s). Preferred in high resolution .jpeg/.png format. Ideal size would be
50mmX 50mm.
● You can send your article any time. Since we publish every month, your article can be included in any
month. There is no such thing called as a dead line.
● Send in your article to editor@testingcircus.com with a subject line “Article for Testing Circus –
Author Name – Title of the article”
● If you think you can write a column in Testing Circus for at least 6 months, please submit 3 articles in
advance. We are open to any idea that may improve the user experience of Testing Circus.
● We will publish the articles in our website a week after the pdf magazine is published.
Article
Submission
GuidelinesDo you have something to share with the
testing world? We can make your voice
heard to testers.
http://www.testingcircus.com/article-submission-guidelines
www.TestingCircus.com March 2014 - 02 -
Volume 5 - Edition 3 - March 2014
Testing Circus
www.TestingCircus.com March 2014 - 03 -
Topic Author Page #
Interview with Smita Mishra Jay Philips 5
Examples of Testable Requirements Ulrika Park 10
Taking Testing to the Cloud Vipin Jain & Anubha Jain 14
A Common Pitfall of Test Cases Raj Subramanian 22
Mobile Testing on the Cloud Gagneet Singh 23
How Quicker can mean Slower Peter Morgan 25
7 Types of Testers - What is your identity Arslan Ali 27
Book Review - “Don’t Hire the Best” WoBo 31
A Fake Tester’s Diary, Part - 39 Fake Software Tester 33
Testers to Follow Editorial Team 35
Test Environment for Security Testing Santhosh Tuppad 37
Running Webdriver Script on a VM Mihai Sarlea 39
Test Automation: New Venues, New Challenges ToolsJournal.com 48
TestComplete Support For Automated Mobile Testing ToolsJournal.com 50
Squish Coco Supports Code Coverage Of C# & Tcl Code ToolsJournal.com 52
Testing Events Around the World TestEvents.com 53
Testing Circus Team
Founder & Editor – Ajoy Kumar Singha
Team -
Ÿ Srinivas Kadiyala
Ÿ Jaijeet Pandey
Ÿ Pankaj Sharma
Ÿ Bharati Singha
Ÿ Chanderkant Saini
Ÿ Dwarika Dhish Mishra
Editorial Enquiries: editor@testingcircus.com
Ads and Promotions: ads@testingcircus.com
Testing Circus India
Chaturbhuj Niwas, 1st Floor,
Sector 17C, Shukrali,
Gurgaon - 122001
India.
© Copyright 2010-2014. ALL RIGHTS RESERVED. Any
unauthorized reprint or use of articles from this magazine
is prohibited. No part of this magazine may be reproduced
or transmitted in any form or by any means, electronic or
mechanical, including photocopying, recording, or by any
information storage and retrieval system without express
written permission from the author / publisher.
Edition Number : 42 (since September 2010)
*On the Cover Page - Smita Mishra
W
hat is
w
here?
Write to us at editor@testingcircus.com
www.TestingCircus.com March 2014 - 04 -
Testing Circus is three and half years old. This magazine has been running on
volunteer effort, assisted by few passionate testers from time to time. It has been an
amazing journey so far. But, like all other good things in life, it is coming to an end.
We are announcing discontinuation of Testing Circus after this edition.
It is a sad moment for all of us. Running a regular magazine purely based on
volunteer effort is hard task, specially when all volunteers including me have full
time jobs to look after. We were always supported by many testers by contributing
some awesome articles but let me also admit that it is always a challenge to gather
good articles when you don’t pay the writers and busy testers do not reply to your
emails and direct messages. There was too much effort for our volunteers.
Testing Circus site will remain as it is. We will continue to publish our old articles
in the site. We will continue to promote good testing stuffs in Facebook/Twitter. But
our regular monthly magazine will stop.
I would like to thank all you of you who supported us in this wonderful journey. We
will cross the roads again.
Till then, stay hungry, stay foolish and test passionately.
- Ajoy Kumar Singha
@TestingCircus // @AjoySingha
From the Keyboard of Editor
SMITA MISHRA
Smita Mishra is the founder and chief consultant at QAzone
Infosystems, which is a pure-play software testing organization. She is
a first generation Entrepreneur and is a Test professional who has
spent over 12 years practicing testing and leading test efforts of
varying sizes, cutting across all key domains and technologies. In the
past, she has worked with multiple organizations, likes of - HCL
Technologies Ltd, Fidelity Investments, Nucleus Software Exports Ltd,
Churchill Insurance (Now RBS).
In her current role, she is involved in creating test teams, managing
testing for software companies, leading the overall test strategy for
them. She is also engaging constantly with different forums to assist
growth for women in her field and women in general too.
Organization: QAZone Infosystems Pvt Ltd
Current Role/Designation: Chief Test Consultant & CEO
Location: New Delhi, India
Interview with Testers
www.TestingCircus.com March 2014 - 05 -
1. Tell us about your journey to becoming a software
tester. How did it start and how this has been so far?
Was it planned or by accident?
My journey to become a good tester still continues.
However, it began in 2001, with my first job at
Nucleus Software Exports Limited, where I was
campus placed after my engineering. A batch of
about 20 engineers were picked from across the
nation and put together as the founding
(independent) test team. Honestly, I had no
introduction to “Testing” until then other than the
few chapters I read on SDLC, Verification and
validation in a book of software engineering by
“Roger Pressman”. It was planned by my employers
not me but wasn’t really an accident. Once in testing,
I quickly began to learn it and found it interesting.
I will agree that though I really enjoyed my work,
the initial years were nothing exciting. But over the
years, I realized that I enjoy testing more than
anything else. There have been tremendous learning
through the 13 yrs spent in the field. The more I
learn the more I see I have yet to learn.
Hence, I can easily say that the journey so far has
been terrific. 2014 seems to be extraordinarily good
year for me and I am loving every moment of it, as a
test professional.
2. When did you realize your passion was software
testing?
I would be wrong if I said that it was love at first
sight. It was not. But as they say – good music grows
on you even if you don’t like it much in the first go.
It was similar in my case. I realized I enjoyed testing
early in my work life. But it wasn’t until 2006 that I
began to love it. I attended the first conference in my
life – it was a QAI conference where I was presenting
a practice paper on estimations. Met many testers
there and I loved the feeling of belonging to a
community too. Then in 2007 I went to work in GE
Healthcare account under the most dynamic
technology leader and sharp as razor – Gazanfar
Hakeem and an excellent Test Manager Smita Sethi
(who also happens to be a very sharp professional). I
got exposed to very technical testing in DW-BI (ETL
Testing) and performance testing of these ETLs and
implementing lean methodology. I would term that
as the turning point in my career on how I began to
look at my testing career. I realized I was cut out to
do this.
3. Do you regret being associated with software
testing today? Given a chance would you move from
testing to any other field in IT?
* Interviewed by Jay Philips
www.TestingCircus.com March 2014 - 06 -
No, clearly no. Quietly no. Loudly no. Simply no.
I am proud to be a part of an ever evolving field and
that has a huge legacy of achievers. I can easily say I
never had second thoughts about my work. Also, I
would find it very difficult to work on a single
technology or domain all my life – and to look
conversely at it - which other field can give me so
much exposure in understanding “what really
matters” in IT field, while keeping me hands on with
all latest technology.
4. What does QAZone do? How are its services
different from the services offered by other similar
organizations?
QAZone is a pure play software testing organization
that offers Testing solutions and services, Test
consulting, Test training and test support services
(Test data management, test environment setup). We
offer Business as usual testing as well as specialized
testing. We have built a client base of over 20 clients
in last 3+ years.
Our business focus is our key differentiator. Our
testing solutions are designed to work for your
business. We look at technology as a platform to
make business happen. All our testing has a
component of domain focus which becomes a very
serious need of our clients when we are dealing with
regulated environments like Aerospace and
Healthcare.
Our next differentiator is that against the trend - we
believe in hiring locally, even if it means training
potential resources at our expense. For example –
anytime we need a resource in US, our first approach
is to find a resource locally. This has so far proven to
be a very fruitful approach.
5. Last year you organized a conference called
ThinkTest in India. Was that the first year of the
conference? What were some key sessions and
feedback?
We had planned for a conference ThinkTest 2013,
which would have been the first of our initiative in
this direction. We were very keen to bring James
Bach to this part of India (and we still want this to
happen). Unfortunately, due to unforeseen
circumstances at my personal end and not having
suitable alternatives, we couldn’t take this forward.
We will keep the test community updated of future
plans related to this.
6. In February 2014 you held the first meetup for Test
Practitioner’s Club. What made you want to create
this meetup?
It always feels easier to work with people whom you
have met and can relate to them as a face and as the
vibes you share with them. All leading test
conferences are still physical conferences not
webinars. That’s because learning happens in many
forms today and one of them is networking post
conferences. With the thought of building a network
of co-learners, we created a LinkedIn group – Test
Practitioner’s Club and planned for its meet-ups that
would help local testers to learn more from each
other and get introduced to global platforms too. If
we could – we would have such test meet- ups all
over the country. We are planning to do one meet
every month at the very least. We will keep posting
the updates from our meets at my blog, for all to
read.
7. I noticed that you are certified in ISTQB and QAI
(CSTE). What made you decide to go for both
certifications?
Yes, that’s true – I have indeed done both the
certifications. I went for ISTQB (in 2005) because
until then I had no certifications. And under our
training and development program, my organization
paid for this certification. However, later on I went
for QAI (CSTE), as it was not widely done so far due
to its cost and students failing in it and this made it
SMITA MISHRA
www.TestingCircus.com March 2014 - 07 -
appear more respected and challenging. I paid for
this one myself.
8. Do you recommend that all testers get both
certifications? Are there any other certifications you
would recommend to other testers?
I would let testers decide this for themselves. But I
can say it with reasonable confidence that these
certifications do not prepare testers to handle serious
testing problems. I enjoy being part of miagi-do and
discussing actual test issues with real testers there
and how to resolve them. I have heard very good
things about BBST and most of the leaders I follow
are associated with it. I would suggest all testers to
atleast go through the content and format and decide
for themselves.
If learning is the objective and certification is not
really the goal, I would suggest testers to go through
James Bach’s site for RST classes and RTI (online)
classes.
9. According to you, what is lacking in today’s
commercialized training industry, especially in
testing?
With all due respect to the 2 certifications I did, I
believe an ideal test certification would be one which
would have more practical problems to deal with, as
tests for their students and would involve real life
situations of testing than focusing on glossary of
testing and terminologies. Also, I would invest in
domain led testing certifications for my team. So far,
there are no such certifications available which
would certify testers for regulated environments like
those of aerospace and healthcare. I have heard of a
new certification that Cem Kaner has come up with. I
need to look into that one too, if it has domain based
testing certifications.
10. What qualities will you look for in a candidate
when you want to recruit someone for software
testing job?
I like to work with enthusiastic people who enjoy
learning new things and working on new areas. This
is true not just for software testers but front office
executive, accountant, HR and Admin folks - all of
them.
Many today do not feel the need to look at the past
experience and hire only for attitude. However,
when I am hiring testers particularly – I like to see
the kind of work they have done before and like to
hear their stories of projects on exactly what was that
they were testing and what were the key defects
found. What was their approach and the key
challenges faced and how did they bring up the
challenges to the notice of relevant stakeholders and
how did they do the contingency or mitigation as
applicable.
Formally speaking, we have 2 set of patterns – one
for freshers and other for lateral hiring.
For laterals particularly - We have a process that
helps us map the required technical skills with the
available resumes. So, anytime we have a need we
look into our database of testers who have applied
with us in past and from the matching resumes, we
shortlist based on availability / joining time and
expected compensation. Post this, we share certain
project links with these potential candidates and
expect them to submit us their Bug Reports in a
given time frame. Finally, after a candidate is found
technically suitable, we setup face to face
interactions to evaluate more HR aspects and general
communication.
For freshers, it’s really about what they have to offer
beyond their formal education. No firm expectations,
as long as they are able to convince us of them being
fast learners and having aspirational attitude, they
might.
SMITA MISHRA
www.TestingCircus.com March 2014 - 08 -
11. What will you suggest to people who want to join
IT industry as software testers?
I am very happy to be part of the software testing
community. And I can only suggest the new folks
that learn what is testing and then get into it. I find
people to be doing testing for all the wrong reasons
like – its easier that Development, it doesn’t need
coding, timelines are easier and hence the work
pressure, etc. None of this is true. Testing is
becoming as challenging as any other part is with the
ever evolving IT and
When I look back, I think the biggest gap that I see
during my initial work tenure was how closed and
we were in our approach to testing. working in silos
as a closed group and not being exposed to the
whole world full of knowledge and awesome
teachers and trainers willing to invest time in you.
12. Name few people you would like to thank,
people who helped you directly or indirectly in your
career as a software testing professional.
I would like to thank my family for always standing
by me – ALWAYS. But a special mention of my son
here for being the world’s most loving and caring
son and being very understanding when his mommy
needs to work.
At work front there are too many names and I would
like to thank each good and bad interaction, because
it has helped me grow in one way or other.
Everyone I know – Thanks for being part of my life.
13. One last question – Do you read Testing Circus
Magazine? If yes, what is your feedback to improve
this magazine?
Yes, I do read Testing Circus Magazine. I first heard
about it in 2010 I guess when my manager Akash
had been invited to be interviewed in it. But I never
followed it then. I began to read Testing Circus since
sometime in 2013 onwards.
I liked the original format of the magazine. But the
new format is truly amazing. I really enjoyed the
editions coming out in 2014. And I think they are
doing all the right things to get noticed and to put in
right content. My only words will be – continue the
good work.
And maybe you can cover small meetups and group
test events happening across the globe as a regular
column. This could help other meetups to learn what
more can be done and how to be effective.
Blog/Site: http://smitamishrablog.wordpress.com
Twitter ID: @smita_qazone
SMITA MISHRA
http://www.testmile.com
The simple thing is this - write your requirement as a
test. With defined inputs and outputs. Expected results
and expected (and unexpected) data.
If you do this, your life will become bright, shiny, and
you will live happily ever after. Or at least your soft-
ware will become much much, much more reliable.
And you’ll probably find out a lot of things about your
ideas before you’ve invested in building unnecessary
features and details.
You can do this with high level requirements, such as
business goals and overall objectives, as well as with
low level isolated features, and everything in between.
I think that the “everything in between” part is where
we (software industry people) lacks the most care and
insight about the importance of concrete, testable re-
quirements.
When it comes to high level requirements, we may have
business people who do follow up business cases &
objectives, i.e. test the results of the investment, at least
I have seen it done once or twice.
When it comes to very low level requirements, or mi-
cro-requirement as my friend @spindelmanne call
them, TDD do take care of it to some extent. Such as
“When renaming item x the list will keep the same sort
order”. It’s hard to separate micro-requirements from
real business requirements sometimes. “What is really
a valid input string here?” “How should we present the
date format” etc but good developers generally can
make some good micro-requirements decisions.
For the “everything in between” requirements, we have
a lot of work to do to make them testable. From what I
know it seems as there are mainly 3 ways of communi-
cating requirements today.
Either you’re “agile” and have a loosely defined prod-
uct backlog, filled with short user stories and then not
so much more information.
Or you have a heavy regulated requirements process,
with hundreds of pages of use cases or “shall”-require-
ments. Often with abstract statements such as (from
real example): “Purchase has generated a receipt”
Or the ad-hoc requirements: “Let’s send an email to the
developer telling what I need to have”. Example: “We
need to update the purchasing order receipt page. Right
now it doesn’t show the total. The total need to be there.
When can this be done?”.
As a requirements analysts / project manager I have
seen and practiced a way out of these three abstract,
ambiguous, non-informative ways of communicating
requirements. Much thanks to developers who serious-
ly cared about taking TDD to the next level, and by
having the chance to work with testers close by who
taught me how to express what I want as test scenarios.
I’ll share some examples from a previous project.
www.TestingCircus.com March 2014 - 10 -
- Ulrika Park
Requirements People Need
Your Help!
Examples of Testable Requirements
A testable business requirement
I was asked by the business owner to implement a fea-
ture: “Cardholders should be able to edit the rights for a
whole household to use the money on their bonus card”
Since money and banking was involved, it was a bit
complicated to implement.
My first question to the business owner was: “why?” and
how will you know it works?”
After quite a lengthy conversation, he said that what he
really cared about was that the money on the bonus
account was spent. He didn’t want the money to stay on
their bonus cards.
“How can we verify that this target is achieved?” I asked.
“Well.. he said. If the money is spent, then the feature
works.”
“So.. when in time is realistic that we can check this..?”
“Well.. within 6 months we should have a better rate of
spending the bonus money than now” he said.
“Ok. So what do you mean by ‘better’?”
“Hm…” he said. “I’d be content for now if 50% of the
total money paid out to customers bonus accounts
would be spent”.
“Thanks for clarifying! Our feature could help out with
achieving that goal. But to achieve this, other things are
involved. Customers need to know about how to share
bonus money between people in their family. How will
they know? Marketing, customer service.. a lot of factors
might affect if this feature is used by the customer.”
“Yes, of course. But this is what I really care about. And
when you have a feature households can use, we should
do an effort to inform customers”.
Good. Now we had a high level business goal, a testable
business requirement. Even though our feature
wouldn’t be the sole solution to make the business
achieve this goal, knowing the target for sure helped us
a lot in developing the feature.
What to do when you don’t have access to the business
owner? When maybe you just get a bunch of use cases
from somewhere to implement?
Well, in these cases I try to define my own hypothesis
about the main goal and result. “This is how I / we have
interpreted the target since we don’t know” and then
show for those stakeholders I do have access to. Often I
do get some feedback on my hypothetical business goal
statement. Even “You’re totally wrong in your assump-
tion!!” is good to know before developing anything. And
you have a reason to ask for answers.
Before testing or developing any feature, we have to
know or make a clear defined assumption about the
expected result for business.
A testable middle level, user requirement
So now we knew the business goal of the feature. The
feature could be implemented in many ways, with op-
tions from everything from printing and scanning paper
forms to digital authorization functionality.
As a requirements analysts, turning into a tests-before-
development tester, I defined some user stories. (We did
a lot of other things too to understand what solution
might fit, but that’s another story).
The main “middle level” user story:
As main cardholder I want to authorize other card-holders in
my family in order for anyone to use the money on the bonus
account.
Before communicating this to the development team, I
start to think about.. how to test this? What would I test?
I brought in a tester for a chat. The tester was busy with
other assignments, but he did have a few minutes to help
me out. And I asked him “How would you test this
story?”
“Identify scenarios” he told me. And with some coach-
ing I made up some scenarios. (here is just a snapshot to
keep the article short)
Scenario 1: Give authorization to other cardholder in a
household with only 2 cardholders.
Scenario 2: Give authorization to other cardholder in a
household with several cardholders.
Scenario 3: Authorization process is actively canceled
by cardholder
Scenario 4: Authorization process is canceled by un-
planned interruption
etc.
Then, exemplify these scenarios with Gherkin inspired
syntax:
www.TestingCircus.com March 2014 - 11 -
Scenario 1: Give authorization to other cardholder in a
household with only 2 cardholders.
Given that:
Household has 2 and only 2 cardholders
The 2nd cardholder doesn’t currently have the right to
use bonus money
Main cardholder has actively selected the 2nd cardholder
The 2nd cardholder is >= 12 years old
Expected result:
Information is shown: “You have now given authoriza-
tion to <2nd cardholders full name> with SSN: <2nd
cardholders SSN>.
The 2nd cardholder now has authority to use bonus.
Scenario 3: Authorization process is actively canceled by
cardholder
Given that:
Household has 2 and only 2 cardholders
The 2nd cardholder doesn’t currently have the right to
use bonus money
Main cardholder has actively selected the 2nd cardholder
and
has entered external digital identification application to
authorize
and
shuts down the digital identification application
Expected results:
The 2nd cardholder doesn’t have authority to use bonus
Message to main cardholder: The change has been can-
celed.
These were just two examples. Doing this I had now:
1. A specification that can be used for testing
2. Got forced to find out the exact business rules regard-
ing who actually could be authorized. The age limit, for
instance, was discovered by defining the tests. By defin-
ing test scenarios, I got a foundation to ask the right
questions to domain experts. “Can anyone be autho-
rized?”
3. A way to at any time go back and show for who it
might concern (developers, stakeholders, customer ser-
vice) what exact requirements and rules we built for.
These are just small examples from a big domain, so I
expect you have a lot of critique “Where are scenario x??
And those examples could for sure cover much more!
And aren’t that statement a bit vague? What does ‘ac-
tively’ mean really..?”
The point here is not to give the full picture. For that I’d
need to write a book :-) The point is to show that by
defining tests while working with the requirement, the
requirement got much more explicit. We saved time for
our tester, who could focus on exploratory testing when
time came. We saved a lot of time for developers who
actually got the information ahead development on what
would be tested, and what rules should be applied, so
they saved a lot of rework.
Also when having conversation with developers about
the examples we modified them a bit, and removed
unnecessary ones, or added missing. Some of the scenar-
ios could even be quite easily automated while develop-
ing, which also saved us a lot of time.
So, what to do as a tester then? If you’re just not involved
in creating the requirement definitions? Maybe the re-
quirements are just handed off to you in one way or
another, and when you get them they’re not testable at
all?
One thing you can do as a tester, is to make an effort to
be included when other people are working with re-
quirements. Offer your help!
Only once I have been approached by a tester with this
offer!
It’s always me who’ve approached testers to help me
with making testable requirements.
Have you ever as a tester tried to offer your help – and
seriously tried? To the guys working with requirements?
And there are much more efficient ways than to say “If
you need me you know where I am”.
A tester could say something like this: “I know I’m
going to work on testing for this project a couple of
months from now. Is it possible for me to see some of the
requirements already now? It will help me get ramped
up quicker when I’m in the project. I’m in another project
right now, but still I should be able to take one or two
hours to look at what you have. And yeah, incomplete
use cases or draft user stories will work fine too!”
When you do get hold of some kind of requirements
early in the process, take a quick look. Try to define some
simple test cases or scenarios, and maybe you will get an
opportunity to discuss these with the requirement ana-
www.TestingCircus.com March 2014 - 12 -
lysts (or whatever role who works with the require-
ments).. Then why not inviting her for a lunch or a
virtual coffee break (or other social excuse) over Skype in
case of distributed teams?
Requirements people need your help! They just don’t
know it yet. Who, if not you, will seriously invite to
make their work testable?
Ulrika Park is a
requirements geek with
a passion for testing,
methods, learning & the
development of prod-
ucts & services within
organizations and
teams.
With 15 years of
experience in software
development, management & business
she now works at SmartBear. She believes in the
synergy of people, software and quality thinking
to change the world.
www.TestingCircus.com March 2014 - 13 -
Look around in various IT companies, flip pages of some IT magazine, read blogs, see forums and you will be
sure to find the today’s buzz word, the Cloud. You cannot find a word that has created so much of ripples than
the Cloud. Very dramatically, the thinking process on providing computing resources has been shifted to Cloud
computing. With this shift of focus, a lot of confusion has accumulated along the way in people’s mind.
Companies are taking this concept very seriously. Many have moved to cloud, and those who are still on the
“ground” are planning to move to cloud in next couple of years.
People are still confused. They still are not comfortable with the Cloud thought. You ask anyone what a cloud
service is and you still won’t get a very confident answer. Even if you get an answer, ask what benefits they offer
and what are the disadvantages at stake, and you can see the person will be uncomfortable answering this. Make
him skip the discussion by asking how secure they are, and it will be end of discussion. When it comes to testing
applications, we are not fully sure whether to adopt the cloud approach of testing or not. What are the benefits
we get and what are the hurdles on the way?
Let’s take a closer look at the Cloud and how it can benefit out day today testing activities.
So what is a Cloud?
Cloud computing is a business and economic model. This model has been successfully deployed and executed
for various material commodities since its inception, but in the recent years it has been formalized for IT products
and services’.
Let me try and explain the basic difference between a Cloud service and a Non-Cloud service.
Consider that you have to move from one place to another. You can use your car OR use the services of a Taxi.
Both of these vehicles have several similarities
· Both are automobiles, having very similar structure and machinery.
· Both provide basic functionality of transferring people / goods from one place to other.
So where lies the difference?
The difference is in the business model for the service provided by them.
· When the car is owners, the owner has to pay for the fuel, regular maintenance and even possibly a
garage. In turn, the car provides the service solely to the owner - you.
www.TestingCircus.com March 2014 - 14 -
- Vipin Jain
- Anubha Jain
Taking Testing to
The “Cloud”
· When the car is a Taxi, the service provided by the taxi cab can be described as ‘Travel as a service’. Who
owns the car and who pays the maintenance is not your concern. As a customer, you just have to pay to
travel from your place to another desired place. No maintenance fees and no parking fees need to be paid
by you. This responsibility lies with the cab driver.
The Cloud is synonymous with
the phrase ‘On Demand’. You
pay only on demand (when you
require it.)
Cloud gained momentum when
IT industry got associated with
it. Today, we find that there is a
huge range of products and serv-
ices available on demand. All are
“As a service” e.g. ‘Games as a
service’, ‘Java as a service’, ‘Stor-
age as a service’ and the list is
endless.
Fig 1: TaaS or ‘Travel as a service’ – A Metro Cab service
How Testing and Cloud work together?
Cloud-based testing offers a remarkable combination of low costs, pay-per use model and elimination of initial
capital expenditures. The benefits, however, are more than just cost effectiveness. The non-cost factors include
on-demand flexibility, a respite from holding various infrastructure assets, enhanced collaboration, higher levels of
efficiency and, most importantly, reduced time to-market for key business applications. Economically the vendor
and end user gets huge benefits from the Cloud. This comes directly from reduced subscription prices of any
product or service.
Other benefits are:
· Cloud Apps are scalable: The elastic business model, as it is popularly known as, can be customized based
on the requirement
· Auto-Provisioning: Depending on needs of end users, various Cloud vendors provide and withhold the
offerings in a manner which is in an automatic and self-serving format.
· Unlimited resources: At least the end user thinks and feels this. The services / products are available as and
when they are demanded and in the required quantity
The Cloud computing model should be coined as ‘Green Model’ as it maximizes usage of resources and minimizes
wastage making it environment friendly.
Current state of testing in cloud
Many companies are still taking a cautious approach with cloud computing. This is not the case with testing
however. They are largely ready for testing in cloud and following reasons will account for that readiness:
· As we all know, testing is not a one time, but a periodic exercise and each project will require a new
environment to be set up. If a company creates a Test lab, it typically sit unused for longer periods, resulting
in a waste of cost, electricity and space. Lots of published reports indicate that more than 50% of the
www.TestingCircus.com March 2014 - 15 -
technical infrastructure for testing remains underutilized.
· Since testing still being considered as a must but non-critical activity for business, taking testing in cloud
premises is pretty safe as it doesn’t include important company data and has minimum impact on the
organizational business activities.
· Today’s applications are increasingly getting dynamic, more complex, distributed across continents and
more component-based. Testing them is getting more challenging. For instance, with mobile and Web
applications testing, testing needs to be done for multiple operating systems and regular critical updates,
various browsers and their versions, variety of hardware and a large pool of concurrent users to understand
real time performance. It is pretty difficult to follow the age-old approach of creating so many in-house
testing environments. These will become very complex and will need huge capital and resources.
Fujitsu in a 2010 research suggested that testing ranked second (57%) as the most likely workload to be put into the
cloud after Web sites (61%). The on-demand provisioning by Cloud addresses all the above explained issues with
one click. On top of it, the effort and resources saved by using cloud can be redeployed for core business functions.
The Cost Factor
Economic benefits are the main factor influencing companies to take testing to cloud. Another 2010 survey by IDC
hinted the same. As the global economy recesses, companies continue to find ways to regulate costs and improve
ROIs. Cloud testing reduces the unit cost of computing.
www.TestingCircus.com March 2014 - 16 -
Small and medium-size businesses that cannot afford high costs find cloud-based testing as a new lifeline. The
companies no longer need to invest in infrastructure and software licenses. They also do not need to worry about
various configuration issues and maintenance of test environments, and pay only for what they use.
Beyond cost Benefits?
Cloud is not all about cost saving. There are lot more benefits companies can extract using it.
- A standard infrastructure and pre-configured software is available, reducing efforts in getting servers and
licenses.
- On-demand provisioning helps companies to think forward instead of spending time to set up test labs. All
testing resources required for testing exist within the system and can be called upon instantaneously.
- Better analysis and control are offered to test teams to build and execute their tests and identify the
bottlenecks. This helps in identifying possible runtime bugs a lot before they are actually found.
- It’s a great concept in motion between geographical distributed teams. Once a tester logs in and runs a test,
the results are available over the cloud. The developer can then assess it and fix over the cloud itself. This
eliminates back-and-forth communication between teams.
Limitations
- Lack of standards: Absence of universal/ standard solutions to real world problems is a big issue. Each
cloud provider can have its own hardware, operating models and prices and may or may not offer any
interoperability. This poses a huge challenge for companies when they plan to switch to a new vendors
- Security issues: Security in the public cloud is still a huge concern because the data may be stored in a
location which is outside a company’s legal and functioning jurisdiction.
- Usage: The everyday usage costs increases very rapidly if testing is done without a proper usage of
cloud-based test. Though pay-as-you-go clouds are used, it can be expensive if the testing is out of sync with
requirements.
www.TestingCircus.com March 2014 - 17-
- Performance: It is always an issue since public
clouds are used by many users in parallel. Situa-
tions got created a company had to wait for the
required bandwidth to execute their tests.
Conclusion
This paper tries to explain the benefits and limitations
associated with taking testing to Cloud. We have tried to
explain that why companies should start small and gain
confidence slowly to capture maximum benefits of
cloud-based testing. Once they believe that it has led
them in speeding time to market, lowering of costs and
ensuring standards compliance, they can go big. Using
of pay-as-you-go or on-demand services intelligently
and efficiently, companies can reduce cost of operation
and ownership. Companies should pilot cloud-based
testing as early as they feel comfortable before going to
mainstream testing.
Bibliography
- “Determine if a Cloud is Usable,” blog post,
Bloomberg Businessweek, Jan. 31, 2011.
- “Solving the Challenges of Enterprise Mobile Appli-
cation Development With Cloud-Based Testing,”
blog post,
- CIO, Feb. 17, 2011.
- Rajagopal Sattaluri, “Testing Considerations for Ap-
plication Migration to Cloud Computing,”
- Cloud Computing Journal, Feb. 8, 2011.
- “Cloud Computing: The Good, Bad and the Ugly,”
blog post, Dynamic Data Inc., Feb. 1, 2011.
- “Cloud Testing, A Growing Trend,” blog post, So-
nata Software, April 4, 2010.
- Nivedan Prakash, “Cloud Testing: Attracting De-
mand,” Express Computer, Feb. 1, 2010.
www.TestingCircus.com March 2014 - 18 -
Vipin has dedicated last 10 years of his professional career to the software
quality and testing. Currently working with Metacube Software, India, he
developed his key skills in developing automation frameworks and automat-
ing applications. With a proven record of implementing and refining test
processes for various clients across the globe. he is the author of several articles
and seven well sold books in India. He is pretty active within the software
testing community by speaking at International and national conferences,
writing articles and contributing to various blogs and forums.
Anubha is working as Associate Professor & Head, IT department, Internation-
al College for Girls (ICG), located in Jaipur, India. An academician for last 11
years, she has been involved in teaching and mentoring several students in the
field of Computer Science. Knowing that teaching is the best form of giving
knowledge back to society, she worked as a lecturer in Subodh college, Jaipur
before settling in her current role at ICG. She is currently pursuing her PhD, in
field of Information architecture. Anubha is the author of 9 books and a regular
contributor in various forums.
To Subscribe
Click Here
Still relying on
reading
Testing Circus
from tweets
& facebook
updates?
Subscribe
just with your
email id and
get the
magazine
delivered to
your email
every month,
free!
Testing Circus
www.testingcircus.com
www.TestingCircus.com March 2014 - 19 -
I recently gave a talk at a conference. I learnt many
things from my interactions with the attendees at my
session. One specific thing that stood out for me was
how – testers no matter how intelligent, smart, critical
thinking they are, still fall into the common pitfall
of using test cases for communicating testing progress and
test coverage.
If anyone reading this article, is still under this mindset,
I would ask the following questions-
· Do the test cases cover each and every scenario, each
and every part of the system under test?
· Can you give accurate information of testing effort in
terms of test cases? Does it really make sense?
· Are we trying to hide from reality and give some sort
of number to make us and our project managers feel
good when we report the number of test cases covered
to communicate testing progress?
· What do you actually mean by test coverage? Can
any other information about the system influence
this?
Here is the common pitfall of completely relying on test
cases.
Firstly, 100% test coverage of the system is impossible.
“Exhaustive Testing” is a misnomer. This being said, if
anyone says “I have 10000 test cases, I executed all of
them and thus all my testing is completed”, it is flawed
because it is impossible to cover every scenario of a
system by just executing all the test cases
· What if there are some hidden scenarios that the
tester still hasn’t uncovered?
· What if another system influences the system under
test in a different way than expected?
· What if the system behaves differently under differ-
ent situations and our test cases did not cover this?
….and so on. There are so many “What if” questions to
ask ourselves. It’s an endless list.
So the point is, NO; we cannot possibility write every
possible test case to cover each and every scenario of a
system. This is where as testers, we think about testing
approaches complementary to just executing test cases
like exploratory testing, automated testing, combinato-
rial testing and other approaches based on the context
and scope of what we intend to cover. This may help to
uncover other weird scenarios or trigger unexpected
outcomes. With this being the case, it may not be valid
to say, “I have 100% test coverage because I executed all
the test cases”.
Secondly, you may then ask, how should I give up-
dates on testing progress to my stakeholders? In my
experience, I usually do not have a problem in explain-
ing testing progress to my stakeholders even if they are
old school and believe “test cases are the solution to all
the problems”. I would of course have some test cases
that I execute, but also do lot of complementary testing
to test case execution. Finally, I give the approximate
percentage of modules covered/tested in the system as
my testing progress.
So for example- If there is a system A, I generally split
it into different modules M1, M2…….Mn (test areas). I
use different testing approaches to test these modules
www.TestingCircus.com March 2014 - 20 -
- Raj Subramanian
A Common Pitfall
Of
Test Cases
and give an update in terms of the percentage of mod-
ules I have covered in the system. (This is a general
approximation as there is always a possibility that I
haven’t thought about some other module in the system
which may cause problems. To err is human.)
So, say if I had covered M1, M2 out of the 5 modules I
have identified i.e M1, M2……M5, then I may have cov-
ered about 40% of the system ([2 modules/5 modules] *
100, in case you were wondering how I came up with
this percentage ) . This is how I give my updates on
testing progress. This by itself helps to give the stake-
holders some level of visibility into testing progress and
helps to make decisions accordingly.
To summarize, I think as testers we know it is ethically
wrong to make false claims. That being said, just using
test cases to estimate the amount of testing progress and
test coverage falls under this ethical problem as we are
giving wrong impression to our stakeholders who trust
us. Doing this tarnishes tester credibility which is one of
the most important factors that influence a tester’s work
and helps to establish trust among his peers.
Raj Subramanian is a pas-
sionate tester, who has
previous experience work-
ing as a developer and
moved to testing to focus
on his passion. He gradu-
ated from Rochester Insti-
tute of Technology with a
Masters in Software Engi-
neering, worked as a developer for a payroll
processing company and currently works as a test
engineer for a major insurance company with
specific focus on mobile testing. He has been pret-
ty active in terms of learning and contributing to
the testing society by speaking at conferences,
writing articles, blogging and being directly in-
volved in various testing related activities. He
currently lives in Cleveland, Ohio.
Raj blogs at http://www.rajsubra.com/blog/
And tweets with @epsilon11
www.TestingCircus.com March 2014 - 21 -
moolya sucks
we test fast and don’t know to make
more money from our customers.
we are like this only
sales@moolya.com
You have been asked to test the latest Mobile based
application, created painstakingly by your organization
over the last few months. All you have with you is your
exclusive Smartphone, which you saved for, and
bought, over the last few months / years. The choices
are now with you, either beg/borrow/steal from col-
leagues or do the right thing. Fire up that web browser
on your Smartphone, open up ‘Google’ and have a go
at searching for ‘Mobile Test Partners on the Cloud’. I
would generally as a rule go for the second option and
not put ‘my precious’ phone to the test on un-tested
applications.
What you achieve when you go through the search, is a
wide variety of web sites and commercial vendors of-
fering this facility to you, at a fraction of the cost that
you would spend in setting up your own facilities and
test lab. You need to now make an informed decision by
researching and figuring out which one of these would
serve your purpose the best and also give you value for
the money spent.
Mobile testing has come a long way. From the initial
fragmented scenario of having to check on each kind of
screen resolution and phone type and screen size, add
to that the variety of mobile browsers being offered by
the various vendors, to the current situation of having
apps created using HTML5 versus native apps. Adding
to the general confusion is the non-app area, where
companies wish to check the overall ‘responsiveness’ of
their “web pages” across the same wide variety of
devices, which includes [and not restricted to] iOS
(iPhones and iPads), Android (Tablets and Phones –
low end and high end), and recently the Windows 8
(WP8 devices and Windows RT devices and desktop’s
with Windows 7 & 8 included) and Chrome OS (mainly
low end laptops & devices). This vast array of devices
also brings up with them an equally confusing array of
browsers along with them (at the last count it was
something like 9 browsers and growing).
Faced with a similar dilemma, we take refuge in the
well-trodden path of checking on the search engines for
a Web Accessible Mobile Testing Tool (applications and
browser based systems), that would be able to serve our
purpose and not cause the management to jump from
their warm seats when they finally receive the bill for
the services used to test their precious new mobile
application or website.
To get through all this, I listed down a few of the up and
coming offerings which provide a good cross-section of
the devices and are reasonable in costs. Although these
cloud platforms do provide a service which is extreme-
ly useful, keep in mind, you might need to make use of
physically handling the device to run certain tests,
which cannot be checked with automation (but there
are ways and means to handle this, so don’t be disheart-
ened). A hybrid cloud installation in these situations
can be one of the simpler solutions which come handy.
A hybrid cloud provides a small subset of devices at
your physical location and the wider variety at a remote
location. Of the Could Mobile Test providers, the most
important thing to look into is if they provide you with
storage for your tests and a way to run the automated
tests that you have so painstakingly created for your
application from within the cloud infrastructure. This is
along with the use of making sure that these tests run
www.TestingCircus.com March 2014 - 23 -
- Gagneet Singh
Mobile Application Testing
Using the Cloud Infrastructure
over a wide variety of network speeds simulated to
provide the 2G, 3G, 4G and Wireless speeds prevalent in
the World Wide Networks around the world.
Practically a Cloud Based system should be able to cater
for most of the requirements outlined above; and the
good news is that most of them do exactly that. A few of
the prominent ones which I have used recently and the
ones which come to my mind are: pCloudy.com,
appurify.com, perfectomobile.com, HP mobile testing
lab, to name just a few of the upcoming entrants in this
field, where by Gartner estimates, there are 5.6 billion
handsets present. pCloudy.com has stood out as an ex-
cellent upcoming product, being run by engineers who
have previously worked with Nokia and other top mo-
bile hardware development companies. They have built
the framework for testing on mobile devices from the
ground up and have got some really cool features which
go with it. (Disclaimer: I know one of the Co-Founders as
a colleague from my previous companies). I loved the
way they have handled the storage of tests and results on
the site itself, along with providing features such as
mock location maps, allowing the users to experience the
different states of the application in separate geographi-
cal locations. They have recently launched a feature of
having multiple browsers and simplified their launching
from within the cloud interface, making things easier for
users again. And have launched the latest Android
‘KitKat’ version with the Nexus5. All in all, an excellent
package to go for, with reasonable rates.
All said, the main purpose of having a cloud based
mobile test experience is important for any company
wanting to launch its web presence these days. With the
advent of HTML5 and other technologies like Founda-
tion (more on this later), the web has become a place
where people love a responsive site (or application
across iOS, Chrome OS and Windows [Phone] 8) that
caters for whatever device they are working on, and does
not have the staid look and feel when they change from
a Desktop -> Laptop -> Handheld Tablet -> Smartphone.
They want to get the feel that the web site developer /
organization has done their homework and provided
them with a site where they do not have to pinch-in and
out, just to read content. With the expansion of Smart-
phone markets in the developing nations and organiza-
tions wanting to tap into the ‘billions’ of people there to
advance their products, it has become imperative for
these organizations to go through the process of Mobile
Testing their web sites and mobile based applications
across various low end and high end devices to be useful.
Sites like pCloudy.com, appurify.com, etc. are making it
easier and faster to send out mobile products into the
market by providing the required platform at a fraction
of the cost of actually acquiring a complete mobile test
lab. That said, you would still need to work out a small
subset of your tests at a physical location, but I am sure
that this would also become possible over the cloud.
With the advances these start-ups are making with tech-
nology and improvements on their own feature sets. I
certainly would be looking forward to features which
provide a friendly interface and let me run across multi-
ple devices; and for that the Mobile Test on Cloud pro-
viders are definitely a better option than trying to keep
up with the influx of the hardware been thrown into the
market by all the top end and low end hardware manu-
facturers around the world.
Gagneet Singh has been
working in the Quality
Assurance/Test field for
the past 8 years (with an
additional 4 years in Sys-
tem Tools development)
and has been involved
with companies such as
Toshiba, Adobe (Macro-
media) , McAfee, Oracle, Yahoo! and recently Mi-
crosoft.
He likes to blog and to write about the experiences
he faced in the various organizations and situa-
tions. His work has mostly been with Automation
Testing, along with Performance QA. Also, Secu-
rity testing over the much hyped “Cloud Comput-
ing” (using Hadoop and Azure) has figured in his
work area.
Currently working out of this place they call the
“Down Under“, where he lives in Sydney, New
South Wales!
www.TestingCircus.com March 2014 - 24 -
I want to define what I mean by ‘late’ and ‘early’ and
then go on to give some context from a firm I once
worked for – let’s give them the fictitious name “Lin-
man Manufacturing” for convenience. If I don’t define
my terms, we will all get lost. I will use ‘early’ and ‘late’
in an almost intuitive way, although perhaps not as
everyone uses them. They are not of the Humpty
Dumpty variety (Alice Through the Looking Glass by
Lewis Carroll: 'When I use a word,’ Humpty Dumpty
said in rather a scornful tone, ‘it means just what I
choose it to mean — neither more nor less.’).
So, for me ‘early’ is before we were ready, and ‘late’ is
after the planned date. These are pragmatic terms – let’s
not get argue about them now. My hope is that you will
see the usefulness for yourself. To start with, I need to
say something about project deadlines. First of all, it is
not testers (or test managers) who determine when a
work product is implemented (into PROD, obviously).
Test resources can speak into, or provide information,
that helps others make that kind of decision, but the
final decision lies outside “testing”. Secondly, we per-
haps all recognize that if the decision were left to tes-
ters, work products would not get implemented. We
testers are a pessimistic breed, and tend to focus on
things that don’t work. It is not always the 390 require-
ments that work that get our attention, but the 1 that
does not. We concentrate on that.
Lastly on deadlines, some deadlines are more impor-
tant than others. Take the day I am writing this article.
‘Tomorrow’ I am preaching at a local church, and some-
time before standing up, I need to have prepared what
I am going to say. That is an immoveable deadline.
Similarly, the deadline to ‘post’ this article is midnight
tomorrow. If the editor gets it by then, it will be consid-
ered for the next edition. Otherwise, it will be a candi-
date for the edition that follows. This is another
deadline that has significant consequences if it is
missed. Some software projects have hard-wired dead-
lines that cannot be missed. Perhaps there are legisla-
tion changes that need to be complied with. The
introduction of the Euro is a good example. However,
even when there is a hard (i.e. cannot be moved) dead-
line, there is sometimes wriggle-room around the edg-
es. Some parts of company accounting in Euro-zone
countries needed immediate changes when the Euro
was introduced. Other parts did not – activities around
the end of the company financial year may not have
been needed to be correct on Euro day-1 – only at the
end of the company financial year. So even where there
is a hard deadline, some parts may not need to be ready
on the first day, whilst other parts must be available,
and be seen to be working.
Right. The pre-amble is over. I want to tell you about
my time at “Linman Manufacturing”. In my many
months working for them, there were 8 major imple-
mentations, where significant new functionality was
introduced. Although each implementation was sepa-
rate, there were both implicit and explicit points of
interaction; data was loaded into and extracted from
the same database, with some common database tables
used. As time progressed, later functionality relied up-
on the data that had been introduced in earlier phases,
and was loaded month-on-month. I want to concentrate
on the last 4 major deliverables, giving the targeted
delivery date into PROD and the actual date.
www.TestingCircus.com March 2014 - 25 -
- Peter Morgan
Better to Implement Late
Than Early
How Quicker can mean Slower
I mentioned that there 8 major implementations. Of
course, there were other times that software was promot-
ed into the PROD environment. User-requested changes,
software platform upgrades, and system tweaking oc-
curred throughout this time frame. But if the project
team as a whole had been asked before each phase tabled
above, for phases F and G the answer would have been
‘yes’, and for phases E and H ‘no’. In the time that was
given, the best testing, based upon risk, had been carried
out. But there were huge swathes where the answer at
best was “we don’t know”, and some parts were clearly
not working. The architecture was basically to take mul-
tiple-format files on a regular basis (daily / weekly /
monthly) and to load this into a standard star schema
data warehouse, and then extract the summary informa-
tion. For phase H, data had been successfully loaded
many times, but the reporting layer only developed in
outline. Now it is not until data is output that it is possi-
ble to tell whether it was loaded successfully (i.e. it is fit
for purpose). And so it proved to be.
For phase E, there were immediately amendments re-
quired, both to data loading routines and to the report-
ing layer. Seven weeks later (and after seven
implementations), the solution was stabilizing. Let me
tell you, those seven weeks were interesting. Some of the
input processes could not be run for part of that time,
meaning that reported information was only partially
correct, and it was not until nine weeks after the date that
users could gain any confidence in the output. January
2009 was the first time that month-end reconciliation (for
December 2008) could be attempted. That is quite a long
time after the actual implementation date.
If anything, the situation was worse with phase H, and
interestingly enough from the table above, this phase
was both ‘late’ (after the targeted delivery date) and
‘early’ (implemented before it was ready). The first
weeks were spent loading a backlog of data, with 10
months of data to load immediately. The reporting layer
quickly highlighted ‘problems’ with 7 of the 16 data
feeds – problems that could only be rectified by remov-
ing any data loaded to date, and either re-engineering
the data creation process (before it was made available to
load), or changing the data load procedures. There were
times that it seemed data was being backed out faster
than it was loaded – even though once a data feed was
identified as ‘wrong’, nothing else was loaded for that
feed until the problem was resolved. A big score-board
showed what data had been loaded for which feeds on a
month-by-month basis. March 2010 (when data for Feb-
ruary 2010 was loaded) was the first month-end with
stability.
There are business reasons for ensuring that “something
is delivered”, and that progress is seen to be made.
However, for “Linman Manufacturing” the information
loaded into the Data Warehouse was for long-term stra-
tegic planning, not for the day-to-day business opera-
tion. There were, in one sense, no unmovable project
deadlines.
If you have a choice, never implement a software solu-
tion ‘early’. Doing so makes for a frenetic period of
activity just after the implementation date, and some-
times aborted product launches. If you take longer, it can
mean that the software is available earlier. When that
happens, staff can be truly released to go onto the next
project without constant drag-back from activities that
should have been put to bed!
To do otherwise means that quicker is slower.
www.TestingCircus.com March 2014 - 26 -
Peter Morgan is a testing
professional who has been
involved in the ICT indus-
try for more than 30 years,
and worked in the free-
lance marketplace for
much of that time. His time
has sometimes moved from testing to ‘develop-
ment’, but he would add “always using the mind-
set of a tester”. He is passionate about testing and
a firm advocate of testing qualifications. An en-
thusiastic speaker and author, Peter tries to base
his output on hands-on experience, attempting to
relate fine sounding ideas back to how it will
affect Joe or Jane Tester in their everyday working
lives in the war of attrition that we call software
testing. He is a regular at EuroSTAR conferences,
and is speaking at Belgium Testing Days in 2014 –
the forth year in a row. At this time of life, Peter
offers experience, and can sometimes say when
offered a tricky problem: “Been there, done that,
and here are 2 or 3 options that may, just may,
work in this situation”. He continues to learn,
adding technical skills to his impressive range or
hardware / software / business sector portfolio.
I don’t know if anyone has noticed but the world we
live in now is different from the world we used to live
in just 5 years ago. For me, the change that occurred is
far rapid than anticipated, because if we refer this to a
20 or even 10 year past leap then most of the skeptics
would say “Why not!, it should have changed!” but the
shortness and the speed of this period leaves no room
for being skeptic or naïve – We simply need to
understand and adapt!
Mobility has broken free from just being a device to
communicate into an instinct.
We are using our hand held devices in so many different
dimensions other than it was meant to be that it is
changing the complete concept of computing and
networking. Hand held devices are now our nodes to
the clouds; we are now the new form of terminals; “The
Human Terminals”; the value to the word “Touch” has
enhanced itself from Touch screen, and then within a
very short time it again reboots itself to “Human
Senses”! It feels like a history in making where the
systems which are categorized as something purely
mechanical and electrical combination of components
with mix of complex logics and design are re-emerging
to sapiens, like an event in space and time; As I see it,
the systems are reaching to our senses, our beliefs, daily
lives and simply becoming an extension to our self
being, like an implanted new body part. Gadgets are
now not need to be held in hands; they are wearable,
and soon they will represent a mere extension to its
users.
The human mind is now adapting these effects; see for
example our children; how they have adapted the use
of touch screen, games and the use of smart phones.
How people are getting aware of the communication
and sharing bounds and exploits over social networks;
Where to “say and share” what is now dependent on
“where and how” you are registered as a user; Twitters,
Facebook, LinkedIn, Instagram, Four Square, Google,
pint it, flicker and so on. We are not structured anymore;
we are living multiple lives under multiple scenarios.
Somewhat, this is not based entirely on our choices, this
is how the environment around is “turning out to be”
– and we have to live it!
We are now thinking! Or in other more adaptable
senses we are now becoming Context Driven!
These factors are effecting what we used to know as the
solutions and systems; these factors are now playing a
major role in provisions of the right solutions to the core
requirements and needs of the users. And… the same
factors are affecting how we “Test” these systems;
In a way, we are now moving to an age where due to
the magnitude of the contextual effect on any scenario,
generic approaches will simply fall apart and fail! – In
so many words, we need specialists! We need
empowered human beings and their expertise to
address the right issues at the right time; we need to
identify the right coverage to find the important bugs,
and mind you – We need to identify ALL OF THEM!
Under several scenarios and contextual situations and
due to certain trendy and cultural bonds we have not
yet succeeded in creating our own identities as Testers.
We are still based on the “Types” and “Approaches”
scenario. Where, Testers when tend to define what they
do, start explaining the testing types and the approaches
they follow. They tend to list down several technical
tools and programming languages as skills, and not as
“Tools” to support testing. In this stream of self-
www.TestingCircus.com March 2014 - 27 -
- Arslan Ali
7 Types of Testers
What is Your Identity?
discovery they forget the very scarlet thread they cross
from being a “Tester” to “being” a Coder.
Well don’t worry we have worked that out as well;
James Bach has already published twice about the types of
testers there are after receiving several queries regarding the
latter; in his recent blog post, he mentioned the following as
the testers’ types:
(I am quoting him here, rather putting in my own words –
it is better that way)
(SEVEN KINDS OF TESTERS)
· Automated? Manual? There is no such thing as
manual or automated testing. It’s all just testing.
Testing is often supported by tools that attempt
to simulate user interaction with the system.
This is what people call “test automation” even
though it is only automating a crude
approximation of one aspect of testing. If you
have the ambition to be a one-man test team, it
is extremely valuable to learn how to make your
own tools.
· Exploratory? Scripted? There is no such thing as
an exploratory or scripted tester. All good
testing is exploratory to some degree and
scripted to some degree.
· Tester. This is a testing generalist who can
contribute to any test team. Sometimes called a
QA analyst, QA engineer, or test engineer. I
prefer the simplicity of “tester.”
· Omega Tester. The omega tester (which I
sometimes call a test jumper, after the analogy
of a paratrooper) is one who can do anything.
An omega tester is equipped to be the only tester
in a project team, if necessary. Omega testers can
lead testing, or work with a team of other testers.
I am an omega tester. I aspire to be a good one.
· Performance Tester. The performance tester
understands the mathematics and dynamics of
the performance of large-scale systems. They use
tools that create high loads and measure the
performance envelope of systems as they scale
up. Performance testers often don’t think of
themselves as testers.
· Usability Tester. The usability tester is a bit
mythical. I have met only two dedicated
usability testers in my entire career, but I have
seen more of them from a distance. A usability
tester specializes in studying how users feel
about using and learning a product.
· Security Tester. Security testers also often don’t
think of themselves as testers. Security is an
exciting, specialized form of testing that requires
the mastery of a great many facts about a great
many technologies.
· Testing Toolsmith. A testing Toolsmith is a
programmer dedicated to writing and
maintaining tools that help testers. This is what
a lot of people would call an “automated tester”
but you better not use that term around me.
· “SDET” Software Development Engineer in
Test. This means a full on programmer who does
testing using his programming skills.
On the other hand there are several characteristics as a
tester which can stood up and create an exclusive tester
identity; For example, boundary Testing heuristics; now
I know that several of us have used this term in our CVs
and have somehow read or taken part in forums to talk
about this type testing; but how many of us have
actually bifurcated this into “Galumphing”, or
“Steeplechase” or even “Leap and Creep” - I don’t think
any of us!
OR
While discussing about product coverage we have used
the term “San Francisco” depot instead of discussing
the missing out requirements, and blaming the
developers of not providing the right specs or cursing
the Implementation team of not discussing the right
requirements; that is what common professional does!
Why not stand out and use your own language?
I have seen professional testers asking questions and
worrying about “Testing without Requirements” or
“Finding the right bugs” within a short period of time,
but I have not seen any of them discussing about using
or creating their own Heuristics? Why not?
There is also no denial in discussing Quality and its
criteria, but have we ever tried to use and educate
ourselves with the use of the “Quality Criteria Heuristic
called “FEW-HICCUPPS”? Try and do that!
www.TestingCircus.com March 2014 - 28 -
Let us understand now.
Why can’t we move beyond being a technological freak
to a Tester in a meaningful term; Does the mention of
the very word “Heuristic” is not a mention of “Tool” in
your sense. Technology is a wonderful thing, being
technical is in the very genes of a computer professional;
but as testers we need to identify ourselves as someone
with a niche of being a tester and being technical at the
same time; Being technical comes with the package; So
why not mention who you really are? “A Tester”!
Empower yourself, your language, attitude and
reputation as tester. Learn Un-Learn and the re-learn!
Try and improve your language as testers, differentiate
yourself from the peers of Developers and
Implementers. Yet sewed in and work for their support
Or I can only say this then; you are in the wrong game
baby!
Arslan Ali has more than
14 years of Experience
related to IT, Industry and
Training Institutions with
exclusive experience of 5
years in teaching various
disciplines and projects in
IT Institution. He has worked in various roles in
capacity of Software Engineering, Software
Tester, Trainer and Quality Assurance Roles. The
Major focus of his expertise lies in Coordination,
Implementation and Testing of ERPs and
Customized Applications. He is also a trainer for
Context Driven Testing for various companies
and individuals.
Arslan is currently working at Sidat Hyder
Morshed Associates (www.sidathyder.com.pk)
as a “Sr. Consultant – Information Solutions; but
beside that he is also an active founding member
of TestersTestified (www.testerstestified.com)
(@testtified), Outtabox! (www.outtabox.co)
(@OuttaBoxPk) and OISOL – Open Integrated
Solutions (www.oisol.com) as a training
consultant for Software Testing and Context
Driven Testing Workshops.
You can follow him on twitter @arslan0644 and
on LinkedIn at
pk.linkedin.com/in/thegoodchanges/
www.TestingCircus.com March 2014 - 29 -
www.stepinforum.org www.qsitglobal.com
Interaction » Insights » Opportunities
11th
International Software Testing Conference
Today’s tester is a unique being, driven, not by deadlines and mundane tasks, but by a relentless curiosity and
a fascination with possibilities. Awaken the tester in you.This is where you want to be; this is NOW. Walk away
with exciting ideas and quality insights from other industries. Make your mark by challenging the existing
paradigm.
Most Popular Conference among Start-Ups to Fortune 1000 Companies
ExcitedtoparticipateasaSPEAKER/ DELEGATE/SPONSOR?
writetonazreen@stepinforum.org
orcall+91-80-40818888
Over 900+ delegates from India and abroad
attended in STeP-IN SUMMIT in 2013
Testing NOW
STeP-IN
SUMMIT2014
Asia Pacific’s Largest
SoftwareTesting Conference
Produced by: Hosted by:
JUNE 25-27, 2014 AT BANGALORE &
JUNE 18 & 19, 2014 AT PUNE & HYDERABAD
CONFERENCEPROGRAM
View Past Conferences View Past Speakers’Gallery
www.stepinsummit.stepinforum.org
Pre-conferencetutorialworkshops(twoparalleltracks)@Pune,HyderabadandBangalore
TargetAudience:TestLeads,TestArchitects,TestManagers,TestAnalysistsandConsultants
Conference@Bangalore
	 TargetAudience:CXOs,VicePresidents,Directors,BusinessHeads;Customersandend-usersofITproductsandservices;
	 TestManagementprofessionals–Managers,TestLeads,Software,Test Engineers;Quality/SEPGManagement		
	 Professionals–Managers,QALeads,SoftwareQAEngineers; ProductDevelopmentManagers;ChangeAgents
18,19,25
June,2014
26-27
June,2014
BOOK
WORM’S
CORNER
March is the beginning of the appraisal season; that’s why 50% of the
organization actually wonder if they should be continuing in the or-
ganization or not. That’s why I decided to crawl through “Don’t Hire
the Best”.
This book is written by Abhijit Bhaduri. In case you are wondering if
it is a misguide to hiring the right team, then you are partially incor-
rect. This book preaches and lets you come to your own conclusions.
The book tries to sell itself by lecturing about good hires and how to
make good hires, however, it tries to draw the right line between
“hiring right” and “hiring the best”. Unlike many other interviewing
books that are filled with choc-a-bloc aptitude and problem solving
questions, this book also tries to examine the organization culture and
why culture fitment is important for the company. The case studies
with the standard disclaimer of having been altered to protect identi-
ty is very useful and causes the reader to ponder as to how he could
have done things better. Reasons as to why it is important to create a
fitment between the interviewee personality and the management ex-
pectations, individual traits that need measurement, why making the
wrong hire call can result in financial losses are stories talked about in
depth in this book.
What did it teach me? After reading the book, I ensure that I pass on
a set of good and bad candidates to my interviewing team; that varia-
tion causes them to be on their toes and interview without bias. If I
send only good candidates their way, a form of bias sets in over the time which would result in all of us overlooking
the most obvious reason for not hiring the candidate.
Why should you buy this book? Because it’s not free; in addition to that, you will learn valuable lessons on how to
create a team, why the culture fitment is important for your teams and why you need to ask questions beyond the
role competencies while forming teams. When is it important to look into a resume to know what he’s capable of and
when should you look beyond the resume? You will ask yourselves these questions if you were to read this book.
I hope, after reading the book, you don’t hire the best anymore; you will hire “better than the best”.
Love,
WoBo
www.TestingCircus.com March 2014 - 31 -
Request a free demo by sending us an email at support@sahi.co.in
http://www.sahi.co.in
Become our fan -
https://twitter.com/_sahi
http://www.facebook.com/sahi.software
www.TestingCircus.com March 2014 - 32 -
Part 39
www.TestingCircus.com March 2014 - 33 -
http://www.testingcircus.com/category/a-fake-testers-diary/
Random Musings
My 39th month with this magazine; that's when I real-
ized that I had inadvertently become a career counsel-
lor for most of my family members who were in their
final year of college. Is there any scope for "Java"?
Should I learn ".NET"? Is the industry scope for "SQL"
or "Oracle"? With so many questions throw at me from
family members, I've stopped visiting any family func-
tions or any event where I get to meet my extended
family. Quoting client-meetings and a workaholic
manager asking for meetings at mid-night, I manage to
escape attending most of these functions. And then
there is that irritating cousin of mine who keeps brag-
ging of his topper credentials in schools and constantly
asks me if there's more scope for "development" or
"testing"? I have a good mind to tell him that "If people
like you join Development teams, then testing career
will have a lot of scope", but I dare not.
Why do I write this column? --- "When a tester logs a
bug and engages the developer in a sadistic discussion
asking him to fix the bug, the tester is also torturing
himself on some levels; the tester's life is such that he
needs such discussions to survive his job" --- This is
what a famous manager had to say about corporate
culture for testers. Though there has been some
amount of hue and cry against this column, there are a
few instances that agree that this column unearths
what lies beneath. The real thought of documenting
this diary is to try and unmask not just the bad guys,
but also the horrid practices that have created fake
testers all over the place. I also fell prey to fake testing
sometime back and believed in the concept of sanity
testing, smoke testing, BVT testing and that all of them
were different testing types; I also thought that I could
become the greatest tester if I cleared all those exams.
And now, I stand before you, as the FAKE TESTER.
That's what I stand today. And I hope my writings will
stop creating more fake testers.
So far, in most columns, I have (unsuccessfully) tried to
provide an insight as to what's wrong with today's
testers. Most of them can be categorized as
· the inability to find good and authentic testing
teachers
· how there are some people sitting in manage-
ment who actually decide and state what test-
ing is all about and the entire company thinks
that's what testing is all about
· the low pay of the testing engineer -- how many
of them do you think have equal opportunities
at growth and how many of them have 40 hour
working weeks?
· And the inability of the testing team in any
company to function as an independent body,
but function as a sub-body of another manage-
ment function (development or support or mar-
keting or sales or .....) there is no independent
person or body leading the testing vision for the
companies.
And at the end of a year, many testers realize the
disdain that others have for him; the disdain becomes
reality. When this is pointed out to the management,
the management don't try to make good the wrong
things; they do some actions that makes the tester's life
from bad to worse.
For a long time, the sole objective of most testing com-
panies seems to be a single agenda --- bill the client
against the number of test cases executed; to showcase
smartness, they record and replay some of the testing
and quote savings in the name of automation. The
clients cannot back out due to the legal agreements
signed at the beginning. An impact of this is that the
test results for periodic testing that's being sent to
clients cease to matter; and regardless of the frequency
of app usage by customers, testing companies continue
this trend. 1 look at the testing economy would tell you
that 2.5 Billion Dollars was the money given to off-
shoring companies for testing alone in the 3rd quarter
of 2013. With new testing companies sprouting all
over, this is set double in the next few years. And the
testing companies have also started to exploit this.
With substandard test cases and haphazard testing,
they make hefty profits without providing any value
by testing.
An indicator of the lack of quality can be seen by the
most unimportant projects being handed to the
off-shoring testing companies; this can be seen by the
very fact that in most of the offshore company cases,
not much of business revenue would be lost if the
offshore projects are closed down. Secondly, according
to the reports of our special correspondent, in the last 5
months, there has been ZERO new product launches in
non-Asian markets due to testing done from here.
I am unsure if others realize this, but looks like India's
testing industry has started to face its worst crisis; my
personal thought is that the rest of this year will be
gone in people realizing their mistakes and the indus-
try auto-correcting itself. What do you think will hap-
pen? As I mentioned last month, only time will tell.
www.TestingCircus.com March 2014 - 34 -
www.TestingCircus.com March 2014 - 35 -
Anna Royzman
Context-Driven Scholar
https://twitter.com/QA_nna
Sharath Byregowda
A passionate software test professional, Principal Consultant at
SQS, London in their agile group.
https://twitter.com/sharathb
Mike Lyles
International speaker, writer & Sr QA Manager in Performance
Testing, Test Automation & Service Virtualization (20+ yrs IT).
https://twitter.com/mikelyles
Joris Meerts
Publishing the latest news from the software testing weblogs.
Keeper of lists. Testing historian, critic, thinker, member of
#DEWT. #Capgemini
https://twitter.com/testingref
http://Twitter.com/TestingCircus
#Testers2Follow
www.TestingCircus.com March 2014 - 36 -
#FollowUsAtTwitter
http://Twitter.com/TestingCircus
Testing Circus
Testing Circus is a world’s leading English language magazine for software testers
and test enthusiasts. Monthly editions since September 2010. #testing
Follow us at https://twitter.com/TestingCircus
Part 15
- Santhosh Tuppad
Santhosh Tuppad specializes in exploratory testing approach and his core
interests are security, usability and accessibility amidst other quality criteria.
Santhosh loves writing and he has a blog http://tuppad.com/blog. He has also
authored several articles and crash courses in the past. He attends conferences
and confers with testers he meets. Santhosh is known for his skills in testing
and has won the uTest Top Tester of the Year 2010 apart from winning several
testing competitions from uTest and Zappers.
www.TestingCircus.com March 2014 - 37 -
Security Testing Tips
One of the challenges in security testing is, setting up
the test environment. If you are a small scale organiza-
tion and there are no lots of processes, then it could be
easy for you to go ahead and setup a test environment
like you want, but in the bigger organizations when it
has lots of processes it could be hard. There could be
network related blockers that you may want to clear
(Example: You want to download a software which
you want to use for security testing purpose, and that
source of download is blocked by the network of the
organization). If your test environment is a blocker for
you, then I would better recommend to not performing
security testing and thereby, you can at least save costs.
Isolated network of computers
It is important to have a separate network dedicated for
security testing. This is because, you do not want to
affect the other computers on the network if you down-
load some software and it is infected with malware /
adware or any malicious thing.
No website blockers
The network shouldn’t be blocking any website that
you want to browse for learning about some hacks or
downloading any software to aid your hacking activi-
ty. Let your network policy doesn’t end up blocking
your learning.
Administrator rights for computers & other devices
In my experience, I have faced lot of blockers when the
computer that was given to me did not provide admin-
istrative rights. For changing some of the settings in
order test for security, I had to e-mail the infrastructure
team to change the settings and that consumed time.
Before starting, it is important to ask for a computer
with all the rights to change / modify any setting.
Installations of software before commencement
Make sure that you are ready with installations of all
required software before you commence security test-
ing activity. This is because, it will save you time if
something doesn’t get installed or doesn’t work prop-
erly while you are testing for security. So, installing the
required software like proxy, burp suite, Wireshark,
backtrack, kali linux, mantra browser, nmap and lot of
many other tools happens without any hassles.
No code changes to be done
When testers are testing for security, no code changes
should be done. It is important that you have separate
environment and not the same which developer uses
check in his / her code. And also, security testing needs
to be done once functional testing is done and all the
bugs reported are fixed.
DIY: Test Environment for Security Testing
http://www.facebook.com/TestingCircus
www.TestingCircus.com March 2014 - 38 -
Visit our facebook
page to share.
Someone else too
could enjoy what
you're reading
right now.
Since this is my first article for Testing Circus I’d like to thank all the wonderful contributors for sharing info,
because there’s no other way to evolve in the IT industry!
Why would we need to run tests on a Remote VM?
You don’t actually need to do this, but from my own experience I’ve isolated a couple of scenarios where this can
prove vital:
a. Having a test-script (or suite) running on a JavaScript-rich page that needs to have complete focus all the
time. So, no touching the mouse or keyboard while this is running. If this takes 1hr to run, you’re going to
get bored and/or sleepy and let’s not forget you have to justify for that particular hour to your management.
b. The VM can run on your local machine (making it not that “remote”), it can run on another machine
lodged on a cloud-hosting server (e.g. RackSpace) or in your blade-center sitting neatly in your compa-
ny’s basement. Unless the VM is running from your computer’s VBox you’re hogging way less resources by
running this remotely.
c. You might be running across situations where your script runs smooth on your local MacOS / Windows
environment, but, for some weird reason you get weird failures on the CI machine running (in most cases) some
Linux distribution. How will you debug this?
Where to start? VBox setup and install
I’m going to run through the steps needed to build your local Ubuntu VM assuming you have a working Host
Machine (I’m currently running Mac OS 10.8.5, but this is 99% valid for Windows as well)
a. Download and install VBox from Oracle’s website
https://www.virtualbox.org/wiki/Downloads
b. Download Ubuntu. I went for 13.04 64bit but you can choose whatever suits you
http://www.ubuntu.com/download/desktop
c. Start VirtualBox
d. Click the NEW pictogram in the left
e. Populate the fields according to your need
www.TestingCircus.com March 2014 - 39 -
- Mihai Sarlea
Running Remote Selenium 2
WebDriver Scripts On a VM
www.TestingCircus.com March 2014 - 40 -
www.TestingCircus.com March 2014 - 41 -
www.TestingCircus.com March 2014 - 42-
f. From here on it’s a standard Ubuntu installation the likes of which you can find at
http://www.ubuntu.com/download/desktop/install-desktop-latest
Configure Ubuntu
I’m just going to run through the basics of Ubuntu configuration so you can get your VM up and running ASAP.
a. Install OpenJDK-7 and Maven
Since I’m using Java to run my Selenium tests, I’ll need to install the Java Development Kit and Maven (in
my case) and I’m willing to share this nice trick useful also when setting up a fresh CI based on Ubuntu:
This is the sequence that will get you a clean usable OpenJDK on a Deb machine:
apt-get remove openjdk*
apt-get remove java*
apt-get install openjdk-7-jdk
apt-get install openjdk-7-*
apt-get install maven
b. Test the install
java –version
mvn -v
c. Check the IP address and the “visibility” of your host computer
ping your_host_computer_ip
www.TestingCircus.com March 2014 - 43 -
Here’s how I’ve setup the VM in VBox and Ubuntu:
Now you should be able to ping to and from the VBox installed Ubuntu VM.
www.TestingCircus.com March 2014 - 44 -
Setting up SELENIUM 2 on the VM
a. Open up a Terminal window and download the
Selenium Server Standalone jar
wget
http://selenium.googlecode.com/files/selenium-
server-standalone-2.35.0.jar
b. What we’re doing is creating a small Selenium 2
GRID network and that’s why we need to start
up a hub here:
java -jar selenium-server-standalone-2.35.0.jar
-role hub
c. Open up a 2nd Termnal window and setup a
(single) basic node with default config on the
same machine:
java -jar selenium-server-standalone-2.35.0.jar
-role node -hub
http://localhost:4444/grid/register
d. Find your VM’s IP address – open up another
Terminal window:
Ifconfig -a
That’s it, all setup!
Adding the VM to your Selenium WebDriver Java
Framework
This is as easy as adding the lines bellow:
switch (useThisDriver) {
case FIREFOX:
aDriver = new
FirefoxDriver();//profile);
break;
case
FIREFOX_LOCAL_VM_UBUNTU:
DesiredCapabilities capability =
DesiredCapabilities.firefox();
aDriver = new
RemoteWebDriver(new
URL("http://192.168.10.26:4444/wd/hub"), capa-
bility);
break;
DesiredCapabilities capability =
DesiredCapabilities.firefox();
driver = new RemoteWebDriver(new
URL("http://192.168.1.138:4444/wd/hub"),
capability);
driver.findElement(…) // whatever you want to do
If you’re not running the test as part of a framework, just
go ahead and add the lines like this when starting the
test:
Conclusion
I’ve tried to go through every step of setting up a nifty
lightweight GRID 2 on a local (or remote) Ubuntu VM.
The knowledge is applicable, in much the same way to
a Windows VM or any other OS you might need, for the
matter. I hope it helps you get up and running with
your local *nix VM.
Selenium GRID 2 is a very cool feature that can save a
lot of time, but we’ll dive more into this in a later arti-
cle. Till that time, feel free to check out the documenta-
tion Google put up at:
https://code.google.com/p/selenium/wiki/Grid2
More on the framework “Driver” class I used at point 6:
http://blog.cloudtroopers.com/content/%E2%80%9Cdri
ver%E2%80%9D-class-selenium-2-webdriver
Mihai Șarlea is an SQA Auto‑
mation Engineer for
CloudTroopers.com and
sound engineering enthusi-
ast.
Telecom Engineer by forma-
tion, Mihai started flying
above radar while working
as Simlock Security EU region Specialist for Nokia
and, gradually, shifted gears towards QA-ing for
Endava Romania on financial projects. For the last
year and a half, he's been focusing on automated
website testing, continuous integration, light-
weight performance scripts and all adjacent tasks
at CloudTroopers.com.
Outside office hours, Mihai occasionally handles
Front of House Engineering for various clubs in
Cluj-Napoca, Romania and volunteers to help the
less fortunate artists record decent, career kick-
starter, materials.
Feel free to check out his LinkedIn profile:
http://ro.linkedin.com/pub/mihai-sarlea/40/924/b92
www.TestingCircus.com March 2014 - 45 -
Thinking what to do with your career?
We can HELP
www.TalentPlusPlus.com
www.TestingCircus.com March 2014 - 46 -
Tools Journal Testing Corner
Testing Circus in exclusive
partnership with Tools Journal
(http://toolsjournal.com) presents
the Tools Journal Testing Corner.
1. Test Automation: New Venues,
New Challenges
2. SmartBear TestComplete 10
Brings In Support For
Automated Mobile Testing
3. Squish Coco Brings In Support
For Code Coverage Of C# & Tcl
Code
"Thank You" to all subscribers who have joined us
and have been giving us some good feedback. Most
of all Thank you "Testing Circus" for providing us
a good platform to shout our views .
www.TestingCircus.com March 2014 - 47 -
About Us:
A start up journal and aspiring social
community with an aim to gain and
distribute knowledge on software
tools and concepts in Testing, Agile,
Cloud, Mobile and Enterprise
Integration.
http://www.toolsjournal.com
With over 500 products listed with
quality articles, product owner
interviews, we are moving swiftly to
launch product editorial/user
reviews, community module in next
2 months.
Connect With Us
@toolsjournal
www.toolsjournal.com
www.facebook.com/toolsjournal
Thank You
Testing Circus & Tools Journal
While most of the test automation engineers are aware
of the many pitfalls of test automation; there are some
associated to the automation planning & design
considerations. There are some widely used automation
frameworks, & hybrid framework tops them all.
While it’s a good practice to use hybrid framework with
benefits taken from all the conventional frameworks
like keyword, data driven etc…; the real trick in coming
up with a good design is to cater to all the project needs
while keeping the maintenance cost low, and the
learning curve short.
While there is no single way to implement this solution
but again, it’s no rocket science either. One may have to
consider various factors that automation framework has
to handle – technical & non-technical. On technical side
one may have to consider the level of complexity,
abstraction-levels, so that the front-end whether it’s a
GUI or an excel sheet needs to be easy to use & intuitive
at the same time.
While one should use excel file for easy readability,
overusing it may lead to slow execution – contradicting
the very reason automation is used e.g. keyword-driven
framework using excel file has its benefits, it makes
execution bit slower, each read and write to-from excel
file may add up to huge delays by the time automation
suite is completed. Extensive use of excel or any other
file format may lead to bulky automation leading to
slow execution & having its own weight to pull it down.
While it is good idea to give maximum control (if not
all) to the executor to manipulate suite content , it
requires good judgmental call on the skill set of executor
, reusability & degree of flexibility that needs to be
provided so that its benefits are maximized.
Though there have been good amount of focus given
lately on application performance, automation
performance is another area which is unexplored but
said that, doesn’t mean we can overlook the automation
execution speeds.
· Terminology like Library-architecture etc. is
something that is intrinsic to any software
project whether it is automation or
development, so it becomes really necessary,
that we form meaningful library
functions/methods, keep them light, reusable
and ensure proper error handling.
With the advent of various automation tools it becomes
really important to keep a track of its advantages &
disadvantages; before implementing the same.
· With loads of new browsers and devices
mushrooming in - Cross-browser & cross-device
testing is something which can’t be avoided. But
again each browser/device has its own
engine/mechanism to handle web elements
/objects. Taking an eye off such mechanisms can
lead to unexpected results, and automation
performance issues.
Many times failures in automation needn’t be because
of scripting or application alone, browsers also have
their own drawbacks which lead to automation script
failures. While various studies depict various statistics,
one major thing that comes up is IE seems to have many
issues when then comes to test automation, such
statistics need to be looked at before commencing
automation.
· Object identification/locator mechanisms also
play key role. Those who are well versed with
www.TestingCircus.com March 2014 - 48 -
Test Automation: New
Venues, New Challenges
automation testing, don’t have to be told that,
each mechanism has its own benefits, like one
may be robust mechanism, another may be easy
to use e.g. name, id as locators; thereby selecting
a mechanism which fits well in all the areas is
preferred.
· With client side technologies like Silverlight,
HTML5, JavaScript etc. dominating the web-
pages, it becomes really critical to take them into
consideration before committing to the client.
Though the pages may appear normal, the tool,
browser may have issues to work with. Thereby
it is always advisable to do a thorough POC,
before dwelling deeper into full-fledged
automation.
· Like in selenium, it is quite obvious to start
automation using Firefox and committing to
client on providing complete solution, what
further is required is the foresight into client’s
future requirements like cross-browser testing,
mobile device testing.
· It is always recommended to think of the
breadth during planning & design phase &
during implementation focus on individual
automation functionality, creating a self-
contained unit, preferably reusable. An
incremental approach is highly advised - each
automation component or test script needs to be
well tested on various browsers & devices before
moving to next level. This approach would
avoid surprises that come with big-bang
approach. (It is advisable to avoid big-bang
approach, wherein various automation
components are clubbed together to fulfill the
requirements. )
· With extensive usage of open source
technologies & tools (like selenium), it becomes
really important to create a proper test
environment, wherein various versions of the
each tool, process or platform is well
documented. Any change in one may lead to
automation failure.
· Integration with various other tools &/or
application/platform compatibility also is
critical, as slightest change in a version may
impact automation framework e.g. which
version of testlink (test management tool)
integrates with which version of selenium or
which version of UFT supports which browser
& version. Various configurations & versions –
inclusive of auto-updates are to be thought
about & it is really important that each step is
taken carefully to avoid last-minute surprises.
· Requirement to automate huge number of test
cases, and to get proper ROI, Distributed
testing, is fast catching up. Various machines
with various configuration, pose a major
challenge to distributed automation testing; a
single script to be executed for - cross-browser ,
cross-platform, cross-device each having their
own intricate setup (including java version, OS
version – service packs installed etc..) – each
system depending on the system below it,
creating a delicate balance for things to work :
eventually all this can add-up to unimaginable
permutation & combinations making it
mandatory to document the details.
Automation is no more limited to a single machine or a
single tool, but ranges from various tool integration to
its implementation on various machines & browsers,
posing new technological challenges to be faced with
advent of new versions, compatibility issues,
configuration issues making it mandatory to foresee all
the major unforeseen pitfalls to be addressed & risks to
be averted or mitigated, ensuring successful automation
solution delivery.
www.TestingCircus.com March 2014 - 49 -
Taking Testing to Cloud
Taking Testing to Cloud
Taking Testing to Cloud
Taking Testing to Cloud
Taking Testing to Cloud

Weitere ähnliche Inhalte

Kürzlich hochgeladen

UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 

Kürzlich hochgeladen (20)

UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 

Taking Testing to Cloud

  • 1. Your Monthly Magazine on Software Testing www.TestingCircus.com with Smita Mishra Testing Circus Volume 5 - Edition 3 - March 2014
  • 2. Article submission guidelines – ● Subject of article can be based on any area of Software Testing. If you want to publish your article on theme based subject please read our announcement of monthly theme published in our site. Article can be submitted without any theme based subject. ● There is no minimum and maximum length of article. If you feel the article is lengthy, please divide the article into logically separated parts so that we can print them in a monthly series. ● Give a meaningful title to the article. If you want a sub-title as well , then add that in a different line. ● Add images/pictures if necessary. If you are using any image/picture which is not yours own work, please include the source. Take care of copyrighted materials. Images need to sent separately with the article. ● Send us the article in MS word (doc/docx) format only. Pdf files are not accepted. ● Write a short write up on the author(s). Usually 7/8 liners in 3rd person descriptive language. ● Include photograph of author(s). Preferred in high resolution .jpeg/.png format. Ideal size would be 50mmX 50mm. ● You can send your article any time. Since we publish every month, your article can be included in any month. There is no such thing called as a dead line. ● Send in your article to editor@testingcircus.com with a subject line “Article for Testing Circus – Author Name – Title of the article” ● If you think you can write a column in Testing Circus for at least 6 months, please submit 3 articles in advance. We are open to any idea that may improve the user experience of Testing Circus. ● We will publish the articles in our website a week after the pdf magazine is published. Article Submission GuidelinesDo you have something to share with the testing world? We can make your voice heard to testers. http://www.testingcircus.com/article-submission-guidelines www.TestingCircus.com March 2014 - 02 -
  • 3. Volume 5 - Edition 3 - March 2014 Testing Circus www.TestingCircus.com March 2014 - 03 - Topic Author Page # Interview with Smita Mishra Jay Philips 5 Examples of Testable Requirements Ulrika Park 10 Taking Testing to the Cloud Vipin Jain & Anubha Jain 14 A Common Pitfall of Test Cases Raj Subramanian 22 Mobile Testing on the Cloud Gagneet Singh 23 How Quicker can mean Slower Peter Morgan 25 7 Types of Testers - What is your identity Arslan Ali 27 Book Review - “Don’t Hire the Best” WoBo 31 A Fake Tester’s Diary, Part - 39 Fake Software Tester 33 Testers to Follow Editorial Team 35 Test Environment for Security Testing Santhosh Tuppad 37 Running Webdriver Script on a VM Mihai Sarlea 39 Test Automation: New Venues, New Challenges ToolsJournal.com 48 TestComplete Support For Automated Mobile Testing ToolsJournal.com 50 Squish Coco Supports Code Coverage Of C# & Tcl Code ToolsJournal.com 52 Testing Events Around the World TestEvents.com 53 Testing Circus Team Founder & Editor – Ajoy Kumar Singha Team - Ÿ Srinivas Kadiyala Ÿ Jaijeet Pandey Ÿ Pankaj Sharma Ÿ Bharati Singha Ÿ Chanderkant Saini Ÿ Dwarika Dhish Mishra Editorial Enquiries: editor@testingcircus.com Ads and Promotions: ads@testingcircus.com Testing Circus India Chaturbhuj Niwas, 1st Floor, Sector 17C, Shukrali, Gurgaon - 122001 India. © Copyright 2010-2014. ALL RIGHTS RESERVED. Any unauthorized reprint or use of articles from this magazine is prohibited. No part of this magazine may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system without express written permission from the author / publisher. Edition Number : 42 (since September 2010) *On the Cover Page - Smita Mishra W hat is w here?
  • 4. Write to us at editor@testingcircus.com www.TestingCircus.com March 2014 - 04 - Testing Circus is three and half years old. This magazine has been running on volunteer effort, assisted by few passionate testers from time to time. It has been an amazing journey so far. But, like all other good things in life, it is coming to an end. We are announcing discontinuation of Testing Circus after this edition. It is a sad moment for all of us. Running a regular magazine purely based on volunteer effort is hard task, specially when all volunteers including me have full time jobs to look after. We were always supported by many testers by contributing some awesome articles but let me also admit that it is always a challenge to gather good articles when you don’t pay the writers and busy testers do not reply to your emails and direct messages. There was too much effort for our volunteers. Testing Circus site will remain as it is. We will continue to publish our old articles in the site. We will continue to promote good testing stuffs in Facebook/Twitter. But our regular monthly magazine will stop. I would like to thank all you of you who supported us in this wonderful journey. We will cross the roads again. Till then, stay hungry, stay foolish and test passionately. - Ajoy Kumar Singha @TestingCircus // @AjoySingha From the Keyboard of Editor
  • 5. SMITA MISHRA Smita Mishra is the founder and chief consultant at QAzone Infosystems, which is a pure-play software testing organization. She is a first generation Entrepreneur and is a Test professional who has spent over 12 years practicing testing and leading test efforts of varying sizes, cutting across all key domains and technologies. In the past, she has worked with multiple organizations, likes of - HCL Technologies Ltd, Fidelity Investments, Nucleus Software Exports Ltd, Churchill Insurance (Now RBS). In her current role, she is involved in creating test teams, managing testing for software companies, leading the overall test strategy for them. She is also engaging constantly with different forums to assist growth for women in her field and women in general too. Organization: QAZone Infosystems Pvt Ltd Current Role/Designation: Chief Test Consultant & CEO Location: New Delhi, India Interview with Testers www.TestingCircus.com March 2014 - 05 - 1. Tell us about your journey to becoming a software tester. How did it start and how this has been so far? Was it planned or by accident? My journey to become a good tester still continues. However, it began in 2001, with my first job at Nucleus Software Exports Limited, where I was campus placed after my engineering. A batch of about 20 engineers were picked from across the nation and put together as the founding (independent) test team. Honestly, I had no introduction to “Testing” until then other than the few chapters I read on SDLC, Verification and validation in a book of software engineering by “Roger Pressman”. It was planned by my employers not me but wasn’t really an accident. Once in testing, I quickly began to learn it and found it interesting. I will agree that though I really enjoyed my work, the initial years were nothing exciting. But over the years, I realized that I enjoy testing more than anything else. There have been tremendous learning through the 13 yrs spent in the field. The more I learn the more I see I have yet to learn. Hence, I can easily say that the journey so far has been terrific. 2014 seems to be extraordinarily good year for me and I am loving every moment of it, as a test professional. 2. When did you realize your passion was software testing? I would be wrong if I said that it was love at first sight. It was not. But as they say – good music grows on you even if you don’t like it much in the first go. It was similar in my case. I realized I enjoyed testing early in my work life. But it wasn’t until 2006 that I began to love it. I attended the first conference in my life – it was a QAI conference where I was presenting a practice paper on estimations. Met many testers there and I loved the feeling of belonging to a community too. Then in 2007 I went to work in GE Healthcare account under the most dynamic technology leader and sharp as razor – Gazanfar Hakeem and an excellent Test Manager Smita Sethi (who also happens to be a very sharp professional). I got exposed to very technical testing in DW-BI (ETL Testing) and performance testing of these ETLs and implementing lean methodology. I would term that as the turning point in my career on how I began to look at my testing career. I realized I was cut out to do this. 3. Do you regret being associated with software testing today? Given a chance would you move from testing to any other field in IT? * Interviewed by Jay Philips
  • 6. www.TestingCircus.com March 2014 - 06 - No, clearly no. Quietly no. Loudly no. Simply no. I am proud to be a part of an ever evolving field and that has a huge legacy of achievers. I can easily say I never had second thoughts about my work. Also, I would find it very difficult to work on a single technology or domain all my life – and to look conversely at it - which other field can give me so much exposure in understanding “what really matters” in IT field, while keeping me hands on with all latest technology. 4. What does QAZone do? How are its services different from the services offered by other similar organizations? QAZone is a pure play software testing organization that offers Testing solutions and services, Test consulting, Test training and test support services (Test data management, test environment setup). We offer Business as usual testing as well as specialized testing. We have built a client base of over 20 clients in last 3+ years. Our business focus is our key differentiator. Our testing solutions are designed to work for your business. We look at technology as a platform to make business happen. All our testing has a component of domain focus which becomes a very serious need of our clients when we are dealing with regulated environments like Aerospace and Healthcare. Our next differentiator is that against the trend - we believe in hiring locally, even if it means training potential resources at our expense. For example – anytime we need a resource in US, our first approach is to find a resource locally. This has so far proven to be a very fruitful approach. 5. Last year you organized a conference called ThinkTest in India. Was that the first year of the conference? What were some key sessions and feedback? We had planned for a conference ThinkTest 2013, which would have been the first of our initiative in this direction. We were very keen to bring James Bach to this part of India (and we still want this to happen). Unfortunately, due to unforeseen circumstances at my personal end and not having suitable alternatives, we couldn’t take this forward. We will keep the test community updated of future plans related to this. 6. In February 2014 you held the first meetup for Test Practitioner’s Club. What made you want to create this meetup? It always feels easier to work with people whom you have met and can relate to them as a face and as the vibes you share with them. All leading test conferences are still physical conferences not webinars. That’s because learning happens in many forms today and one of them is networking post conferences. With the thought of building a network of co-learners, we created a LinkedIn group – Test Practitioner’s Club and planned for its meet-ups that would help local testers to learn more from each other and get introduced to global platforms too. If we could – we would have such test meet- ups all over the country. We are planning to do one meet every month at the very least. We will keep posting the updates from our meets at my blog, for all to read. 7. I noticed that you are certified in ISTQB and QAI (CSTE). What made you decide to go for both certifications? Yes, that’s true – I have indeed done both the certifications. I went for ISTQB (in 2005) because until then I had no certifications. And under our training and development program, my organization paid for this certification. However, later on I went for QAI (CSTE), as it was not widely done so far due to its cost and students failing in it and this made it SMITA MISHRA
  • 7. www.TestingCircus.com March 2014 - 07 - appear more respected and challenging. I paid for this one myself. 8. Do you recommend that all testers get both certifications? Are there any other certifications you would recommend to other testers? I would let testers decide this for themselves. But I can say it with reasonable confidence that these certifications do not prepare testers to handle serious testing problems. I enjoy being part of miagi-do and discussing actual test issues with real testers there and how to resolve them. I have heard very good things about BBST and most of the leaders I follow are associated with it. I would suggest all testers to atleast go through the content and format and decide for themselves. If learning is the objective and certification is not really the goal, I would suggest testers to go through James Bach’s site for RST classes and RTI (online) classes. 9. According to you, what is lacking in today’s commercialized training industry, especially in testing? With all due respect to the 2 certifications I did, I believe an ideal test certification would be one which would have more practical problems to deal with, as tests for their students and would involve real life situations of testing than focusing on glossary of testing and terminologies. Also, I would invest in domain led testing certifications for my team. So far, there are no such certifications available which would certify testers for regulated environments like those of aerospace and healthcare. I have heard of a new certification that Cem Kaner has come up with. I need to look into that one too, if it has domain based testing certifications. 10. What qualities will you look for in a candidate when you want to recruit someone for software testing job? I like to work with enthusiastic people who enjoy learning new things and working on new areas. This is true not just for software testers but front office executive, accountant, HR and Admin folks - all of them. Many today do not feel the need to look at the past experience and hire only for attitude. However, when I am hiring testers particularly – I like to see the kind of work they have done before and like to hear their stories of projects on exactly what was that they were testing and what were the key defects found. What was their approach and the key challenges faced and how did they bring up the challenges to the notice of relevant stakeholders and how did they do the contingency or mitigation as applicable. Formally speaking, we have 2 set of patterns – one for freshers and other for lateral hiring. For laterals particularly - We have a process that helps us map the required technical skills with the available resumes. So, anytime we have a need we look into our database of testers who have applied with us in past and from the matching resumes, we shortlist based on availability / joining time and expected compensation. Post this, we share certain project links with these potential candidates and expect them to submit us their Bug Reports in a given time frame. Finally, after a candidate is found technically suitable, we setup face to face interactions to evaluate more HR aspects and general communication. For freshers, it’s really about what they have to offer beyond their formal education. No firm expectations, as long as they are able to convince us of them being fast learners and having aspirational attitude, they might. SMITA MISHRA
  • 8. www.TestingCircus.com March 2014 - 08 - 11. What will you suggest to people who want to join IT industry as software testers? I am very happy to be part of the software testing community. And I can only suggest the new folks that learn what is testing and then get into it. I find people to be doing testing for all the wrong reasons like – its easier that Development, it doesn’t need coding, timelines are easier and hence the work pressure, etc. None of this is true. Testing is becoming as challenging as any other part is with the ever evolving IT and When I look back, I think the biggest gap that I see during my initial work tenure was how closed and we were in our approach to testing. working in silos as a closed group and not being exposed to the whole world full of knowledge and awesome teachers and trainers willing to invest time in you. 12. Name few people you would like to thank, people who helped you directly or indirectly in your career as a software testing professional. I would like to thank my family for always standing by me – ALWAYS. But a special mention of my son here for being the world’s most loving and caring son and being very understanding when his mommy needs to work. At work front there are too many names and I would like to thank each good and bad interaction, because it has helped me grow in one way or other. Everyone I know – Thanks for being part of my life. 13. One last question – Do you read Testing Circus Magazine? If yes, what is your feedback to improve this magazine? Yes, I do read Testing Circus Magazine. I first heard about it in 2010 I guess when my manager Akash had been invited to be interviewed in it. But I never followed it then. I began to read Testing Circus since sometime in 2013 onwards. I liked the original format of the magazine. But the new format is truly amazing. I really enjoyed the editions coming out in 2014. And I think they are doing all the right things to get noticed and to put in right content. My only words will be – continue the good work. And maybe you can cover small meetups and group test events happening across the globe as a regular column. This could help other meetups to learn what more can be done and how to be effective. Blog/Site: http://smitamishrablog.wordpress.com Twitter ID: @smita_qazone SMITA MISHRA
  • 10. The simple thing is this - write your requirement as a test. With defined inputs and outputs. Expected results and expected (and unexpected) data. If you do this, your life will become bright, shiny, and you will live happily ever after. Or at least your soft- ware will become much much, much more reliable. And you’ll probably find out a lot of things about your ideas before you’ve invested in building unnecessary features and details. You can do this with high level requirements, such as business goals and overall objectives, as well as with low level isolated features, and everything in between. I think that the “everything in between” part is where we (software industry people) lacks the most care and insight about the importance of concrete, testable re- quirements. When it comes to high level requirements, we may have business people who do follow up business cases & objectives, i.e. test the results of the investment, at least I have seen it done once or twice. When it comes to very low level requirements, or mi- cro-requirement as my friend @spindelmanne call them, TDD do take care of it to some extent. Such as “When renaming item x the list will keep the same sort order”. It’s hard to separate micro-requirements from real business requirements sometimes. “What is really a valid input string here?” “How should we present the date format” etc but good developers generally can make some good micro-requirements decisions. For the “everything in between” requirements, we have a lot of work to do to make them testable. From what I know it seems as there are mainly 3 ways of communi- cating requirements today. Either you’re “agile” and have a loosely defined prod- uct backlog, filled with short user stories and then not so much more information. Or you have a heavy regulated requirements process, with hundreds of pages of use cases or “shall”-require- ments. Often with abstract statements such as (from real example): “Purchase has generated a receipt” Or the ad-hoc requirements: “Let’s send an email to the developer telling what I need to have”. Example: “We need to update the purchasing order receipt page. Right now it doesn’t show the total. The total need to be there. When can this be done?”. As a requirements analysts / project manager I have seen and practiced a way out of these three abstract, ambiguous, non-informative ways of communicating requirements. Much thanks to developers who serious- ly cared about taking TDD to the next level, and by having the chance to work with testers close by who taught me how to express what I want as test scenarios. I’ll share some examples from a previous project. www.TestingCircus.com March 2014 - 10 - - Ulrika Park Requirements People Need Your Help! Examples of Testable Requirements
  • 11. A testable business requirement I was asked by the business owner to implement a fea- ture: “Cardholders should be able to edit the rights for a whole household to use the money on their bonus card” Since money and banking was involved, it was a bit complicated to implement. My first question to the business owner was: “why?” and how will you know it works?” After quite a lengthy conversation, he said that what he really cared about was that the money on the bonus account was spent. He didn’t want the money to stay on their bonus cards. “How can we verify that this target is achieved?” I asked. “Well.. he said. If the money is spent, then the feature works.” “So.. when in time is realistic that we can check this..?” “Well.. within 6 months we should have a better rate of spending the bonus money than now” he said. “Ok. So what do you mean by ‘better’?” “Hm…” he said. “I’d be content for now if 50% of the total money paid out to customers bonus accounts would be spent”. “Thanks for clarifying! Our feature could help out with achieving that goal. But to achieve this, other things are involved. Customers need to know about how to share bonus money between people in their family. How will they know? Marketing, customer service.. a lot of factors might affect if this feature is used by the customer.” “Yes, of course. But this is what I really care about. And when you have a feature households can use, we should do an effort to inform customers”. Good. Now we had a high level business goal, a testable business requirement. Even though our feature wouldn’t be the sole solution to make the business achieve this goal, knowing the target for sure helped us a lot in developing the feature. What to do when you don’t have access to the business owner? When maybe you just get a bunch of use cases from somewhere to implement? Well, in these cases I try to define my own hypothesis about the main goal and result. “This is how I / we have interpreted the target since we don’t know” and then show for those stakeholders I do have access to. Often I do get some feedback on my hypothetical business goal statement. Even “You’re totally wrong in your assump- tion!!” is good to know before developing anything. And you have a reason to ask for answers. Before testing or developing any feature, we have to know or make a clear defined assumption about the expected result for business. A testable middle level, user requirement So now we knew the business goal of the feature. The feature could be implemented in many ways, with op- tions from everything from printing and scanning paper forms to digital authorization functionality. As a requirements analysts, turning into a tests-before- development tester, I defined some user stories. (We did a lot of other things too to understand what solution might fit, but that’s another story). The main “middle level” user story: As main cardholder I want to authorize other card-holders in my family in order for anyone to use the money on the bonus account. Before communicating this to the development team, I start to think about.. how to test this? What would I test? I brought in a tester for a chat. The tester was busy with other assignments, but he did have a few minutes to help me out. And I asked him “How would you test this story?” “Identify scenarios” he told me. And with some coach- ing I made up some scenarios. (here is just a snapshot to keep the article short) Scenario 1: Give authorization to other cardholder in a household with only 2 cardholders. Scenario 2: Give authorization to other cardholder in a household with several cardholders. Scenario 3: Authorization process is actively canceled by cardholder Scenario 4: Authorization process is canceled by un- planned interruption etc. Then, exemplify these scenarios with Gherkin inspired syntax: www.TestingCircus.com March 2014 - 11 -
  • 12. Scenario 1: Give authorization to other cardholder in a household with only 2 cardholders. Given that: Household has 2 and only 2 cardholders The 2nd cardholder doesn’t currently have the right to use bonus money Main cardholder has actively selected the 2nd cardholder The 2nd cardholder is >= 12 years old Expected result: Information is shown: “You have now given authoriza- tion to <2nd cardholders full name> with SSN: <2nd cardholders SSN>. The 2nd cardholder now has authority to use bonus. Scenario 3: Authorization process is actively canceled by cardholder Given that: Household has 2 and only 2 cardholders The 2nd cardholder doesn’t currently have the right to use bonus money Main cardholder has actively selected the 2nd cardholder and has entered external digital identification application to authorize and shuts down the digital identification application Expected results: The 2nd cardholder doesn’t have authority to use bonus Message to main cardholder: The change has been can- celed. These were just two examples. Doing this I had now: 1. A specification that can be used for testing 2. Got forced to find out the exact business rules regard- ing who actually could be authorized. The age limit, for instance, was discovered by defining the tests. By defin- ing test scenarios, I got a foundation to ask the right questions to domain experts. “Can anyone be autho- rized?” 3. A way to at any time go back and show for who it might concern (developers, stakeholders, customer ser- vice) what exact requirements and rules we built for. These are just small examples from a big domain, so I expect you have a lot of critique “Where are scenario x?? And those examples could for sure cover much more! And aren’t that statement a bit vague? What does ‘ac- tively’ mean really..?” The point here is not to give the full picture. For that I’d need to write a book :-) The point is to show that by defining tests while working with the requirement, the requirement got much more explicit. We saved time for our tester, who could focus on exploratory testing when time came. We saved a lot of time for developers who actually got the information ahead development on what would be tested, and what rules should be applied, so they saved a lot of rework. Also when having conversation with developers about the examples we modified them a bit, and removed unnecessary ones, or added missing. Some of the scenar- ios could even be quite easily automated while develop- ing, which also saved us a lot of time. So, what to do as a tester then? If you’re just not involved in creating the requirement definitions? Maybe the re- quirements are just handed off to you in one way or another, and when you get them they’re not testable at all? One thing you can do as a tester, is to make an effort to be included when other people are working with re- quirements. Offer your help! Only once I have been approached by a tester with this offer! It’s always me who’ve approached testers to help me with making testable requirements. Have you ever as a tester tried to offer your help – and seriously tried? To the guys working with requirements? And there are much more efficient ways than to say “If you need me you know where I am”. A tester could say something like this: “I know I’m going to work on testing for this project a couple of months from now. Is it possible for me to see some of the requirements already now? It will help me get ramped up quicker when I’m in the project. I’m in another project right now, but still I should be able to take one or two hours to look at what you have. And yeah, incomplete use cases or draft user stories will work fine too!” When you do get hold of some kind of requirements early in the process, take a quick look. Try to define some simple test cases or scenarios, and maybe you will get an opportunity to discuss these with the requirement ana- www.TestingCircus.com March 2014 - 12 -
  • 13. lysts (or whatever role who works with the require- ments).. Then why not inviting her for a lunch or a virtual coffee break (or other social excuse) over Skype in case of distributed teams? Requirements people need your help! They just don’t know it yet. Who, if not you, will seriously invite to make their work testable? Ulrika Park is a requirements geek with a passion for testing, methods, learning & the development of prod- ucts & services within organizations and teams. With 15 years of experience in software development, management & business she now works at SmartBear. She believes in the synergy of people, software and quality thinking to change the world. www.TestingCircus.com March 2014 - 13 -
  • 14. Look around in various IT companies, flip pages of some IT magazine, read blogs, see forums and you will be sure to find the today’s buzz word, the Cloud. You cannot find a word that has created so much of ripples than the Cloud. Very dramatically, the thinking process on providing computing resources has been shifted to Cloud computing. With this shift of focus, a lot of confusion has accumulated along the way in people’s mind. Companies are taking this concept very seriously. Many have moved to cloud, and those who are still on the “ground” are planning to move to cloud in next couple of years. People are still confused. They still are not comfortable with the Cloud thought. You ask anyone what a cloud service is and you still won’t get a very confident answer. Even if you get an answer, ask what benefits they offer and what are the disadvantages at stake, and you can see the person will be uncomfortable answering this. Make him skip the discussion by asking how secure they are, and it will be end of discussion. When it comes to testing applications, we are not fully sure whether to adopt the cloud approach of testing or not. What are the benefits we get and what are the hurdles on the way? Let’s take a closer look at the Cloud and how it can benefit out day today testing activities. So what is a Cloud? Cloud computing is a business and economic model. This model has been successfully deployed and executed for various material commodities since its inception, but in the recent years it has been formalized for IT products and services’. Let me try and explain the basic difference between a Cloud service and a Non-Cloud service. Consider that you have to move from one place to another. You can use your car OR use the services of a Taxi. Both of these vehicles have several similarities · Both are automobiles, having very similar structure and machinery. · Both provide basic functionality of transferring people / goods from one place to other. So where lies the difference? The difference is in the business model for the service provided by them. · When the car is owners, the owner has to pay for the fuel, regular maintenance and even possibly a garage. In turn, the car provides the service solely to the owner - you. www.TestingCircus.com March 2014 - 14 - - Vipin Jain - Anubha Jain Taking Testing to The “Cloud”
  • 15. · When the car is a Taxi, the service provided by the taxi cab can be described as ‘Travel as a service’. Who owns the car and who pays the maintenance is not your concern. As a customer, you just have to pay to travel from your place to another desired place. No maintenance fees and no parking fees need to be paid by you. This responsibility lies with the cab driver. The Cloud is synonymous with the phrase ‘On Demand’. You pay only on demand (when you require it.) Cloud gained momentum when IT industry got associated with it. Today, we find that there is a huge range of products and serv- ices available on demand. All are “As a service” e.g. ‘Games as a service’, ‘Java as a service’, ‘Stor- age as a service’ and the list is endless. Fig 1: TaaS or ‘Travel as a service’ – A Metro Cab service How Testing and Cloud work together? Cloud-based testing offers a remarkable combination of low costs, pay-per use model and elimination of initial capital expenditures. The benefits, however, are more than just cost effectiveness. The non-cost factors include on-demand flexibility, a respite from holding various infrastructure assets, enhanced collaboration, higher levels of efficiency and, most importantly, reduced time to-market for key business applications. Economically the vendor and end user gets huge benefits from the Cloud. This comes directly from reduced subscription prices of any product or service. Other benefits are: · Cloud Apps are scalable: The elastic business model, as it is popularly known as, can be customized based on the requirement · Auto-Provisioning: Depending on needs of end users, various Cloud vendors provide and withhold the offerings in a manner which is in an automatic and self-serving format. · Unlimited resources: At least the end user thinks and feels this. The services / products are available as and when they are demanded and in the required quantity The Cloud computing model should be coined as ‘Green Model’ as it maximizes usage of resources and minimizes wastage making it environment friendly. Current state of testing in cloud Many companies are still taking a cautious approach with cloud computing. This is not the case with testing however. They are largely ready for testing in cloud and following reasons will account for that readiness: · As we all know, testing is not a one time, but a periodic exercise and each project will require a new environment to be set up. If a company creates a Test lab, it typically sit unused for longer periods, resulting in a waste of cost, electricity and space. Lots of published reports indicate that more than 50% of the www.TestingCircus.com March 2014 - 15 -
  • 16. technical infrastructure for testing remains underutilized. · Since testing still being considered as a must but non-critical activity for business, taking testing in cloud premises is pretty safe as it doesn’t include important company data and has minimum impact on the organizational business activities. · Today’s applications are increasingly getting dynamic, more complex, distributed across continents and more component-based. Testing them is getting more challenging. For instance, with mobile and Web applications testing, testing needs to be done for multiple operating systems and regular critical updates, various browsers and their versions, variety of hardware and a large pool of concurrent users to understand real time performance. It is pretty difficult to follow the age-old approach of creating so many in-house testing environments. These will become very complex and will need huge capital and resources. Fujitsu in a 2010 research suggested that testing ranked second (57%) as the most likely workload to be put into the cloud after Web sites (61%). The on-demand provisioning by Cloud addresses all the above explained issues with one click. On top of it, the effort and resources saved by using cloud can be redeployed for core business functions. The Cost Factor Economic benefits are the main factor influencing companies to take testing to cloud. Another 2010 survey by IDC hinted the same. As the global economy recesses, companies continue to find ways to regulate costs and improve ROIs. Cloud testing reduces the unit cost of computing. www.TestingCircus.com March 2014 - 16 -
  • 17. Small and medium-size businesses that cannot afford high costs find cloud-based testing as a new lifeline. The companies no longer need to invest in infrastructure and software licenses. They also do not need to worry about various configuration issues and maintenance of test environments, and pay only for what they use. Beyond cost Benefits? Cloud is not all about cost saving. There are lot more benefits companies can extract using it. - A standard infrastructure and pre-configured software is available, reducing efforts in getting servers and licenses. - On-demand provisioning helps companies to think forward instead of spending time to set up test labs. All testing resources required for testing exist within the system and can be called upon instantaneously. - Better analysis and control are offered to test teams to build and execute their tests and identify the bottlenecks. This helps in identifying possible runtime bugs a lot before they are actually found. - It’s a great concept in motion between geographical distributed teams. Once a tester logs in and runs a test, the results are available over the cloud. The developer can then assess it and fix over the cloud itself. This eliminates back-and-forth communication between teams. Limitations - Lack of standards: Absence of universal/ standard solutions to real world problems is a big issue. Each cloud provider can have its own hardware, operating models and prices and may or may not offer any interoperability. This poses a huge challenge for companies when they plan to switch to a new vendors - Security issues: Security in the public cloud is still a huge concern because the data may be stored in a location which is outside a company’s legal and functioning jurisdiction. - Usage: The everyday usage costs increases very rapidly if testing is done without a proper usage of cloud-based test. Though pay-as-you-go clouds are used, it can be expensive if the testing is out of sync with requirements. www.TestingCircus.com March 2014 - 17-
  • 18. - Performance: It is always an issue since public clouds are used by many users in parallel. Situa- tions got created a company had to wait for the required bandwidth to execute their tests. Conclusion This paper tries to explain the benefits and limitations associated with taking testing to Cloud. We have tried to explain that why companies should start small and gain confidence slowly to capture maximum benefits of cloud-based testing. Once they believe that it has led them in speeding time to market, lowering of costs and ensuring standards compliance, they can go big. Using of pay-as-you-go or on-demand services intelligently and efficiently, companies can reduce cost of operation and ownership. Companies should pilot cloud-based testing as early as they feel comfortable before going to mainstream testing. Bibliography - “Determine if a Cloud is Usable,” blog post, Bloomberg Businessweek, Jan. 31, 2011. - “Solving the Challenges of Enterprise Mobile Appli- cation Development With Cloud-Based Testing,” blog post, - CIO, Feb. 17, 2011. - Rajagopal Sattaluri, “Testing Considerations for Ap- plication Migration to Cloud Computing,” - Cloud Computing Journal, Feb. 8, 2011. - “Cloud Computing: The Good, Bad and the Ugly,” blog post, Dynamic Data Inc., Feb. 1, 2011. - “Cloud Testing, A Growing Trend,” blog post, So- nata Software, April 4, 2010. - Nivedan Prakash, “Cloud Testing: Attracting De- mand,” Express Computer, Feb. 1, 2010. www.TestingCircus.com March 2014 - 18 - Vipin has dedicated last 10 years of his professional career to the software quality and testing. Currently working with Metacube Software, India, he developed his key skills in developing automation frameworks and automat- ing applications. With a proven record of implementing and refining test processes for various clients across the globe. he is the author of several articles and seven well sold books in India. He is pretty active within the software testing community by speaking at International and national conferences, writing articles and contributing to various blogs and forums. Anubha is working as Associate Professor & Head, IT department, Internation- al College for Girls (ICG), located in Jaipur, India. An academician for last 11 years, she has been involved in teaching and mentoring several students in the field of Computer Science. Knowing that teaching is the best form of giving knowledge back to society, she worked as a lecturer in Subodh college, Jaipur before settling in her current role at ICG. She is currently pursuing her PhD, in field of Information architecture. Anubha is the author of 9 books and a regular contributor in various forums.
  • 19. To Subscribe Click Here Still relying on reading Testing Circus from tweets & facebook updates? Subscribe just with your email id and get the magazine delivered to your email every month, free! Testing Circus www.testingcircus.com www.TestingCircus.com March 2014 - 19 -
  • 20. I recently gave a talk at a conference. I learnt many things from my interactions with the attendees at my session. One specific thing that stood out for me was how – testers no matter how intelligent, smart, critical thinking they are, still fall into the common pitfall of using test cases for communicating testing progress and test coverage. If anyone reading this article, is still under this mindset, I would ask the following questions- · Do the test cases cover each and every scenario, each and every part of the system under test? · Can you give accurate information of testing effort in terms of test cases? Does it really make sense? · Are we trying to hide from reality and give some sort of number to make us and our project managers feel good when we report the number of test cases covered to communicate testing progress? · What do you actually mean by test coverage? Can any other information about the system influence this? Here is the common pitfall of completely relying on test cases. Firstly, 100% test coverage of the system is impossible. “Exhaustive Testing” is a misnomer. This being said, if anyone says “I have 10000 test cases, I executed all of them and thus all my testing is completed”, it is flawed because it is impossible to cover every scenario of a system by just executing all the test cases · What if there are some hidden scenarios that the tester still hasn’t uncovered? · What if another system influences the system under test in a different way than expected? · What if the system behaves differently under differ- ent situations and our test cases did not cover this? ….and so on. There are so many “What if” questions to ask ourselves. It’s an endless list. So the point is, NO; we cannot possibility write every possible test case to cover each and every scenario of a system. This is where as testers, we think about testing approaches complementary to just executing test cases like exploratory testing, automated testing, combinato- rial testing and other approaches based on the context and scope of what we intend to cover. This may help to uncover other weird scenarios or trigger unexpected outcomes. With this being the case, it may not be valid to say, “I have 100% test coverage because I executed all the test cases”. Secondly, you may then ask, how should I give up- dates on testing progress to my stakeholders? In my experience, I usually do not have a problem in explain- ing testing progress to my stakeholders even if they are old school and believe “test cases are the solution to all the problems”. I would of course have some test cases that I execute, but also do lot of complementary testing to test case execution. Finally, I give the approximate percentage of modules covered/tested in the system as my testing progress. So for example- If there is a system A, I generally split it into different modules M1, M2…….Mn (test areas). I use different testing approaches to test these modules www.TestingCircus.com March 2014 - 20 - - Raj Subramanian A Common Pitfall Of Test Cases
  • 21. and give an update in terms of the percentage of mod- ules I have covered in the system. (This is a general approximation as there is always a possibility that I haven’t thought about some other module in the system which may cause problems. To err is human.) So, say if I had covered M1, M2 out of the 5 modules I have identified i.e M1, M2……M5, then I may have cov- ered about 40% of the system ([2 modules/5 modules] * 100, in case you were wondering how I came up with this percentage ) . This is how I give my updates on testing progress. This by itself helps to give the stake- holders some level of visibility into testing progress and helps to make decisions accordingly. To summarize, I think as testers we know it is ethically wrong to make false claims. That being said, just using test cases to estimate the amount of testing progress and test coverage falls under this ethical problem as we are giving wrong impression to our stakeholders who trust us. Doing this tarnishes tester credibility which is one of the most important factors that influence a tester’s work and helps to establish trust among his peers. Raj Subramanian is a pas- sionate tester, who has previous experience work- ing as a developer and moved to testing to focus on his passion. He gradu- ated from Rochester Insti- tute of Technology with a Masters in Software Engi- neering, worked as a developer for a payroll processing company and currently works as a test engineer for a major insurance company with specific focus on mobile testing. He has been pret- ty active in terms of learning and contributing to the testing society by speaking at conferences, writing articles, blogging and being directly in- volved in various testing related activities. He currently lives in Cleveland, Ohio. Raj blogs at http://www.rajsubra.com/blog/ And tweets with @epsilon11 www.TestingCircus.com March 2014 - 21 -
  • 22. moolya sucks we test fast and don’t know to make more money from our customers. we are like this only sales@moolya.com
  • 23. You have been asked to test the latest Mobile based application, created painstakingly by your organization over the last few months. All you have with you is your exclusive Smartphone, which you saved for, and bought, over the last few months / years. The choices are now with you, either beg/borrow/steal from col- leagues or do the right thing. Fire up that web browser on your Smartphone, open up ‘Google’ and have a go at searching for ‘Mobile Test Partners on the Cloud’. I would generally as a rule go for the second option and not put ‘my precious’ phone to the test on un-tested applications. What you achieve when you go through the search, is a wide variety of web sites and commercial vendors of- fering this facility to you, at a fraction of the cost that you would spend in setting up your own facilities and test lab. You need to now make an informed decision by researching and figuring out which one of these would serve your purpose the best and also give you value for the money spent. Mobile testing has come a long way. From the initial fragmented scenario of having to check on each kind of screen resolution and phone type and screen size, add to that the variety of mobile browsers being offered by the various vendors, to the current situation of having apps created using HTML5 versus native apps. Adding to the general confusion is the non-app area, where companies wish to check the overall ‘responsiveness’ of their “web pages” across the same wide variety of devices, which includes [and not restricted to] iOS (iPhones and iPads), Android (Tablets and Phones – low end and high end), and recently the Windows 8 (WP8 devices and Windows RT devices and desktop’s with Windows 7 & 8 included) and Chrome OS (mainly low end laptops & devices). This vast array of devices also brings up with them an equally confusing array of browsers along with them (at the last count it was something like 9 browsers and growing). Faced with a similar dilemma, we take refuge in the well-trodden path of checking on the search engines for a Web Accessible Mobile Testing Tool (applications and browser based systems), that would be able to serve our purpose and not cause the management to jump from their warm seats when they finally receive the bill for the services used to test their precious new mobile application or website. To get through all this, I listed down a few of the up and coming offerings which provide a good cross-section of the devices and are reasonable in costs. Although these cloud platforms do provide a service which is extreme- ly useful, keep in mind, you might need to make use of physically handling the device to run certain tests, which cannot be checked with automation (but there are ways and means to handle this, so don’t be disheart- ened). A hybrid cloud installation in these situations can be one of the simpler solutions which come handy. A hybrid cloud provides a small subset of devices at your physical location and the wider variety at a remote location. Of the Could Mobile Test providers, the most important thing to look into is if they provide you with storage for your tests and a way to run the automated tests that you have so painstakingly created for your application from within the cloud infrastructure. This is along with the use of making sure that these tests run www.TestingCircus.com March 2014 - 23 - - Gagneet Singh Mobile Application Testing Using the Cloud Infrastructure
  • 24. over a wide variety of network speeds simulated to provide the 2G, 3G, 4G and Wireless speeds prevalent in the World Wide Networks around the world. Practically a Cloud Based system should be able to cater for most of the requirements outlined above; and the good news is that most of them do exactly that. A few of the prominent ones which I have used recently and the ones which come to my mind are: pCloudy.com, appurify.com, perfectomobile.com, HP mobile testing lab, to name just a few of the upcoming entrants in this field, where by Gartner estimates, there are 5.6 billion handsets present. pCloudy.com has stood out as an ex- cellent upcoming product, being run by engineers who have previously worked with Nokia and other top mo- bile hardware development companies. They have built the framework for testing on mobile devices from the ground up and have got some really cool features which go with it. (Disclaimer: I know one of the Co-Founders as a colleague from my previous companies). I loved the way they have handled the storage of tests and results on the site itself, along with providing features such as mock location maps, allowing the users to experience the different states of the application in separate geographi- cal locations. They have recently launched a feature of having multiple browsers and simplified their launching from within the cloud interface, making things easier for users again. And have launched the latest Android ‘KitKat’ version with the Nexus5. All in all, an excellent package to go for, with reasonable rates. All said, the main purpose of having a cloud based mobile test experience is important for any company wanting to launch its web presence these days. With the advent of HTML5 and other technologies like Founda- tion (more on this later), the web has become a place where people love a responsive site (or application across iOS, Chrome OS and Windows [Phone] 8) that caters for whatever device they are working on, and does not have the staid look and feel when they change from a Desktop -> Laptop -> Handheld Tablet -> Smartphone. They want to get the feel that the web site developer / organization has done their homework and provided them with a site where they do not have to pinch-in and out, just to read content. With the expansion of Smart- phone markets in the developing nations and organiza- tions wanting to tap into the ‘billions’ of people there to advance their products, it has become imperative for these organizations to go through the process of Mobile Testing their web sites and mobile based applications across various low end and high end devices to be useful. Sites like pCloudy.com, appurify.com, etc. are making it easier and faster to send out mobile products into the market by providing the required platform at a fraction of the cost of actually acquiring a complete mobile test lab. That said, you would still need to work out a small subset of your tests at a physical location, but I am sure that this would also become possible over the cloud. With the advances these start-ups are making with tech- nology and improvements on their own feature sets. I certainly would be looking forward to features which provide a friendly interface and let me run across multi- ple devices; and for that the Mobile Test on Cloud pro- viders are definitely a better option than trying to keep up with the influx of the hardware been thrown into the market by all the top end and low end hardware manu- facturers around the world. Gagneet Singh has been working in the Quality Assurance/Test field for the past 8 years (with an additional 4 years in Sys- tem Tools development) and has been involved with companies such as Toshiba, Adobe (Macro- media) , McAfee, Oracle, Yahoo! and recently Mi- crosoft. He likes to blog and to write about the experiences he faced in the various organizations and situa- tions. His work has mostly been with Automation Testing, along with Performance QA. Also, Secu- rity testing over the much hyped “Cloud Comput- ing” (using Hadoop and Azure) has figured in his work area. Currently working out of this place they call the “Down Under“, where he lives in Sydney, New South Wales! www.TestingCircus.com March 2014 - 24 -
  • 25. I want to define what I mean by ‘late’ and ‘early’ and then go on to give some context from a firm I once worked for – let’s give them the fictitious name “Lin- man Manufacturing” for convenience. If I don’t define my terms, we will all get lost. I will use ‘early’ and ‘late’ in an almost intuitive way, although perhaps not as everyone uses them. They are not of the Humpty Dumpty variety (Alice Through the Looking Glass by Lewis Carroll: 'When I use a word,’ Humpty Dumpty said in rather a scornful tone, ‘it means just what I choose it to mean — neither more nor less.’). So, for me ‘early’ is before we were ready, and ‘late’ is after the planned date. These are pragmatic terms – let’s not get argue about them now. My hope is that you will see the usefulness for yourself. To start with, I need to say something about project deadlines. First of all, it is not testers (or test managers) who determine when a work product is implemented (into PROD, obviously). Test resources can speak into, or provide information, that helps others make that kind of decision, but the final decision lies outside “testing”. Secondly, we per- haps all recognize that if the decision were left to tes- ters, work products would not get implemented. We testers are a pessimistic breed, and tend to focus on things that don’t work. It is not always the 390 require- ments that work that get our attention, but the 1 that does not. We concentrate on that. Lastly on deadlines, some deadlines are more impor- tant than others. Take the day I am writing this article. ‘Tomorrow’ I am preaching at a local church, and some- time before standing up, I need to have prepared what I am going to say. That is an immoveable deadline. Similarly, the deadline to ‘post’ this article is midnight tomorrow. If the editor gets it by then, it will be consid- ered for the next edition. Otherwise, it will be a candi- date for the edition that follows. This is another deadline that has significant consequences if it is missed. Some software projects have hard-wired dead- lines that cannot be missed. Perhaps there are legisla- tion changes that need to be complied with. The introduction of the Euro is a good example. However, even when there is a hard (i.e. cannot be moved) dead- line, there is sometimes wriggle-room around the edg- es. Some parts of company accounting in Euro-zone countries needed immediate changes when the Euro was introduced. Other parts did not – activities around the end of the company financial year may not have been needed to be correct on Euro day-1 – only at the end of the company financial year. So even where there is a hard deadline, some parts may not need to be ready on the first day, whilst other parts must be available, and be seen to be working. Right. The pre-amble is over. I want to tell you about my time at “Linman Manufacturing”. In my many months working for them, there were 8 major imple- mentations, where significant new functionality was introduced. Although each implementation was sepa- rate, there were both implicit and explicit points of interaction; data was loaded into and extracted from the same database, with some common database tables used. As time progressed, later functionality relied up- on the data that had been introduced in earlier phases, and was loaded month-on-month. I want to concentrate on the last 4 major deliverables, giving the targeted delivery date into PROD and the actual date. www.TestingCircus.com March 2014 - 25 - - Peter Morgan Better to Implement Late Than Early How Quicker can mean Slower
  • 26. I mentioned that there 8 major implementations. Of course, there were other times that software was promot- ed into the PROD environment. User-requested changes, software platform upgrades, and system tweaking oc- curred throughout this time frame. But if the project team as a whole had been asked before each phase tabled above, for phases F and G the answer would have been ‘yes’, and for phases E and H ‘no’. In the time that was given, the best testing, based upon risk, had been carried out. But there were huge swathes where the answer at best was “we don’t know”, and some parts were clearly not working. The architecture was basically to take mul- tiple-format files on a regular basis (daily / weekly / monthly) and to load this into a standard star schema data warehouse, and then extract the summary informa- tion. For phase H, data had been successfully loaded many times, but the reporting layer only developed in outline. Now it is not until data is output that it is possi- ble to tell whether it was loaded successfully (i.e. it is fit for purpose). And so it proved to be. For phase E, there were immediately amendments re- quired, both to data loading routines and to the report- ing layer. Seven weeks later (and after seven implementations), the solution was stabilizing. Let me tell you, those seven weeks were interesting. Some of the input processes could not be run for part of that time, meaning that reported information was only partially correct, and it was not until nine weeks after the date that users could gain any confidence in the output. January 2009 was the first time that month-end reconciliation (for December 2008) could be attempted. That is quite a long time after the actual implementation date. If anything, the situation was worse with phase H, and interestingly enough from the table above, this phase was both ‘late’ (after the targeted delivery date) and ‘early’ (implemented before it was ready). The first weeks were spent loading a backlog of data, with 10 months of data to load immediately. The reporting layer quickly highlighted ‘problems’ with 7 of the 16 data feeds – problems that could only be rectified by remov- ing any data loaded to date, and either re-engineering the data creation process (before it was made available to load), or changing the data load procedures. There were times that it seemed data was being backed out faster than it was loaded – even though once a data feed was identified as ‘wrong’, nothing else was loaded for that feed until the problem was resolved. A big score-board showed what data had been loaded for which feeds on a month-by-month basis. March 2010 (when data for Feb- ruary 2010 was loaded) was the first month-end with stability. There are business reasons for ensuring that “something is delivered”, and that progress is seen to be made. However, for “Linman Manufacturing” the information loaded into the Data Warehouse was for long-term stra- tegic planning, not for the day-to-day business opera- tion. There were, in one sense, no unmovable project deadlines. If you have a choice, never implement a software solu- tion ‘early’. Doing so makes for a frenetic period of activity just after the implementation date, and some- times aborted product launches. If you take longer, it can mean that the software is available earlier. When that happens, staff can be truly released to go onto the next project without constant drag-back from activities that should have been put to bed! To do otherwise means that quicker is slower. www.TestingCircus.com March 2014 - 26 - Peter Morgan is a testing professional who has been involved in the ICT indus- try for more than 30 years, and worked in the free- lance marketplace for much of that time. His time has sometimes moved from testing to ‘develop- ment’, but he would add “always using the mind- set of a tester”. He is passionate about testing and a firm advocate of testing qualifications. An en- thusiastic speaker and author, Peter tries to base his output on hands-on experience, attempting to relate fine sounding ideas back to how it will affect Joe or Jane Tester in their everyday working lives in the war of attrition that we call software testing. He is a regular at EuroSTAR conferences, and is speaking at Belgium Testing Days in 2014 – the forth year in a row. At this time of life, Peter offers experience, and can sometimes say when offered a tricky problem: “Been there, done that, and here are 2 or 3 options that may, just may, work in this situation”. He continues to learn, adding technical skills to his impressive range or hardware / software / business sector portfolio.
  • 27. I don’t know if anyone has noticed but the world we live in now is different from the world we used to live in just 5 years ago. For me, the change that occurred is far rapid than anticipated, because if we refer this to a 20 or even 10 year past leap then most of the skeptics would say “Why not!, it should have changed!” but the shortness and the speed of this period leaves no room for being skeptic or naïve – We simply need to understand and adapt! Mobility has broken free from just being a device to communicate into an instinct. We are using our hand held devices in so many different dimensions other than it was meant to be that it is changing the complete concept of computing and networking. Hand held devices are now our nodes to the clouds; we are now the new form of terminals; “The Human Terminals”; the value to the word “Touch” has enhanced itself from Touch screen, and then within a very short time it again reboots itself to “Human Senses”! It feels like a history in making where the systems which are categorized as something purely mechanical and electrical combination of components with mix of complex logics and design are re-emerging to sapiens, like an event in space and time; As I see it, the systems are reaching to our senses, our beliefs, daily lives and simply becoming an extension to our self being, like an implanted new body part. Gadgets are now not need to be held in hands; they are wearable, and soon they will represent a mere extension to its users. The human mind is now adapting these effects; see for example our children; how they have adapted the use of touch screen, games and the use of smart phones. How people are getting aware of the communication and sharing bounds and exploits over social networks; Where to “say and share” what is now dependent on “where and how” you are registered as a user; Twitters, Facebook, LinkedIn, Instagram, Four Square, Google, pint it, flicker and so on. We are not structured anymore; we are living multiple lives under multiple scenarios. Somewhat, this is not based entirely on our choices, this is how the environment around is “turning out to be” – and we have to live it! We are now thinking! Or in other more adaptable senses we are now becoming Context Driven! These factors are effecting what we used to know as the solutions and systems; these factors are now playing a major role in provisions of the right solutions to the core requirements and needs of the users. And… the same factors are affecting how we “Test” these systems; In a way, we are now moving to an age where due to the magnitude of the contextual effect on any scenario, generic approaches will simply fall apart and fail! – In so many words, we need specialists! We need empowered human beings and their expertise to address the right issues at the right time; we need to identify the right coverage to find the important bugs, and mind you – We need to identify ALL OF THEM! Under several scenarios and contextual situations and due to certain trendy and cultural bonds we have not yet succeeded in creating our own identities as Testers. We are still based on the “Types” and “Approaches” scenario. Where, Testers when tend to define what they do, start explaining the testing types and the approaches they follow. They tend to list down several technical tools and programming languages as skills, and not as “Tools” to support testing. In this stream of self- www.TestingCircus.com March 2014 - 27 - - Arslan Ali 7 Types of Testers What is Your Identity?
  • 28. discovery they forget the very scarlet thread they cross from being a “Tester” to “being” a Coder. Well don’t worry we have worked that out as well; James Bach has already published twice about the types of testers there are after receiving several queries regarding the latter; in his recent blog post, he mentioned the following as the testers’ types: (I am quoting him here, rather putting in my own words – it is better that way) (SEVEN KINDS OF TESTERS) · Automated? Manual? There is no such thing as manual or automated testing. It’s all just testing. Testing is often supported by tools that attempt to simulate user interaction with the system. This is what people call “test automation” even though it is only automating a crude approximation of one aspect of testing. If you have the ambition to be a one-man test team, it is extremely valuable to learn how to make your own tools. · Exploratory? Scripted? There is no such thing as an exploratory or scripted tester. All good testing is exploratory to some degree and scripted to some degree. · Tester. This is a testing generalist who can contribute to any test team. Sometimes called a QA analyst, QA engineer, or test engineer. I prefer the simplicity of “tester.” · Omega Tester. The omega tester (which I sometimes call a test jumper, after the analogy of a paratrooper) is one who can do anything. An omega tester is equipped to be the only tester in a project team, if necessary. Omega testers can lead testing, or work with a team of other testers. I am an omega tester. I aspire to be a good one. · Performance Tester. The performance tester understands the mathematics and dynamics of the performance of large-scale systems. They use tools that create high loads and measure the performance envelope of systems as they scale up. Performance testers often don’t think of themselves as testers. · Usability Tester. The usability tester is a bit mythical. I have met only two dedicated usability testers in my entire career, but I have seen more of them from a distance. A usability tester specializes in studying how users feel about using and learning a product. · Security Tester. Security testers also often don’t think of themselves as testers. Security is an exciting, specialized form of testing that requires the mastery of a great many facts about a great many technologies. · Testing Toolsmith. A testing Toolsmith is a programmer dedicated to writing and maintaining tools that help testers. This is what a lot of people would call an “automated tester” but you better not use that term around me. · “SDET” Software Development Engineer in Test. This means a full on programmer who does testing using his programming skills. On the other hand there are several characteristics as a tester which can stood up and create an exclusive tester identity; For example, boundary Testing heuristics; now I know that several of us have used this term in our CVs and have somehow read or taken part in forums to talk about this type testing; but how many of us have actually bifurcated this into “Galumphing”, or “Steeplechase” or even “Leap and Creep” - I don’t think any of us! OR While discussing about product coverage we have used the term “San Francisco” depot instead of discussing the missing out requirements, and blaming the developers of not providing the right specs or cursing the Implementation team of not discussing the right requirements; that is what common professional does! Why not stand out and use your own language? I have seen professional testers asking questions and worrying about “Testing without Requirements” or “Finding the right bugs” within a short period of time, but I have not seen any of them discussing about using or creating their own Heuristics? Why not? There is also no denial in discussing Quality and its criteria, but have we ever tried to use and educate ourselves with the use of the “Quality Criteria Heuristic called “FEW-HICCUPPS”? Try and do that! www.TestingCircus.com March 2014 - 28 -
  • 29. Let us understand now. Why can’t we move beyond being a technological freak to a Tester in a meaningful term; Does the mention of the very word “Heuristic” is not a mention of “Tool” in your sense. Technology is a wonderful thing, being technical is in the very genes of a computer professional; but as testers we need to identify ourselves as someone with a niche of being a tester and being technical at the same time; Being technical comes with the package; So why not mention who you really are? “A Tester”! Empower yourself, your language, attitude and reputation as tester. Learn Un-Learn and the re-learn! Try and improve your language as testers, differentiate yourself from the peers of Developers and Implementers. Yet sewed in and work for their support Or I can only say this then; you are in the wrong game baby! Arslan Ali has more than 14 years of Experience related to IT, Industry and Training Institutions with exclusive experience of 5 years in teaching various disciplines and projects in IT Institution. He has worked in various roles in capacity of Software Engineering, Software Tester, Trainer and Quality Assurance Roles. The Major focus of his expertise lies in Coordination, Implementation and Testing of ERPs and Customized Applications. He is also a trainer for Context Driven Testing for various companies and individuals. Arslan is currently working at Sidat Hyder Morshed Associates (www.sidathyder.com.pk) as a “Sr. Consultant – Information Solutions; but beside that he is also an active founding member of TestersTestified (www.testerstestified.com) (@testtified), Outtabox! (www.outtabox.co) (@OuttaBoxPk) and OISOL – Open Integrated Solutions (www.oisol.com) as a training consultant for Software Testing and Context Driven Testing Workshops. You can follow him on twitter @arslan0644 and on LinkedIn at pk.linkedin.com/in/thegoodchanges/ www.TestingCircus.com March 2014 - 29 -
  • 30. www.stepinforum.org www.qsitglobal.com Interaction » Insights » Opportunities 11th International Software Testing Conference Today’s tester is a unique being, driven, not by deadlines and mundane tasks, but by a relentless curiosity and a fascination with possibilities. Awaken the tester in you.This is where you want to be; this is NOW. Walk away with exciting ideas and quality insights from other industries. Make your mark by challenging the existing paradigm. Most Popular Conference among Start-Ups to Fortune 1000 Companies ExcitedtoparticipateasaSPEAKER/ DELEGATE/SPONSOR? writetonazreen@stepinforum.org orcall+91-80-40818888 Over 900+ delegates from India and abroad attended in STeP-IN SUMMIT in 2013 Testing NOW STeP-IN SUMMIT2014 Asia Pacific’s Largest SoftwareTesting Conference Produced by: Hosted by: JUNE 25-27, 2014 AT BANGALORE & JUNE 18 & 19, 2014 AT PUNE & HYDERABAD CONFERENCEPROGRAM View Past Conferences View Past Speakers’Gallery www.stepinsummit.stepinforum.org Pre-conferencetutorialworkshops(twoparalleltracks)@Pune,HyderabadandBangalore TargetAudience:TestLeads,TestArchitects,TestManagers,TestAnalysistsandConsultants Conference@Bangalore TargetAudience:CXOs,VicePresidents,Directors,BusinessHeads;Customersandend-usersofITproductsandservices; TestManagementprofessionals–Managers,TestLeads,Software,Test Engineers;Quality/SEPGManagement Professionals–Managers,QALeads,SoftwareQAEngineers; ProductDevelopmentManagers;ChangeAgents 18,19,25 June,2014 26-27 June,2014
  • 31. BOOK WORM’S CORNER March is the beginning of the appraisal season; that’s why 50% of the organization actually wonder if they should be continuing in the or- ganization or not. That’s why I decided to crawl through “Don’t Hire the Best”. This book is written by Abhijit Bhaduri. In case you are wondering if it is a misguide to hiring the right team, then you are partially incor- rect. This book preaches and lets you come to your own conclusions. The book tries to sell itself by lecturing about good hires and how to make good hires, however, it tries to draw the right line between “hiring right” and “hiring the best”. Unlike many other interviewing books that are filled with choc-a-bloc aptitude and problem solving questions, this book also tries to examine the organization culture and why culture fitment is important for the company. The case studies with the standard disclaimer of having been altered to protect identi- ty is very useful and causes the reader to ponder as to how he could have done things better. Reasons as to why it is important to create a fitment between the interviewee personality and the management ex- pectations, individual traits that need measurement, why making the wrong hire call can result in financial losses are stories talked about in depth in this book. What did it teach me? After reading the book, I ensure that I pass on a set of good and bad candidates to my interviewing team; that varia- tion causes them to be on their toes and interview without bias. If I send only good candidates their way, a form of bias sets in over the time which would result in all of us overlooking the most obvious reason for not hiring the candidate. Why should you buy this book? Because it’s not free; in addition to that, you will learn valuable lessons on how to create a team, why the culture fitment is important for your teams and why you need to ask questions beyond the role competencies while forming teams. When is it important to look into a resume to know what he’s capable of and when should you look beyond the resume? You will ask yourselves these questions if you were to read this book. I hope, after reading the book, you don’t hire the best anymore; you will hire “better than the best”. Love, WoBo www.TestingCircus.com March 2014 - 31 -
  • 32. Request a free demo by sending us an email at support@sahi.co.in http://www.sahi.co.in Become our fan - https://twitter.com/_sahi http://www.facebook.com/sahi.software www.TestingCircus.com March 2014 - 32 -
  • 33. Part 39 www.TestingCircus.com March 2014 - 33 - http://www.testingcircus.com/category/a-fake-testers-diary/ Random Musings My 39th month with this magazine; that's when I real- ized that I had inadvertently become a career counsel- lor for most of my family members who were in their final year of college. Is there any scope for "Java"? Should I learn ".NET"? Is the industry scope for "SQL" or "Oracle"? With so many questions throw at me from family members, I've stopped visiting any family func- tions or any event where I get to meet my extended family. Quoting client-meetings and a workaholic manager asking for meetings at mid-night, I manage to escape attending most of these functions. And then there is that irritating cousin of mine who keeps brag- ging of his topper credentials in schools and constantly asks me if there's more scope for "development" or "testing"? I have a good mind to tell him that "If people like you join Development teams, then testing career will have a lot of scope", but I dare not. Why do I write this column? --- "When a tester logs a bug and engages the developer in a sadistic discussion asking him to fix the bug, the tester is also torturing himself on some levels; the tester's life is such that he needs such discussions to survive his job" --- This is what a famous manager had to say about corporate culture for testers. Though there has been some amount of hue and cry against this column, there are a few instances that agree that this column unearths what lies beneath. The real thought of documenting this diary is to try and unmask not just the bad guys, but also the horrid practices that have created fake testers all over the place. I also fell prey to fake testing sometime back and believed in the concept of sanity testing, smoke testing, BVT testing and that all of them were different testing types; I also thought that I could become the greatest tester if I cleared all those exams. And now, I stand before you, as the FAKE TESTER. That's what I stand today. And I hope my writings will stop creating more fake testers. So far, in most columns, I have (unsuccessfully) tried to provide an insight as to what's wrong with today's testers. Most of them can be categorized as · the inability to find good and authentic testing teachers · how there are some people sitting in manage- ment who actually decide and state what test- ing is all about and the entire company thinks that's what testing is all about · the low pay of the testing engineer -- how many of them do you think have equal opportunities at growth and how many of them have 40 hour working weeks? · And the inability of the testing team in any company to function as an independent body, but function as a sub-body of another manage- ment function (development or support or mar- keting or sales or .....) there is no independent person or body leading the testing vision for the companies. And at the end of a year, many testers realize the disdain that others have for him; the disdain becomes reality. When this is pointed out to the management, the management don't try to make good the wrong things; they do some actions that makes the tester's life from bad to worse. For a long time, the sole objective of most testing com- panies seems to be a single agenda --- bill the client against the number of test cases executed; to showcase
  • 34. smartness, they record and replay some of the testing and quote savings in the name of automation. The clients cannot back out due to the legal agreements signed at the beginning. An impact of this is that the test results for periodic testing that's being sent to clients cease to matter; and regardless of the frequency of app usage by customers, testing companies continue this trend. 1 look at the testing economy would tell you that 2.5 Billion Dollars was the money given to off- shoring companies for testing alone in the 3rd quarter of 2013. With new testing companies sprouting all over, this is set double in the next few years. And the testing companies have also started to exploit this. With substandard test cases and haphazard testing, they make hefty profits without providing any value by testing. An indicator of the lack of quality can be seen by the most unimportant projects being handed to the off-shoring testing companies; this can be seen by the very fact that in most of the offshore company cases, not much of business revenue would be lost if the offshore projects are closed down. Secondly, according to the reports of our special correspondent, in the last 5 months, there has been ZERO new product launches in non-Asian markets due to testing done from here. I am unsure if others realize this, but looks like India's testing industry has started to face its worst crisis; my personal thought is that the rest of this year will be gone in people realizing their mistakes and the indus- try auto-correcting itself. What do you think will hap- pen? As I mentioned last month, only time will tell. www.TestingCircus.com March 2014 - 34 -
  • 35. www.TestingCircus.com March 2014 - 35 - Anna Royzman Context-Driven Scholar https://twitter.com/QA_nna Sharath Byregowda A passionate software test professional, Principal Consultant at SQS, London in their agile group. https://twitter.com/sharathb Mike Lyles International speaker, writer & Sr QA Manager in Performance Testing, Test Automation & Service Virtualization (20+ yrs IT). https://twitter.com/mikelyles Joris Meerts Publishing the latest news from the software testing weblogs. Keeper of lists. Testing historian, critic, thinker, member of #DEWT. #Capgemini https://twitter.com/testingref http://Twitter.com/TestingCircus #Testers2Follow
  • 36. www.TestingCircus.com March 2014 - 36 - #FollowUsAtTwitter http://Twitter.com/TestingCircus Testing Circus Testing Circus is a world’s leading English language magazine for software testers and test enthusiasts. Monthly editions since September 2010. #testing Follow us at https://twitter.com/TestingCircus
  • 37. Part 15 - Santhosh Tuppad Santhosh Tuppad specializes in exploratory testing approach and his core interests are security, usability and accessibility amidst other quality criteria. Santhosh loves writing and he has a blog http://tuppad.com/blog. He has also authored several articles and crash courses in the past. He attends conferences and confers with testers he meets. Santhosh is known for his skills in testing and has won the uTest Top Tester of the Year 2010 apart from winning several testing competitions from uTest and Zappers. www.TestingCircus.com March 2014 - 37 - Security Testing Tips One of the challenges in security testing is, setting up the test environment. If you are a small scale organiza- tion and there are no lots of processes, then it could be easy for you to go ahead and setup a test environment like you want, but in the bigger organizations when it has lots of processes it could be hard. There could be network related blockers that you may want to clear (Example: You want to download a software which you want to use for security testing purpose, and that source of download is blocked by the network of the organization). If your test environment is a blocker for you, then I would better recommend to not performing security testing and thereby, you can at least save costs. Isolated network of computers It is important to have a separate network dedicated for security testing. This is because, you do not want to affect the other computers on the network if you down- load some software and it is infected with malware / adware or any malicious thing. No website blockers The network shouldn’t be blocking any website that you want to browse for learning about some hacks or downloading any software to aid your hacking activi- ty. Let your network policy doesn’t end up blocking your learning. Administrator rights for computers & other devices In my experience, I have faced lot of blockers when the computer that was given to me did not provide admin- istrative rights. For changing some of the settings in order test for security, I had to e-mail the infrastructure team to change the settings and that consumed time. Before starting, it is important to ask for a computer with all the rights to change / modify any setting. Installations of software before commencement Make sure that you are ready with installations of all required software before you commence security test- ing activity. This is because, it will save you time if something doesn’t get installed or doesn’t work prop- erly while you are testing for security. So, installing the required software like proxy, burp suite, Wireshark, backtrack, kali linux, mantra browser, nmap and lot of many other tools happens without any hassles. No code changes to be done When testers are testing for security, no code changes should be done. It is important that you have separate environment and not the same which developer uses check in his / her code. And also, security testing needs to be done once functional testing is done and all the bugs reported are fixed. DIY: Test Environment for Security Testing
  • 38. http://www.facebook.com/TestingCircus www.TestingCircus.com March 2014 - 38 - Visit our facebook page to share. Someone else too could enjoy what you're reading right now.
  • 39. Since this is my first article for Testing Circus I’d like to thank all the wonderful contributors for sharing info, because there’s no other way to evolve in the IT industry! Why would we need to run tests on a Remote VM? You don’t actually need to do this, but from my own experience I’ve isolated a couple of scenarios where this can prove vital: a. Having a test-script (or suite) running on a JavaScript-rich page that needs to have complete focus all the time. So, no touching the mouse or keyboard while this is running. If this takes 1hr to run, you’re going to get bored and/or sleepy and let’s not forget you have to justify for that particular hour to your management. b. The VM can run on your local machine (making it not that “remote”), it can run on another machine lodged on a cloud-hosting server (e.g. RackSpace) or in your blade-center sitting neatly in your compa- ny’s basement. Unless the VM is running from your computer’s VBox you’re hogging way less resources by running this remotely. c. You might be running across situations where your script runs smooth on your local MacOS / Windows environment, but, for some weird reason you get weird failures on the CI machine running (in most cases) some Linux distribution. How will you debug this? Where to start? VBox setup and install I’m going to run through the steps needed to build your local Ubuntu VM assuming you have a working Host Machine (I’m currently running Mac OS 10.8.5, but this is 99% valid for Windows as well) a. Download and install VBox from Oracle’s website https://www.virtualbox.org/wiki/Downloads b. Download Ubuntu. I went for 13.04 64bit but you can choose whatever suits you http://www.ubuntu.com/download/desktop c. Start VirtualBox d. Click the NEW pictogram in the left e. Populate the fields according to your need www.TestingCircus.com March 2014 - 39 - - Mihai Sarlea Running Remote Selenium 2 WebDriver Scripts On a VM
  • 43. f. From here on it’s a standard Ubuntu installation the likes of which you can find at http://www.ubuntu.com/download/desktop/install-desktop-latest Configure Ubuntu I’m just going to run through the basics of Ubuntu configuration so you can get your VM up and running ASAP. a. Install OpenJDK-7 and Maven Since I’m using Java to run my Selenium tests, I’ll need to install the Java Development Kit and Maven (in my case) and I’m willing to share this nice trick useful also when setting up a fresh CI based on Ubuntu: This is the sequence that will get you a clean usable OpenJDK on a Deb machine: apt-get remove openjdk* apt-get remove java* apt-get install openjdk-7-jdk apt-get install openjdk-7-* apt-get install maven b. Test the install java –version mvn -v c. Check the IP address and the “visibility” of your host computer ping your_host_computer_ip www.TestingCircus.com March 2014 - 43 -
  • 44. Here’s how I’ve setup the VM in VBox and Ubuntu: Now you should be able to ping to and from the VBox installed Ubuntu VM. www.TestingCircus.com March 2014 - 44 -
  • 45. Setting up SELENIUM 2 on the VM a. Open up a Terminal window and download the Selenium Server Standalone jar wget http://selenium.googlecode.com/files/selenium- server-standalone-2.35.0.jar b. What we’re doing is creating a small Selenium 2 GRID network and that’s why we need to start up a hub here: java -jar selenium-server-standalone-2.35.0.jar -role hub c. Open up a 2nd Termnal window and setup a (single) basic node with default config on the same machine: java -jar selenium-server-standalone-2.35.0.jar -role node -hub http://localhost:4444/grid/register d. Find your VM’s IP address – open up another Terminal window: Ifconfig -a That’s it, all setup! Adding the VM to your Selenium WebDriver Java Framework This is as easy as adding the lines bellow: switch (useThisDriver) { case FIREFOX: aDriver = new FirefoxDriver();//profile); break; case FIREFOX_LOCAL_VM_UBUNTU: DesiredCapabilities capability = DesiredCapabilities.firefox(); aDriver = new RemoteWebDriver(new URL("http://192.168.10.26:4444/wd/hub"), capa- bility); break; DesiredCapabilities capability = DesiredCapabilities.firefox(); driver = new RemoteWebDriver(new URL("http://192.168.1.138:4444/wd/hub"), capability); driver.findElement(…) // whatever you want to do If you’re not running the test as part of a framework, just go ahead and add the lines like this when starting the test: Conclusion I’ve tried to go through every step of setting up a nifty lightweight GRID 2 on a local (or remote) Ubuntu VM. The knowledge is applicable, in much the same way to a Windows VM or any other OS you might need, for the matter. I hope it helps you get up and running with your local *nix VM. Selenium GRID 2 is a very cool feature that can save a lot of time, but we’ll dive more into this in a later arti- cle. Till that time, feel free to check out the documenta- tion Google put up at: https://code.google.com/p/selenium/wiki/Grid2 More on the framework “Driver” class I used at point 6: http://blog.cloudtroopers.com/content/%E2%80%9Cdri ver%E2%80%9D-class-selenium-2-webdriver Mihai Șarlea is an SQA Auto‑ mation Engineer for CloudTroopers.com and sound engineering enthusi- ast. Telecom Engineer by forma- tion, Mihai started flying above radar while working as Simlock Security EU region Specialist for Nokia and, gradually, shifted gears towards QA-ing for Endava Romania on financial projects. For the last year and a half, he's been focusing on automated website testing, continuous integration, light- weight performance scripts and all adjacent tasks at CloudTroopers.com. Outside office hours, Mihai occasionally handles Front of House Engineering for various clubs in Cluj-Napoca, Romania and volunteers to help the less fortunate artists record decent, career kick- starter, materials. Feel free to check out his LinkedIn profile: http://ro.linkedin.com/pub/mihai-sarlea/40/924/b92 www.TestingCircus.com March 2014 - 45 -
  • 46. Thinking what to do with your career? We can HELP www.TalentPlusPlus.com www.TestingCircus.com March 2014 - 46 -
  • 47. Tools Journal Testing Corner Testing Circus in exclusive partnership with Tools Journal (http://toolsjournal.com) presents the Tools Journal Testing Corner. 1. Test Automation: New Venues, New Challenges 2. SmartBear TestComplete 10 Brings In Support For Automated Mobile Testing 3. Squish Coco Brings In Support For Code Coverage Of C# & Tcl Code "Thank You" to all subscribers who have joined us and have been giving us some good feedback. Most of all Thank you "Testing Circus" for providing us a good platform to shout our views . www.TestingCircus.com March 2014 - 47 - About Us: A start up journal and aspiring social community with an aim to gain and distribute knowledge on software tools and concepts in Testing, Agile, Cloud, Mobile and Enterprise Integration. http://www.toolsjournal.com With over 500 products listed with quality articles, product owner interviews, we are moving swiftly to launch product editorial/user reviews, community module in next 2 months. Connect With Us @toolsjournal www.toolsjournal.com www.facebook.com/toolsjournal Thank You Testing Circus & Tools Journal
  • 48. While most of the test automation engineers are aware of the many pitfalls of test automation; there are some associated to the automation planning & design considerations. There are some widely used automation frameworks, & hybrid framework tops them all. While it’s a good practice to use hybrid framework with benefits taken from all the conventional frameworks like keyword, data driven etc…; the real trick in coming up with a good design is to cater to all the project needs while keeping the maintenance cost low, and the learning curve short. While there is no single way to implement this solution but again, it’s no rocket science either. One may have to consider various factors that automation framework has to handle – technical & non-technical. On technical side one may have to consider the level of complexity, abstraction-levels, so that the front-end whether it’s a GUI or an excel sheet needs to be easy to use & intuitive at the same time. While one should use excel file for easy readability, overusing it may lead to slow execution – contradicting the very reason automation is used e.g. keyword-driven framework using excel file has its benefits, it makes execution bit slower, each read and write to-from excel file may add up to huge delays by the time automation suite is completed. Extensive use of excel or any other file format may lead to bulky automation leading to slow execution & having its own weight to pull it down. While it is good idea to give maximum control (if not all) to the executor to manipulate suite content , it requires good judgmental call on the skill set of executor , reusability & degree of flexibility that needs to be provided so that its benefits are maximized. Though there have been good amount of focus given lately on application performance, automation performance is another area which is unexplored but said that, doesn’t mean we can overlook the automation execution speeds. · Terminology like Library-architecture etc. is something that is intrinsic to any software project whether it is automation or development, so it becomes really necessary, that we form meaningful library functions/methods, keep them light, reusable and ensure proper error handling. With the advent of various automation tools it becomes really important to keep a track of its advantages & disadvantages; before implementing the same. · With loads of new browsers and devices mushrooming in - Cross-browser & cross-device testing is something which can’t be avoided. But again each browser/device has its own engine/mechanism to handle web elements /objects. Taking an eye off such mechanisms can lead to unexpected results, and automation performance issues. Many times failures in automation needn’t be because of scripting or application alone, browsers also have their own drawbacks which lead to automation script failures. While various studies depict various statistics, one major thing that comes up is IE seems to have many issues when then comes to test automation, such statistics need to be looked at before commencing automation. · Object identification/locator mechanisms also play key role. Those who are well versed with www.TestingCircus.com March 2014 - 48 - Test Automation: New Venues, New Challenges
  • 49. automation testing, don’t have to be told that, each mechanism has its own benefits, like one may be robust mechanism, another may be easy to use e.g. name, id as locators; thereby selecting a mechanism which fits well in all the areas is preferred. · With client side technologies like Silverlight, HTML5, JavaScript etc. dominating the web- pages, it becomes really critical to take them into consideration before committing to the client. Though the pages may appear normal, the tool, browser may have issues to work with. Thereby it is always advisable to do a thorough POC, before dwelling deeper into full-fledged automation. · Like in selenium, it is quite obvious to start automation using Firefox and committing to client on providing complete solution, what further is required is the foresight into client’s future requirements like cross-browser testing, mobile device testing. · It is always recommended to think of the breadth during planning & design phase & during implementation focus on individual automation functionality, creating a self- contained unit, preferably reusable. An incremental approach is highly advised - each automation component or test script needs to be well tested on various browsers & devices before moving to next level. This approach would avoid surprises that come with big-bang approach. (It is advisable to avoid big-bang approach, wherein various automation components are clubbed together to fulfill the requirements. ) · With extensive usage of open source technologies & tools (like selenium), it becomes really important to create a proper test environment, wherein various versions of the each tool, process or platform is well documented. Any change in one may lead to automation failure. · Integration with various other tools &/or application/platform compatibility also is critical, as slightest change in a version may impact automation framework e.g. which version of testlink (test management tool) integrates with which version of selenium or which version of UFT supports which browser & version. Various configurations & versions – inclusive of auto-updates are to be thought about & it is really important that each step is taken carefully to avoid last-minute surprises. · Requirement to automate huge number of test cases, and to get proper ROI, Distributed testing, is fast catching up. Various machines with various configuration, pose a major challenge to distributed automation testing; a single script to be executed for - cross-browser , cross-platform, cross-device each having their own intricate setup (including java version, OS version – service packs installed etc..) – each system depending on the system below it, creating a delicate balance for things to work : eventually all this can add-up to unimaginable permutation & combinations making it mandatory to document the details. Automation is no more limited to a single machine or a single tool, but ranges from various tool integration to its implementation on various machines & browsers, posing new technological challenges to be faced with advent of new versions, compatibility issues, configuration issues making it mandatory to foresee all the major unforeseen pitfalls to be addressed & risks to be averted or mitigated, ensuring successful automation solution delivery. www.TestingCircus.com March 2014 - 49 -