crowd sourced testing 101 - explains crowd testing in the context of crowd sourcing. Covers the various models, the ecosystem and the applicability of crowd sourced testing in the various stages of product/application development.

1. Basics of crowd testing
Crowd Testing Meetup, Bangalore
Jan, 2014

2. Agenda
•
•
•
•
What is crowd sourcing?
Crowd Sourced Testing
The ecosystem
The $$ aspect

3. What is the “CROWD SOURCING”?
Put Crowd Testing in perspective

4. “CROWD SOURCING”
noun ˈ d-ˈ r-siŋ
krau̇ sȯ
Code, Testing, Funding, Trans
lation, Micro Tasks, etc.
the practice of obtaining needed services, ideas, or
content by soliciting contributions from a large group of
people and especially from the online community rather
than from traditional employees or suppliers
First Known Use: 2006
source: Merriam-Webster

5. Crowdsourcing Types
source: Crowd Sourcing Umbrella

6. Crowdsourcing is seen in…
Funding
• Project based
• All or nothing
Labour
• Virtual pool
• On-demand
• Semi-managed
Knowledge
• Mass
collaboration
• Open & free
Non-exhaustive list of well-known sites
* Logos & Trademarks are owned by their respective companies & organizations
Communities
• Like-minded
users
• Niche areas
• Enterprise
driven

7. Everything cannot be crowd sourced…
When
Example
Need constant iterations
• Constantly improving content on a website
• Refining a company logo
Outcome has to be managed
• Developing a complex game
• Test cases have to be documented
Delivery has to be guaranteed
• A critical module in a large software product
• Marketing campaigns in holiday seasons
Evolving requirements
• Prototyping on a new travel website
• Finding a use for a new type of adhesive
Exclusivity required
• A non-reusable logo for your company
• A vase design that defines your brand

8. Crowdsourced Testing
aka Crowd Testing

9. Mechanics of Crowd Testing
Web site/application, mobile app
$$
Comments, Defects
$$
$$
1
Tester
Tester
Contests
Hourly
Outcome
Tester
Curated / Managed

10. Crowd Testing in the Testing space
Release
Usage
Beta
Implementation
Functional
Integration
Alpha
User
Acceptance
System
Unit Testing
Regression
Performance
- Ideal for crowd testing
Requirements
Usability Testing
Prototype Testing

11. Players in Crowd Testing
Usability
Functional
Beta/User
* Logos & Trademarks are owned by their respective companies & organizations
Non-exhaustive list

12. What are the cost benefits?
Element
Crowd Testing
Outsourced
In-house
# of testers
-
2.5
2
Cost per tester (in US$)
-
800
1000
0.75
1
1.5
Total Cost (approx.)
$1500
$2000
$3000
Difference
-50%
-33%
# of defects
100
Cost per defect (in US$)
15
Duration
-
The above assumes testing costs in India for the local Indian market
# of defects assumed are real quality defects
In outsourced model, additional 0.5 is assumed to be the cost of the coordinator from the vendor
Outsourced model assumes a Time & Material contract
Duration of the project has direct cost implications for outsourced testing or in-house

13. Future of Crowd Testing
Truly crowd
managed
Technology
supported
Manually
Curated
-
-
Now
Extension to outsourcing
Cheaper testers’ cost
Poor quality
Technology enabled
Faster quality output
Outcome based
-
Self regulated
Result oriented
Embedded into larger crowd models

14. Thank you
reach@strata9.com