The document discusses different types of testing performed at various stages of the software development lifecycle, including unit testing, integration testing, system testing, and acceptance testing. It provides details on the goals, test bases, and typical participants for each type of testing. The document also covers topics like integration strategies, performance testing, usability testing, and maintenance testing.
Automating Google Workspace (GWS) & more with Apps Script
Testing in the lifecycle
1. TESTING IN THE LIFECYCLE
Hoang V. Nguyen
startnewday85@gmail.com
startnewday85.blogspot.com
Dept. of Computer Science – FIT - HUA
Tester Training Course
4. User Acceptance
requirements Testing
System
System Testing
requirements
Integration
Global Design
Testing
Detailed Design Unit Testing
implementation
Tester Training Course 2011
5. Also know as component, module, program testing
Each “part” is tested in isolation
lowest level and most thorough look at detail
Goals:
• to ensure that the code written for the unit meets its specification,
prior to its integration with other units
• both functional and non-functional characteristics
Test bases:
• component requirements
• detailed design
• the code
Tester Training Course 2011
6. Who do it?
• usually done by programmer
Reports
• Defects are typically fixed as soon as they are found, with out
formally recording the incidents found
Tools:
• Programmer often writes pieces of code for unit test
• Mock frameworks
• Dependency Injection and IoC Containers
• Unit test frameworks
Tester Training Course 2011
7. Programmers:
• have to create pieces of code are separately testable
Test Driven Development
• An approach to software development using Unit Test
• Test –first approach
• is an iterative approach to unit test
• is good in practice
A A Driver
• Stubs and Drivers
B Stubs B
Tester Training Course 2011
8. more than two components/systems
Goals:
• discover defects in interface and interaction between
components/systems
• what the set can perform that is not possible individually
• both functional and non-functional characteristics
Test bases:
• software and system design
• software architecture
• workflows and use-cases
Tester Training Course 2011
9. There may be more than one level of integration
testing
• component integration testing
• system integration testing
Who do it?
• Developer
• Designer
• Independent tester
Tester Training Course 2011
11. In theory
• if we have already tested components why not just combine them
all at once? Wouldn’t this save time?
• based on false assumption of no faults
In practice: difficult to isolate any errors found
• takes longer to locate and fix faults
• re-testing after fixes more extensive
==> don’t save time
Tester Training Course 2011
12. In theory
• start with a tested component/system
• increment: add a tested component/system and test it
• baselines
Advantages
• easier fault location and fix
• easier recovery from disaster/problems
Some types
• top-down
• bottom-up
• functional
Tester Training Course 2011
13. Baselines
• baseline 0: a tested component a
a
• baseline 1: a+b b c
• baseline 2: a+b+c
• baseline 3: a+b+c+d d e f g
• …
Need to call to lower level h i j k l m
components not yet integrated
n o Stubs
Stubs: simulate missing
components
Tester Training Course 2011
14. Advantages:
• critical control structure tested first and most often
• can demonstrate system early (show working
menus)
Disadvantages:
• needs stubs
• detail left until last
• may be difficult to "see" detailed output (but
should have been tested in component test)
• may look more finished than it is
Tester Training Course 2011
15. Baselines Drivers
• baseline 0: component n
a
• baseline 1: n+i b c
• baseline 2: n+i+o
• baseline 3: n+i+o+d d e f g
• …
Needs driver to call the h i j k l m
base line configuration
Also needs stubs for some n o
baselines
Drivers: test harness/scaffolding
Stubs
Tester Training Course 2011
16. Advantages:
• lowest levels tested first and most thoroughly (but should
have been tested in unit testing)
• good for testing interfaces to external environment
(hardware, network)
• visibility of detail
Disadvantages
• no working system until last baseline
• needs both drivers and stubs
• major control problems found last
Tester Training Course 2011
19. Baselines
• baseline 0: component a
a
• baseline 1: a+b b c
• baseline 2: a+b+d
• baseline 3: a+b+d+h d e f g
• …
Need Stubs h i j k l m
Shouldn’t need drivers(if top
down) n o
Tester Training Course 2011
20. Advantages:
• control level tested first and most often
• visibility of detail
• real working partial system earliest
Disadvantages
• needs stubs
Tester Training Course 2011
21. Thread Integration
order of processing some event
a
determines integration order
interrupt, user transaction b c
minimum capability in time
d e f g
advantages:
• critical processing first h i j k l m
• early warning of
performance problems
n o
disadvantages:
• may need complex drivers and stubs
Tester Training Course 2011
22. Integration Guidelines
minimise support software needed
integrate each component only once
each baseline should produce an easily
verifiable result
integrate small numbers of components at once
• one at a time for critical or fault-prone components
• combine simple related components
Tester Training Course 2011
23. integration should be planned in the
architectural design phase
the integration order then determines the
build order
• components completed in time for their
baseline
• component development and integration
testing can be done in parallel - saves time
Tester Training Course 2011
24. Question
Which of the following statements about
component testing is not true?
Component testing should be performed by
developer
Component testing is alse know as isolation or
module testing
Component testing should have completion criteria
planned
Component testing does not involve regression
testing
Tester Training Course 2011
25. Question
Which of the following combinations correctly describes a
valid aproach to component testing
i. Functional testing of the component in isolation
ii. Structure-based testing of the code with out recording
incidents
iii. Automated tests that rare run until the component
passes
iv. Functional testing of the interfaces between modules
i and ii
i, ii and iii
iii
ii and iv
Tester Training Course 2011
26. Question
Testing activity which is performed to expose
defects in the interfaces and in the interaction
between integrated components is:
System Testing
Component Testing
Integration Testing
Unit Testing
Tester Training Course 2011
27. Question
A common test technique during
component test is
Statement and branch testing
Usability testing
Security testing
Performance testing
Tester Training Course 2011
28. Question
The process starting with the terminal modules
is called
Top-down integration
Module integration
Bottom-up integration
None of the above
Tester Training Course 2011
29. Question
Why is incremental integration preferred
over big bang integration?
Because incremental integration has better early
defects screening and isolation ability
Because big bang integration is suitable only for real
time applications
Incremental integration is preferred over big bang
integration only for bottom-up development model
Because incremental integration can compensate
for weak and inadequate component testing
Tester Training Course 2011
30. Last integration step
end-to-end perspective
Goals:
• discover failure in behaviors of whole system in environment
• both functional and non-functional characteristics
Test bases
• System and software requirement specifications
• Use cases
• system, user and operation manuals
Who do it
• often and should be independent test team
Tester Training Course 2011
32. Performance Tests
Timing Tests
• response and service times
• database back-up times
Capacity & Volume Tests
• maximum amount or processing rate
• number of records on the system
• graceful degradation
Endurance Tests (24-hr operation?)
• robustness of the system
• memory allocation
Tester Training Course 2011
33. Multi-User Tests
Concurrency Tests
• small numbers, large benefits
• detect record locking problems
Load Tests
• the measurement of system behaviour under
realistic multi-user load
Stress Tests
• go beyond limits for the system - know what will
happen
• particular relevance for e-commerce
Tester Training Course 2011
34. Usability Tests
messages tailored and meaningful to (real)
users?
coherent and consistent interface?
sufficient redundancy of critical information?
within the "human envelope"? (7±2 choices)
feedback (wait messages)?
clear mappings (how to escape)?
Who should design / perform these tests?
Tester Training Course 2011
35. Security Tests
passwords
encryption
hardware permission devices
levels of access to information
authorisation
covert channels
physical security
Tester Training Course 2011
36. Configuration and Installation
Configuration Tests
• different hardware or software environment
• configuration of the system itself
• upgrade paths - may conflict
Installation Tests
• distribution (CD, network, etc.) and timings
• physical aspects: electromagnetic fields, heat,
humidity, motion, chemicals, power supplies
• uninstall (removing installation)
Tester Training Course 2011
37. Reliability / Qualities
Reliability
• "system will be reliable" - how to test this?
• "2 failures per year over ten years"
• Mean Time Between Failures (MTBF)
• reliability growth models
Other Qualities
• maintainability, portability, adaptability, etc.
Tester Training Course 2011
38. Back-up and Recovery
Back-ups
• computer functions
• manual procedures (where are tapes stored)
Recovery
• real test of back-up
• manual procedures unfamiliar
• should be regularly rehearsed
• documentation should be detailed, clear and
thorough
Tester Training Course 2011
39. Documentation Testing
Documentation review
• check for accuracy against other documents
• gain consensus about content
• documentation exists, in right format
Documentation tests
• is it usable? does it work?
• user manual
• maintenance documentation
Tester Training Course 2011
40. Final stage of validation
Goals
• To provide the end users with confidence
that the system will function according to
their expectations
Test bases
• Requirement s specification
Who do it
• Customers/users
• Also can include testers
Tester Training Course 2011
42. Done by users
To check that the system meets
their business needs
Can include
• Factory acceptance testing
• Site acceptance testing
Tester Training Course 2011
43. Also called operational readiness testing
To check that the processes and procedures
are in place to allow the system to be used
and maintained
Include checking:
• Back-up facilities
• Procedures for disaster recovery
• Training for end users
• Maintenance procedures
• Data load and migration tasks
• Security procedures
Tester Training Course 2011
45. The criteria for accepting a system can
documented in a contact
To check that these criteria have been met
• Not what the user wish they had asked for!
• This system, not wish system
Tester Training Course 2011
47. Similarities:
• When software is stable
• To collect comments
fault found
how the product meets their expectations
improvement suggestions
Differences :
• Alpha testing is done on developer site
• Beta testing is done on customer site
Tester Training Course 2011
48. Question
System Integration testing should be
done after
Integration testing
System testing
Unit testing
Component integration testing
Tester Training Course 2011
49. Question
When is beta testing?
Testing performed by potential customers at the
developers location.
Testing performed by portential customer at their
own locations.
Testing performed by product developers at the
customer ’s location
Testing performed by product developers at their
own locations.
Tester Training Course 2011
50. Question
Which testing is used to verify that the
system can perform properly when internal
program or system limitations have been
exceeed
Stress testing
Load testing
Performance testing
Volume testing
Tester Training Course 2011
51. Additional features being required
The system being migrated to a new
operating platform
The system being retired
Planned upgrade to COTS-based system
New faults being found requiring fixing
(these can be ‘hot fixes’)
Tester Training Course 2011
53. To ensure that software system still
works correctly (both new and old
characteristics)
We have to:
• Breath test to establish overall confidence
• Depth test to investigate changes and critical areas
Predominantly regression testing
Tester Training Course 2011
54. What to test in maintenance testing
Test any new or changed code
Impact analysis
• what could this change have an impact on?
• how important is a fault in the impacted area?
• test what has been affected, but how much?
most important affected areas?
areas most likely to be affected?
whole system?
The answer: “It depends”
Tester Training Course 2011
55. Question
Which of the following is correct?
Impact Analysis assesses the effect on the system of
a defect found in regression testing
Impact Analysis assesses the effect of a new person
joining the regression test team
Impact Analysis assesses whether or not a defect
found in regression testing has been fixed correctly
Impact Analysis assesses the effect of a change to
the system to determine how much regression to do
Tester Training Course 2011
56. Question
Testing where in we subject the target of the
test, to varying workloads to measure and
evaluate the performance behaviors and
ability of the target and of the test to
continue to function properly under these
different workloads.
Unit testing
Load testing
Usability testing
Intergration testing
Tester Training Course 2011
57. Question
Methodologies adopted while
performing maintenance testing:
Breadth Test and Depth Test
Re-Testing
Confirmation Testing
Sanity testing
Tester Training Course 2011
58. Question
What is the normal order of activities in
which software testing is orgranized?
Unit, integration, system, validation
System, integration, unit, validation
Unit, integration, validation, system
None of the above
Tester Training Course 2011
59. Question
Beta testing is performed at developing
organization’s site where as Alpha testing is
performed by people at their own locations.
True
False
Tester Training Course 2011
60. Question
Beta testing is
Performed by customers at their own site
Performed by customers at their software
developer’s site
Performed by an independent test team
Useful for test bespoke software
Tester Training Course 2011
61. Question
Impact Analysis helps to decide
Exit Criteria
How much regression testing should be done
How many more test cases need to written
Different tools to perform regression testing
Tester Training Course 2011