Test Design techniques: Structured and Experienced-based techniques

1. Test Design techniques: Structured and Experienced-based techniques Author: Khuong Nguyen

2. Structured based testing techniques Cyclomatic complexity Experienced based testing techniques Choosing appropriate testing techniques Sample exam Agenda

3. Understand and apply common Structured-Based Testing techniques Know how to calculate Cyclomatic Complexity Understand Experienced-Based Testing techniques Learn to choose appropriate testing techniques in specific circumstances Objectives

4. Software testing foundations Certified Tester Foundation Level Syllabus 2007 Software Testing Dictionary BS7925-2: Component Testing References

5. Structure-based techniques

6. Structure-based testing/white-box testing is based on an identified structure of the software or system It is a method for writing a set of white-box test cases that exercise the paths in the code How to measure thoroughly the test cases exercise the code Introduction

7. Test coverage measures in some specific way the amount of testing performed by a set of tests. Wherever we can count things and can tell whether or not each of those things has been tested by some test, then we can measure coverage What is test coverage?

8. Statement testing and coverage Decision testing and coverage Other structure-based techniques Basic white box test case design techniques

9. In component testing, statement coverage is the assessment of the percentage of executable statements that have been exercised by a test case suite. (TRUE OR FALSE) Statement testing derives test cases to execute specific statements, normally to increase statement coverage. Every statement is requested to be executed at least one Definition of statement coverage

10. Translate the source code to control flow graph Defines nodes(Statement) and control flow between statements(edges) Conditional statements(IF, CASE) and loops(WHILE, FOR) have more than one edges going out Verify that each statement have been executed Steps

11. Examples IF 1 IF 2 WHILE 3 4 6 ENDIF 7 ENDIF 1 If (a>0) then 2 If (b<1) then 3 While (c<2) 4 c++; 5 End while End IF 7 End IF 5 Node, statement Edge, Control flow

12. Statement coverage IF 1 IF 2 WHILE 3 4 6 ENDIF 7 ENDIF 100%SC: All nodes can be reached by a single test case 1, 2, 3, 4, 5, 6, 7 5

13. It is a very weak criterion Value of the technique: Unreachable source code (dead code) can be detected Empty ELSE-parts are not considered Statement Coverage Coverage is measured by using test tools

14. What is minimum number of test cases required for full statement coverage? 1 test for statement coverage 2 tests for statement coverage 3 tests for statement coverage Read P Read Q IF P+Q > 100 THEN Print “Large” ENDIF If P > 50 THEN Print “P Large” ENDIF Exercise

15. Decision coverage, related to branch coverage, is the assessment of the percentage of decision outcomes (TRUE and FALSE) Require every decision outcome: all possible cases-statement Decision testing is a form of control flow testing as it generates a specific flow of control through the decision points Definition of decision/branch coverage

16. Translate the source code to control flow graph Defines nodes(Statement) and control flow between statements(edges) Conditional statements(IF, CASE) and loops(WHILE, FOR) have more than one edges going out Verify that all possible branches of the control flow are tested Steps

17. It is a stronger criterion than statement coverage Be able to detect missing statement in empty branches Decision coverage is stronger than statement coverage: 100% decision coverage guarantees 100% statement coverage, but not vice versa. Decision coverage requires test of every decision outcome: both THEN and ELSE in the IF-statement all possibilities for the CASE-statement and fall-through case both execution of the FOR loop and the bypass Decision/branch Coverage

18. Examples IF 1 IF 2 WHILE 3 4 6 ENDIF 7 ENDIF 1 If (a>0) then 2 If (b<1) then 3 While (c<2) 4 c++; 5 End while 6 End IF 7 End IF 5 Node, statement Edge, Control flow

21. Data flow Testing Branch Condition Testing Branch Condition Combination Testing Modified Condition Decision Testing LCSAJ Testing Other structure-based test techniques

22. There are stronger levels of structural coverage beyond decision coverage (condition coverage and multiple condition coverage) The concept of coverage can also be applied at other test levels (e.g. at integration level) where the percentage of modules, components or classes that have been exercised by a test case suite could be expressed as module, component or class coverage Tool support is useful for the structural testing of code The basis of white box techniques is the source code. Adequate test case design techniques chosen depending on the complexity of the program structure Other structure-based test techniques (cont.)

23. Testing contains the combination of different techniques The expected risk guides to select testing techniques and the intensity of execution Basis of the selection of the white box technique is the structure of the test object Which techniques should be chosen?

24. Cyclomatic Complexity

25. Cyclomatic complexity or conditional complexity: Was developed by McCabe Is the software metric (measurement) to measure the complexity of the program’s source code Definition

26. M = E − N + 2P M = cyclomatic complexity E = the number of edges (links) of the graph N = the number of nodes of the graph P = the number of connected components Number of predicates + 1 How to calculate

27. Examples of cyclomatic complexity E=1, N=2, P=1 M=1-2+2=1 E=4, N=4, P=1 M=4-4+2=2 E=2, N=4, P=2 M=2-4+4=2 E=4, N=5, P=1 M=4-5+2=1

28. Cyclomatic complexity of programming constructs 1 1 if E then 2 A else B 4 C 2 3 4 E = 4, N = 4, P =1 M = 2

29. C

30. C

31. Predicate notes P = ? Complexity C = ?

32. Predicate notes P = ? Complexity C = ?

33. Predicate notes P = ? Complexity C = ?

34. Complex systems are hard to understand hard to change hard to reuse McCabe found that modules with a cyclomatic complexity greater than 10 were hard to test and error prone. Look for procedures with high cyclomatic complexity and rewrite them, focus testing on them, or focus reviewing on them. Metrics view

35. Experience-based techniques

36. Experienced-based test design technique: Procedure to derive and/or select test cases based on the tester’s experience, knowledge and intuition The techniques can be useful in identifying special tests not easily captured by formal techniques, especially when applied after more formal approaches Definition

37. Tester select test cases to uncover expected problems based on the skills, experienced and knowledge of the tester The term of error guessing is used very often in practice A commonly used experienced-based technique is error guessing Introduction

38. Exploratory testing Error guessing Basic experience-based test techniques

39. Error guessing is a test design technique where the experience of the tester is used to anticipate what defect might be present in the system under test. Testers design tests specifically to expose errors Error Guessing - Definition

40. Error guessing is a technique that should be used as a complement to other formal techniques Testers anticipate defects based on their experience, skills Knowledge in developing similar applications and using similar technologies are also used when designing test cases. “Guess” where errors happened based on intuition and experience of tester. Error guessing

41. These defect and failure lists can be built based on experience, available defect and failure data, and from common knowledge about why software fails. Error guessing

42. Exploratory testing is test design technique where the tester actively controls the design of the test as those tests are performed and uses information gained while testing to design new and better test Exploratory testing is exploring, finding out about the software: What it does What it doesn’t do What works What doesn’t work Definition of Exploratory testing

43. Exploratory testing is a hands-on approach in which testers are involved in minimum planning and maximum test execution Activities in exploratory testing is concurrent: Test planning Test design Test execution Test logging Learning Main features of exploratory testing

44. Result of one test case influence the design and execution of further test cases During testing, a “mental” model of program under test is created. It contains: How the program works How it behaves How it should behave Focus on finding further aspect/behavior of the program. Main features of exploratory testing

45. Planning: involve the creation of the test charter that includes: Test charter Short declaration of scope Time-boxed of effort Objectives and possible approaches to be used Test design and test execution are performed parallel: Do not have formal documents: test conditions, test cases, test scripts… Some notes written during exploratory testing session Test logging is undertaken as test execution is performed. The key aspect of exploratory testing is learning about the software, its strengths and weaknesses How to implement

46. Approach Execute few test cases Analyze results Consider “special” to next test cases Knowledge about the test object under test is collected

47. Possible elements of the test object are “explored” to decide which parts will be tested Result of Exploratory testing helps to determine which test techniques can be applied if there is time left The test of “test charter” should not take more than one or two hours of uninterrupted test time Benefit

48. The technique is applicable when: There no poor specification The time is severely restricted because it uses much less time than other techniques It can serves to complement formal testing It can serve to check on the test process, to help ensure that the most serious defects are found. When we use exploratory testing

49. Why? With which goal is the test run? What is to be tested? How? Which testing method should be used? Which problems should be found? Common questions for executing test charter

50. Choosing test techniques

51. What are the pros and cons of: Specification-based testing techniques Structure-based testing techniques Experience-based testing techniques What is the best practice? When to use what?

52. The choice of which test techniques to use depends on a number of factors: the type of system regulatory standards customer or contractual requirements level of risk, type of risk test objective, documentation available knowledge of the testers time and budget development life cycle, use case models previous experience of types of defects found. Which techniques should be chosen?

53. Testing contains the combination of different techniques The critically and the expected risk guide to select testing techniques and the intensity of execution Basis of the selection of the white box technique is the structure of the test object Which techniques should be chosen?

54. Thank You KhuongNguyen, Senior Quality Control Engineer Post graduate student, Australian Business School, Funds Management, UNSW, 2032, AustraliaEmail: khuong0602@gmail.com; op_khuong_nguyen@yahoo.com