Learn more about Quality Assurance & Software Testing

1. Where are we? Specification Design Testing Code & Test Quality Assurance

4. What Testing Shows? errors requirements conformance performance an indication of quality © SEPA

5. SW Testing starts with coding But, Quality Assurance is a continuous process Testing is an incremental process. Starts from small modules and integrated to higher level modules. But, planning the testing process should be started early in system engineering state. When Testing Starts?

6. When Testing Starts? SystemEngineering Requirement Analysis Design Coding Test Plan Unit Test Integration Test ValidationTest SystemTest plans plans System test plan. SW+HW Function performance etc. BB some GB BB GB Acceptance Test Developer Tester Developer Tester Tester User

7. Quality Assurance: monitoring and improving the whole SW development process. Includes, Verification : Are we building right SW? and Validation : Did we build the SW right? Requirement Specification SW Verify Validate walkthrough, inspection static, dynamic Testing is part of SW Quality Assurance essential

8. Testing is the process of exercising a program with the intent of finding errors within set time and effort prior to delivery to the end user. Not to show that a SW is working correctly! OK! Why Testing such Essential? Testing not only executing

12. Exhaustive Testing There are 10 14 possible paths! If we execute one test per millisecond, it would take 3,170 years to test this program!! © SEPA loop < 20 X

16. Why need independent tester? Developer Independent tester Advantage : Already knew the system. but, will test &quot;gently&quot; and, is driven by &quot;delivery&quot; Disadvantage : Must learn the system but, will attempt to break it and, is driven by quality © SEPA

21. Two Important Testing Techniques Black-Box : To test the functionality, behavior and performance against its specification. Implementation logic is not necessary White-Box : To test the correctness of the internal logic of a module. Therefore, Implementation detail is necessary © SEPA boundary-value equivalent Partition white-box methods black-box methods basis-path control structure

22. Black-Box Testing Functional, Behavior, Performance, requirements (expected output) events input output Internal logic not required © SEPA

23. White-Box Testing Based on “coverage” : goal is to ensure that all the statements and conditions have been executed at least once … statement edge path To test the internal logic © SEPA

24. Why Cover? logic errors and incorrect assumptions are inversely proportional to a path's execution probability we often believe that a path is not likely to be executed; in fact, reality is often counter intuitive typographical errors are random; it's likely that untested paths will contain some © SEPA

25. PATH coverage not feasible There are 10 14 possible paths! If we execute one test per millisecond, it would take 3,170 years to test this program!! © SEPA loop < 20 X

26. Select Most Suitable Paths - but how? Linearly Independent Paths : each new path introduces at least one different executable or simple conditional statement OR a new edge © SEPA This also ensures statement and edge coverage : every statement (and edge) is executed at least once loop < 20 X

36. Cyclomatic Complexity A number of industry studies have indicated that the higher V(G), the higher the probability or errors. V(G) modules modules in this range are more error prone © SEPA

37. Basis Path Testing- Independent Paths Next, we derive the independent paths: Since V(G) = 4, there are four paths Path 1: 1,2,4,7,8 Path 2: 1,2,4,5,7,8 Path 3: 1,2,4,5,6,8 Path 4: 1,2,3,2,4,.….,8 Finally, we derive test cases to exercise these paths. 5 2 1 3 4 8 6 7

38. Basis Path Testing- TEST CASES Path 1: 1,2,4,7,8 N=0 , target=2 ,( A is empty ) Path 2: 1,2,4,5,7,8 N=1, target =2, A[1]= 4( any number > 2) Path 3: 1,2,4,5,6,8 N=1, target =2, A[1]= 2 Path 4: 1,2,3, 2, 4,.….,8 N=1, target =2, A[1]= 1 ( any number <2) But, more general cases are better; FALSE FALSE TRUE FALSE 5 2 1 3 4 8 6 7

42. Loop Testing Nested Loops Concatenated Loops Unstructured Loops Simple loop © SEPA

43. Loop Testing: Simple Loops © SEPA Minimum conditions—Simple Loops 1. skip the loop entirely 2. only one pass through the loop 3. two passes through the loop 4. m passes through the loop m < n 5. (n-1), n, and (n+1) passes through the loop where n is the maximum number of allowable passes

45. Loop Testing: Nested Loops © SEPA Start at the innermost loop. Set all outer loops to their minimum iteration parameter values. Test the min+1, typical, max-1 and max for the innermost loop, while holding the outer loops at their minimum values. Move out one loop and set it up as in step 2, holding all other loops at typical values. Continue this step until the outermost loop has been tested. If the loops are independent of one another then treat each as a simple loop else* treat as nested loops for example, the final loop counter value of loop 1 is used to initialize loop 2. Nested Loops Concatenated Loops

52. Equivalence Partitioning user queries mouse picks output formats prompts FK input data © SEPA

53. Sample Equivalence Classes user supplied commands responses to system prompts file names computational data physical parameters bounding values initiation values output data formatting responses to error messages graphical data (e.g., mouse picks) data outside bounds of the program physically impossible data proper value supplied in wrong place Valid data Invalid data © SEPA

62. Boundary Value Analysis &quot;Bugs lurk in corners and congregate at boundaries ...&quot; Boris Beizer WHY ? - unknown Select test cases to cover all the boundary values of input and output data elements and data structures © SEPA

63. Boundary Value Analysis user queries mouse picks output formats prompts FK input data output domain input domain © SEPA

68. Testing is an incremental process SystemEngineering Requirement Analysis Design Coding Test Plan Unit Test Integration Test ValidationTest SystemTest plans plans System test plan. SW+HW Function performance etc. BB some GB BB GB Acceptance Test Developer Tester Developer Tester Tester User

69. Unit Testing module to be tested test cases results software engineer interface local data structures boundary conditions error handling paths global data effects independent paths loops, conditions ©SEPA

70. Unit Test Environment Module stub stub driver RESULTS test cases not stand-alone Stubs and drivers are overheads and should be small enough to throw away after their use ©SEPA

74. Top Down Integration top module is tested with stubs stubs are replaced one at a time, &quot;depth first&quot; as new modules are integrated, some subset of tests is re-run A B C D E F G Regression test ©SEPA

75. Bottom-Up Integration drivers are replaced one at a time, &quot;depth first&quot; A B C D E F G as new modules are integrated, Regression test some subset of tests is re-run. May be automated ©SEPA

76. Cluster-based Integration Worker modules are grouped into builds and integrated A B C D E F G cluster ©SEPA

77. Integration Example- Case study processDetail ProcessReporter getProcessDetail determineEndDetail displayProcessRepor t getStartDetail getDuration addTime addDate getStartTime leap getStartDate daysInMonth processReport A cluster for validation

79. High Order Testing validation test system test Acceptance tests alpha and beta test other specialized testing ©SEPA

82. Acceptance Tests Developer’s site customer Alpha Test Beta Test Customer’s site ©SEPA By customers in their normal environment developer customer

83. Debugging: A Diagnostic Process ©SEPA

84. The Debugging Process test cases results Debugging suspected causes identified causes corrections regression tests new test cases ©SEPA

85. Debugging Effort time required to diagnose the symptom and determine the cause time required to correct the error and conduct regression tests ©SEPA

86. Symptoms & Causes symptom cause symptom and cause may be geographically separated symptom may disappear when another problem is fixed cause may be due to a combination of non-errors cause may be due to a system or compiler error cause may be due to assumptions that everyone believes symptom may be intermittent/irregular ©SEPA

87. Consequences of Bugs damage mild annoying disturbing serious extreme catastrophic infectious Bug Type Bug Categories: function-related bugs, system-related bugs, data bugs, coding bugs, design bugs, documentation bugs, standards violations, etc. ©SEPA

88. Debugging Techniques brute force / testing Backtracking Cause elimination by Binary partition induction deduction deduction deduction ©SEPA

89. Debugging: Final Thoughts Don't run off half-cocked, think about the symptom you're seeing. Use tools (e.g., dynamic debugger) to gain more insight. If at an impasse, get help from someone else. Be absolutely sure to conduct regression tests when you do &quot;fix&quot; the bug. 1. 2. 3. 4. ©SEPA

92. public class Date { private int day, month, year; public Date( ) { day=0; month=0;year=0; } public Date(int day, int month, int year) { this.day = day; this.month = month; this.year = year; } public boolean leap( ) { // should change to private return ((year % 400 == 0)|| ((year % 100 > 0) && (year % 4 ==0))); } } public class Instance { private Date date; private Time time; public Instance(Date date, Time time){ this.date = date; this.time = time; } public void getInstance() { date.getDate(); time.getTime(); } } getDate( ) Date day month year getDate( ) displDate( ) addDate( ) dInMonth( ) leap( ) Instance date time getInstance( ) displInstance( ) addInstance( ) Date Day : integer month : integer year : integer getDate( ) displDate( ) addDate(d: integer, nD:Date) -dInMonth( ) -leap( ) Instance date:Date time:Time getInstance( ) displInstance( ) addInstance( )

96. Structure Chart: hierarchical Class Collaborations: network ProcessReporter getProcessDetail getStartDetail getDuration getStartTime getStartDate Process: Instance:Start Date:Start Time:Start 1.disInst( ) 2. disDate( ) 6. addDate( ) 3.disTime( ) 4. addTime( ) Instance:End 8.Instance( ) Date:End Time:End 9. disDate( ) 10. disTime( ) 5. Time( )

98. CASE STUDY - SEQUENCE DIAGRAM

100. Cluster : A group of classes that collaborate to perform a particular request Start with receiver classes Instance:Start Date:Start Time:Start 1. addInst( ) 4. addDate( ) 2. addTime( ) Instance:End 8.Instance( ) Date:End Time:End 10. disDate( ) 9. disTime( ) 7. Date( ) 3. Time( ) 5. dInMon( ) 6. leap( )