How to use the power of Team Foundation Server 2010 to improve your software quality

1. Improved software testing using Visual Studio and TFS 2010 Alexander Vanwynsberghe Software Engineer @vanwynsberghea

2. What is this presentation about? Software testing, why? Use structured testing The capabilities of TFS 2010 Summary Alexander Vanwynsberghe – 09/05/2011

3. Software testing, why? It’s all about the quality Lets you measure how well software is designed Find the bugs! Checks how the product meets the requirements Testing does not make software perfect Alexander Vanwynsberghe – 09/05/2011

4. Use structured testing What does that mean? http://www.flickr.com/photos/59068359@N04/5563821201

5. 1. Make test plans A summary of the test activities that will be performed Shows how the tests will be organized A list of tests which, if performed, will identify all of the requirements that have been set Alexander Vanwynsberghe – 09/05/2011

6. 2. Define test cases Description of a specific interaction that a tester will have in order to test a single behavior of the software Step-by-step guide for the tester Consists of: Preconditions To-do Result Alexander Vanwynsberghe – 09/05/2011

7. 3. Execute tests Run all test cases from the appropriate test plan First iteration test focuses on new features Regression testing usually involves executing all test cases which have previously been executed Test parallel with the developers, not only just before a release Alexander Vanwynsberghe – 09/05/2011

8. 4. Automate tests User interfaces can be test-automated Eliminate repetitive tasks Can save the testers a lot of time if many iterations of testing are required Do not automate one-time-only tests Alexander Vanwynsberghe – 09/05/2011

9. 5. Report bugs And assign them to developer(s) http://www.flickr.com/photos/harmony19490/362411276/

10. The ‘how-not-to-do-it’ process http://http://www.flickr.com/photos/frau_k/3984560746//

11. The developer.. All coding done Some small ‘debugger-tests’ Check-in/commit Deploy to a development environment “Test please” to tester… Alexander Vanwynsberghe – 09/05/2011

12. The tester tests.. No plan available (no user stories) Lucky to find bugs Waste a lot of time on irrelevant things This is called “Exploratory testing” Alexander Vanwynsberghe – 09/05/2011

13. ..and reports the bug.. Email subject: Mayor bug found!! Email content: “I can’t quite remember what I did, but my data is gone !?” Where is my data? Alexander Vanwynsberghe – 09/05/2011

14. The dev tests and replies.. Email subject: RE: mayor bug found!! Alexander Vanwynsberghe – 09/05/2011

16. Tester tests everything

17. “Unable to reproduce” = “Works on my machine”

18. Bug never gets fixed

19. Ship a buggy productAlexander Vanwynsberghe – 09/05/2011

20. There is a better way http://http://www.flickr.com/photos/frau_k/3984560746//

21. Use the power of TFS 2010 All data is stored in TFS Alexander Vanwynsberghe – 09/05/2011

22. Manual testing http://www.flickr.com/photos/avalsitarb/403407215/

23. Manual testing with MTM 2010 Alexander Vanwynsberghe – 09/05/2011

25. Define test cases

26. Execute test plans

27. Step-by-step execution

28. Screen recording

29. IntelliTrace logging

30. Inline bug creation with attachments Alexander Vanwynsberghe – 09/05/2011

33. Necessary information to reproduce the bug

34. IntelliTrace to get to the ‘bug point’ in code

35. Video recording

36. Comments of the testerAlexander Vanwynsberghe – 09/05/2011

38. Link the bug to the changeset

39. The changes for a particular bug are traceable

40. Developer sets the bug work-item to ‘Resolved’

41. Tester can test again and check the influence of the code change on other tests (Test Impact Analysis)Alexander Vanwynsberghe – 09/05/2011

43. Don’t fix it at first

44. Create an automated test from the recorded test

45. Let the test fail

46. Fix the bug

47. Run the test again

48. TDDWill be explained later Alexander Vanwynsberghe – 09/05/2011

49. Demo MTM 2010 http://www.flickr.com/photos/avalsitarb/403407215/

50. Automatic testing http://www.flickr.com/photos/slimwhitman/2317727338

51. Automatic testing with VS2010 Alexander Vanwynsberghe – 09/05/2011

53. Unit testing

54. Integration testing

55. User Interface = Coded UI testing

56. Database (stored procedures) = Database testing

57. Web Performance testing

58. Load testingAlexander Vanwynsberghe – 09/05/2011

60. Readable

61. Understand!

62. What? Scenario? What will happen?

63. Maintainable

64. Trustworthy

65. If all unit tests passed = bug freeAlexander Vanwynsberghe – 09/05/2011

66. The structure of a unit test Arrange Act Assert [TestMethod] public void Add_TwoPlusThree_EqualsFive() { // arrange int expected = 5; int first = 2; int second = 3; var calculator = new Calculator(); // act int actual = calculator.Add(first, second); // assert Assert.AreEqual(expected, actual, “addition failed”); } Alexander Vanwynsberghe – 09/05/2011

68. UI

69. Business Logic

70. Database

71. Touches:

72. File system

73. Database

74. Other shared resources

75. Not in memoryAlexander Vanwynsberghe – 09/05/2011

77. Created using recording, or using code

78. For web, Win-forms and Silverlight

79. Can check control behaviors (like label values)Alexander Vanwynsberghe – 09/05/2011

81. Can execute T-SQL statements like:

82. SELECT * FROM Content.Workspaces WHERE WorkspaceId = 1 --will fail, because it existsAlexander Vanwynsberghe – 09/05/2011

84. Validate the responses to see if the application is working as expected

85. Created by recording a human user clicking through a Web application

86. Web tests can be made smarter by using Validation Rules, Extraction Rules, and Test Context Alexander Vanwynsberghe – 09/05/2011

87. Web performance testing Alexander Vanwynsberghe – 09/05/2011

89. Simulate virtual users by setting the number of iterations

90. Collects all sorts of information like:

91. Response times of the executed requests,

92. Performance counters of the tested application infrastructure (like CPU, Memory, I/O, …)

93. The health of your load-testing infrastructure (load controller and agents)Alexander Vanwynsberghe – 09/05/2011

94. Load testing Alexander Vanwynsberghe – 09/05/2011

95. Demo automatic testing http://www.flickr.com/photos/avalsitarb/403407215/

96. What you may not forget http://www.flickr.com/photos/niffgurd/3324865930

98. It gives confidence to those selling the product

99. The earlier a defect is found, the less development rework and re-test is required

100. Use proven high performance as a selling point

101. Testing needs disciplineAlexander Vanwynsberghe – 09/05/2011

102. Alexander Vanwynsberghe @vanwynsberghea Engelse Wandeling 2, K18 B-8500 Kortrijk, Belgium alexander.vanwynsberghe@cognistreamer.comwww.cognistreamer.com