Testing SharePoint Solutions Overview
•Download as PPTX, PDF•
4 likes•6,676 views
This document discusses testing SharePoint solutions using Visual Studio 2010 tools. It provides an overview of common testing challenges and scenarios that lack testing rigor. It then demonstrates the Visual Studio 2010 testing capabilities for unit testing, load testing, coded UI testing, and lab management to help address these issues and establish a more robust testing practice. Automated testing tools are shown to integrate with the development lifecycle in Visual Studio and Team Foundation Server.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (16)
Similar to Testing SharePoint Solutions Overview
Similar to Testing SharePoint Solutions Overview (20)
More from Spiffy
More from Spiffy (20)
Recently uploaded
Recently uploaded (20)
Testing SharePoint Solutions Overview
- 1. Testing SharePoint SolutionsOverview Ervin Loh Visual Studio ALM MVP K365Labs SdnBhd Light Up SharePoint
- 2. Presenter Ervin Loh Profile Ervin Loh is currently the Application Lifecycle Management Program Manager at K365Labs Sdn Bhd. He works on a variety of Application Lifecycle Management, Software Configuration Management, and Lab Management products. He is also active in the IT community space by contributing contents to Microsoft Malaysia's START.NET and Ultimate program workshops and talks in conferences, events and user groups.
- 3. ASP.NET Developers, SharePoint Developers Target Audience
- 4. Is this how you kill bugs?
- 8. An Unfortunately Common Scenario Developer writes code Developer makes sure the code compiles Developer checks in code Repeat x15 developers for 4 weeks Developer lead does a build of all code from developers Application is installed in test environment Tester tries to test the application Application doesn’t work Developer blames tester
- 9. The Demonstrations Scenario The Configurations A fully configured Visual Studio 2010, Team Foundation Server 2010, and SharePoint Server 2010 environment. The Tasks A lot of tests.
- 10. Topics Overviewofthetestingtools in Visual Studio 2010 Testing SharePoint projectswith Visual Studio 2010 10
- 11. Have you heard any of these? “my testers are spending too long testing the same thing” “tooling is expensive (time, licenses, people)” “the developers say the defects are useless” “when is my software ready to ship?” “developers and testers work in silos and don’t communicate/speak the same language” “we have a changed requirement, what do I need to test?”
- 12. What if you could… reduce the time it takes to determine the root cause of a bug reduce the time it takes to replicate a bug uncovered by user actions reduce the time it takes to isolate differences between the test and production environment enable users to easily run acceptance tests and track their results reduce the time it takes to verify the status of a reported bug fix
- 13. Where does testing happen? 70% of testing happens here Majority of test tools target here Black Box Testing White Box Testing API Testing
- 14. Visual Studio 2010 Test Capabilities Load Test Web Test Coded UI Test Microsoft Test Runner Unit Testing Test Case Management Lab Management Data Collectors (Historical Debugging, Test Impact, Metrics) Team Foundation Server Reporting
- 16. Why is fixing bugs difficult? Insufficient information to reproduce Not every step is documented Can’t see exactly how the bug was triggered Different environments OS, service packs, installed software,… Creating a clean environment takes time
- 17. Microsoft Test Manager 2010 Plan, Manage and Execute (manual) tests from one place Create test cases Build test suites Define configurations Run test cases File bugs into TFS Including captured data Automate testing Integrated with TFS
- 18. VS Agents: Diagnostic Data Adapters System Info IntelliTrace™ Video Capture Steps Performed System Info IntelliTrace™ Visual Studio Agents 2010 Separate download Test Controller & Test Agents Capture data while testing Save data with bug report Easy for developer to reproduce bug
- 19. Lab Manager 2010 Setup, Teardown, and Restore virtual environments Maintain known state Build automation Build deployment Test execution Network isolation Run multiple copies Connect from VS2010 Linked to bug form
- 20. 70% of all functional testing is still done manually.
- 21. DemoFunctional Testing with Microsoft Test Manager
- 23. Load Testing
- 24. Coded UI Testing
- 26. Load Testing Stress-testing: simulate many users at once Need multiple servers to create substantial load Multiple agents generate load on the application Single controller coordinates the agents and captures data PerfMon-style output
- 27. Coded UI Testing Simulate a single use case Uses a recording toolbar, like Web Performance test Records the user’s interaction with the UI Generates code to reproduce that interaction Code can be edited and customized at will Has some issues with SharePoint e.g. no support for datasheets Visual Studio Feature Pack 2 adds support for Silverlight
- 28. Unit Testing Using built-in Unit Test with SharePoint has problems: Issues with 32-bit vs 64-bit, and .NET 3.5 vs .NET 4.0 Need to stub/mock SharePoint API e.g. SPSite, SPWeb, SPList Additional tools enable unit testing for SharePoint projects Microsoft Research: Pex & Moles Integrates with VS2010, free to MSDN subscribers http://research.microsoft.com/en-us/projects/pex/pexsharepoint.pdf Third party: NUnit, TypeMock
- 29. DemoAutomated Testing with Microsoft Visual Studio
- 31. Are we ready to ship? Are we making progress on running test plans? How are our builds doing over time? Are we fixing bugs? What’s the quality of our bug fixes? Ready to Ship?
- 32. DemoReporting
- 35. Resources Test Manager 2010 and Lab Manager http://msdn.microsoft.com/en-us/library/ms182409.aspx http://msdn.microsoft.com/en-us/library/dd997438.aspx Visual Studio 2010 Agents http://www.microsoft.com/downloads/en/details.aspx?FamilyID=a3216d2a-0530-4f6c-a7c9-0df37c54a902 Pex and Moles http://research.microsoft.com/en-us/projects/pex/default.aspx
- 36. Questions?
- 37. © 2011 Microsoft Corporation. All rights reserved. Microsoft, Visual Studio, the Visual Studio logo, and [list other trademarks referenced] are trademarks of the Microsoft group of companies. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Editor's Notes
- These are some of the comments that we have heard from our customers. Are any of these being said by your testers?Broken down another way, can you answer these affirmatively?Do you know consistently when your software will ship?Are the defects created by your tester truly effective?No? <click to next>
- Does this sound like a wish list? How much money is lost in the time that it takes to track down a bug?How much money is lost in the time it takes to isolate the differences between your development in test environments?
- It is also important to understand where most testing happens in the spectrum of general testing to the more technical specialist testing.The Generalist Testers are usually professional testers with no coding background. Often these testers are experts in the business process or tool that is being developed. On the opposite side of the spectrum is the Specialist. This is a tester with strong coding skills.A fun side note: Microsoft’s testers are usually converted developers and tend to be on the specialist side of the graph.Black-box testing is a method of testing software that tests the functionality of an application as opposed to its internal structures or workings (see white-box testing). Specific knowledge of the application's code/internal structure and programming knowledge in general is not required. Test cases are built around specifications and requirements, i.e., what the application is supposed to do. It uses external descriptions of the software, including specifications, requirements, and design to derive test cases. These tests can be functional or non-functional, though usually functional. The test designer selects valid and invalid inputs and determines the correct output. There is no knowledge of the test object's internal structure.White-box testing (a.k.a. clear box testing, glass box testing, transparent box testing, or structural testing) is a method of testing software that tests internal structures or workings of an application as opposed to its functionality (black-box testing). An internal perspective of the system, as well as programming skills, are required and used to design test cases. The tester chooses inputs to exercise paths through the code and determine the appropriate outputs. It is analogous to testing nodes in a circuit, e.g. in-circuit testing (ICT). While white-box testing can be applied at the unit, integration and system levels of the software testing process, it is usually done at the unit level. It can test paths within a unit, paths between units during integration, and between subsystems during a system level test. Though this method of test design can uncover many errors or problems, it might not detect unimplemented parts of the specification or missing requirements. White-box test design techniques include: Control flow testing Data flow testing Branch testing Path testingAPI testing (application programming interface) – is a specific type of White Box testing of the application focusing on public and private APIs<Question to Audience>Looking at this spectrum, where does most testing happen today? <collect answers and click>Where do most testing tools target today? <collect answers and click>
- Describe the pieces as it applies to the spectrum as the slide builds out.
- Currently about 70% of functional testing is done manually—that is, a software tester follows a script to execute a series of steps to verify the outcome of a test. Microsoft Test Manager 2010 is an example of a tool that someone doing manual testing would use to be more productive. Imagine if you could still get the benefits of functional testing without the overhead and resource cost of doing in manually.Consider a regression test—a test designed to identify if a bug fix begins to fail. With a regression test (and even other functional tests) you are repeatedly testing something that was known to work at one point. These too are typically done manually. This is a massive resource hit. Software testers spend countless hours testing functionality that works, solely to ensure it still works. Imagine how much more productive they could be if they could focus their efforts on creating and running new test cases that covered parts of the system not currently tested instead of spending their time testing the same thing, over and over again.
- Here are two examples <Build> of the dashboard you get when running on top of one of the full versions of SharePoint Server. First, we have the Burndown dashboard showing project progress. <Build> Second, we have the Quality dashboard. So what does the quality dashboard tell us? <next>
- As you can see the Quality dashboard has four main graphs. <Build>First you can see if your test team is making progress on running test plans. <Build>Second, you can see how build are doing over time. What’s the trend like? Are you having lots of success or failure? <Build>Third, what’s your bug trend like. Are you closing out bugs or are you stagnate. Or is the velocity of your bug filing far exceeding your team’s ability to fix, test, and close out bugs? <Build>Finally, are you seeing a bad trend related to bug reactivations—bugs that were closed reopened by test as not fixed? All of this information is there for you in a quick, heads up dashboard format.