This document discusses performance testing with SharePoint. It covers identifying the test environment and performance criteria, designing tests, configuring the test environment, constructing and running tests, and analyzing results. Specific topics include using Visual Studio 2008 for load testing, SharePoint considerations like timers jobs and SQL Server, and lessons learned like the importance of planning tests thoroughly and allowing time for issues.

1. Performance Testing w/ SharePoint SharePoint Saturday Charlotte Jeff Dalton Twitter.com/jefferydalton Sharepoint.nailhead.net

2. About Me 6/20/2009 2 SharePoint Saturday Charlotte / Performance Testing with SharePoint

3. Agenda Foundation Performance Testing w/ Visual Studio 2008 SharePoint Consideration Lessons Learned 6/20/2009 3 SharePoint Saturday Charlotte / Performance Testing with SharePoint

4. Performance Testing Basics Identify Environment - Capture Application and Infrastructure architecture Identify Performance Acceptance Criteria - Requirements Design Tests – Capture Scenarios, Navigation Paths, Workload Model Configure Test Environment Construct Tests Run Tests / Analyze Results / Create Test Reports Rinse / Repeat as Needed 6/20/2009 4 SharePoint Saturday Charlotte / Performance Testing with SharePoint

5. Identify Environment Architecture Logical Physical System Core Functions / Business Process What happens if this system is down? User Activites How do people use this system? 6/20/2009 SharePoint Saturday Charlotte / Performance Testing with SharePoint 5

6. Identify Performance Acceptance Criteria Establish Requirements SLA / OLA, Regulations, Contracts Request X must respond < 5 seconds Press Releases must be available with 2 minutes of release Application Requirements Application must support 5 orders per second. Determine Thresholds Processor Utilization should not exceed 80% for 5 consecutive seconds. Establish Goals 95% of page requests should execute in less than 2 seconds. 6/20/2009 SharePoint Saturday Charlotte / Performance Testing with SharePoint 6

7. Design Tests Define usage scenarios (web transactions) Requirements and Goals. Common Usage Scenarios Business Critical Scenarios Technical Concerns Stakeholder Concerns High Visibility usage. Strive for real word usage User delays / think times User abandonment Common user errors Work Load models 6/20/2009 SharePoint Saturday Charlotte / Performance Testing with SharePoint 7

8. Configure Test Environment Setup Test Lab Network Hardware Setup / Configure load testing software Setup monitoring software Considerations How much load before bottleneck on test generators. Synchronize System Clocks Validate Network components (Switches, Firewall, Load Balancer …) 6/20/2009 SharePoint Saturday Charlotte / Performance Testing with SharePoint 8

9. Construct Tests Plan on this taking longer than expected Build scripts to create simulated data SharePoint Test Data Population Tool (CodePlex) SharePoint Site Builder (CodePlex) Ensure validation rules are working correctly Plan on issues (usually test tools lag behind technology) 6/20/2009 SharePoint Saturday Charlotte / Performance Testing with SharePoint 9

10. Run Tests / Analyze Results / Create Test Reports Make sure proper people are in place when running tests. Collect / Analyze data as soon as tests are finished (everything is fresh in memory) Meticulous documenting start / stop times, special circumstance with test. Archive Test Results Run Tests Multiple times (to reduce chance of anomalies) Stakeholder Reports / Technical Reports Content is tailored to audience 6/20/2009 SharePoint Saturday Charlotte / Performance Testing with SharePoint 10

11. Agenda Foundation Performance Testing w/ Visual Studio 2008 SharePoint Consideration Lessons Learned 6/20/2009 11 SharePoint Saturday Charlotte / Performance Testing with SharePoint

12. Performance Testing w/ Visual Studio 2008 Visual Studio 2005 or 2008 Team Test or Team Suite Visual Studio Team System 2008 Test Load Agent Now let’s get out of slides 6/20/2009 SharePoint Saturday Charlotte / Performance Testing with SharePoint 12

13. Agenda Foundation Performance Testing w/ Visual Studio 2008 SharePoint Consideration Lessons Learned 6/20/2009 13 SharePoint Saturday Charlotte / Performance Testing with SharePoint

14. SharePoint Considerations Web Front Ends Caching Enabled / Content DB Size / Site Collection Size Is Security included? SQL Server Do Not Ignore Timer Jobs How will these impact your operations? Have you included these in your tests? SSP Search, BDC, User Profiles, Session State, Excel Services,… Solution Deployments 6/20/2009 SharePoint Saturday Charlotte / Performance Testing with SharePoint 14

15. Agenda Foundation Performance Testing w/ Visual Studio 2008 SharePoint Consideration Lessons Learned 6/20/2009 15 SharePoint Saturday Charlotte / Performance Testing with SharePoint

16. Lessons Learned Plan, Plan, Plan Know your environment Establish your goals Know your tests Know your tools Test, Test, Test (or search / destroy bottlenecks) Leave plenty of time to shake out the tests Size Matters (when Testing SharePoint) Witnessed 100% performance degradation between empty site collection and one with 22k webs. 6/20/2009 SharePoint Saturday Charlotte / Performance Testing with SharePoint 16

17. Lessons Learned Think Times Matter / End User Variations Matter Pure load is nice, but it doesn’t reflect real world Try to mix-up those IP-Addresses Running all tests over 2 – 4 IP-Addresses does not simulate real world. Dedicated Test Labs Are Good, Ummmkay Testing in Production is Bad, Ummmkay 6/20/2009 SharePoint Saturday Charlotte / Performance Testing with SharePoint 17

18. Lessons Learned Load Agents can max out quickly (do not want to over stress those machines) Over worked Load Agent will skew test results. Testing for under 30 minutes may cause you to miss out on major problems. Rule of thumb is about 1000 concurrent users for 1 VSTS Load Agent (most I’ve been able to run is around 300) 6/20/2009 SharePoint Saturday Charlotte / Performance Testing with SharePoint 18

19. References Patterns and Practices: Performance Testing Guidelines for Web Applications Web and Load Testing with Visual Studio Team System Visual Studio 2008 Testing (Load Test) Advanced Load Testing Features of Visual Studio Team System MSDN: Working with Load Tests Lessons Learned (Part 1) Lessons Learned (Part 2) More Lessons Learned 6/20/2009 19 SharePoint Saturday Charlotte / Performance Testing with SharePoint