Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Full Stack Load Testing
Load Testing On Alfresco 5.1
Derek Hulley
Alfresco Founding Engineer
Introduction
•  Resource Limits
•  Load Test Accuracy
•  Load Test Software
Introduction
Resource Limitations
Limiting Factors
•  Some Basic Server Limitations
–  Memory consumption
–  CPU usage
–  IOPS / Bandwidth / IO
•  Other Ser...
Moving Parts
Resources

Caches	
  
Start with a
guess and try it
out in anger
 Alfresco	
  and	
  
Share	
  
20%	
  
Transforma4on
s	
  
10%	
  
SOLR	
  
30%...
Focus

e.g. SOLR
Resources

Alfresco	
  and	
  
Share	
  
20%	
  
Transforma4on
s	
  
10%	
  
SOLR	
  
30%	
  
Database	
 ...
Resource Focus Example: SOLR
•  CPU: underused

•  Memory: underused

•  IO: underused
•  Network: Keep and eye on this!

...
Reduce spend
and iterate …


Resources

Alfresco	
  and	
  
Share	
  
23%	
  
Transforma4on
s	
  
12%	
  
SOLR	
  
18%	
  ...
Load Test Accuracy
Load Test Risk Factors: Scale
•  Increases Cost
–  Your Time
–  Getting the data in does not normally produce results
•  B...
Load Test Risk Factors: Documents
•  Document Types
–  Image thumbnail generation
–  Text to PDF for preview
–  Document l...
Load Test Risk Factors: API Usage Profile
•  Load Profile
–  Burst load
–  Trickle load
•  Data Structure
–  Paging
–  Sorti...
Load Test Risk Factors: UI Complexity
•  Share APIs
–  /private
•  Upload File
–  Stream binary
–  Then all of these at on...
Load Test Risk Factors: UI Complexity
LoadTestAccuracy
Load Test Risk Factors: UI Complexity
•  Page-Object Pattern: Browsers do what they do
Login Event Processor
 Login Page
 ...
Load Test Software
Alfresco Benchmark Server and Tests
References
•  Martin Bergljung’s walkthrough
•  Alfresco Wiki: Benchmark_Testing_with_Alfresco
•  GitHub: AlfrescoBenchmar...
Alfresco
Benchmark
Suite
TestSoftware
Benchmark	
  Server	
  
Tomcat	
  7	
  
Rest	
  API	
  
MongoDB	
  
Config	
  Data	
 ...
Load Tests: /tests/ent-signup
•  Create users
•  User base reused
–  Prerequisite for other tests
TestSoftware
Load Tests: /tests/dataload
•  Reuse user base
•  Create
–  Sites
–  Site Members
•  Site data
–  Folders
–  Files uploade...
Load Tests: /tests/dataload
•  AlfrescoBenchmark/alfresco-text-gen

•  Text data generated in server
•  Option to push dat...
Load Tests: GitHub benchmark-cmis
•  Burst load
•  Easy to extend
TestSoftware
Load Tests: FTP sample files vs spoofing
•  FTP server
–  A few thousand samples
–  Use for Share tests
•  Transformations
•...
Share Load Testing: /tests/share
•  Concurrent Users:
–  A new user logs in every N seconds
–  The user logs out after spe...
Summary
Summary
•  Build up load testing software and skills
•  Design tests around existing tools
–  Understand risks and where t...
Questions
Thank you!
Condividi su
#AlfrescoDayRoma
Nächste SlideShare
Wird geladen in …5
×

Alfresco Day Roma 2015: Full Stack Load Testing

755 Aufrufe

Veröffentlicht am

Alfresco Day Roma 2015: Full Stack Load Testing on Alfresco, Derek Hulley, Founding Engineer, Alfresco

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Alfresco Day Roma 2015: Full Stack Load Testing

  1. 1. Full Stack Load Testing Load Testing On Alfresco 5.1
  2. 2. Derek Hulley Alfresco Founding Engineer
  3. 3. Introduction •  Resource Limits •  Load Test Accuracy •  Load Test Software Introduction
  4. 4. Resource Limitations
  5. 5. Limiting Factors •  Some Basic Server Limitations –  Memory consumption –  CPU usage –  IOPS / Bandwidth / IO •  Other Server Limitations –  Monitoring (getting at logs, charts, monitors, etc) –  Backups –  CRON jobs •  Features –  Supported file types –  Auditing –  Jodconverter –  Clustering Resources
  6. 6. Moving Parts Resources Caches  
  7. 7. Start with a guess and try it out in anger Alfresco  and   Share   20%   Transforma4on s   10%   SOLR   30%   Database   30%   Content   5%   Load  Balancer   5%   Resource  Guess  0   Alfresco  and  Share   Transforma2ons   SOLR   Database   Content   Load  Balancer   Resources Monitor all Run for days Resolve Issues Repeat Adjust resources
  8. 8. Focus
 e.g. SOLR Resources Alfresco  and   Share   20%   Transforma4on s   10%   SOLR   30%   Database   30%   Content   5%   Load  Balancer   5%   Resource  Guess  0   Alfresco  and  Share   Transforma2ons   SOLR   Database   Content   Load  Balancer  
  9. 9. Resource Focus Example: SOLR •  CPU: underused •  Memory: underused •  IO: underused •  Network: Keep and eye on this! Resources
  10. 10. Reduce spend and iterate …
 Resources Alfresco  and   Share   23%   Transforma4on s   12%   SOLR   18%   Database   35%   Content   6%   Load  Balancer   6%   Resource  Guess  2   Alfresco  and  Share   Transforma2ons   SOLR   Database   Content   Load  Balancer  
  11. 11. Load Test Accuracy
  12. 12. Load Test Risk Factors: Scale •  Increases Cost –  Your Time –  Getting the data in does not normally produce results •  Bulk Loading –  Different hardware for this phase? –  Simulate bulk loading as accurately as necessary •  Number of Documents –  Easier to account for with experience –  Data storage costs –  Backup/Restore procedures •  Users and Groups –  LDAP sync –  Complex Group Hierarchy –  Site Group Permissions –  Mass-authentication LoadTestAccuracy
  13. 13. Load Test Risk Factors: Documents •  Document Types –  Image thumbnail generation –  Text to PDF for preview –  Document library listings (many transformations a.s.a.p) •  Document Size –  Network and IO costs and limits –  CPU used for streaming –  PDF to text memory consumption –  External converters •  More CPU or more infrastructure LoadTestAccuracy
  14. 14. Load Test Risk Factors: API Usage Profile •  Load Profile –  Burst load –  Trickle load •  Data Structure –  Paging –  Sorting –  Permissions •  The Internet –  CMIS across the Atlantic –  Southern hemisphere client apps LoadTestAccuracy
  15. 15. Load Test Risk Factors: UI Complexity •  Share APIs –  /private •  Upload File –  Stream binary –  Then all of these at once: •  Register user activity •  Generate thumbnail •  Generate PDF preview and display •  Get and display document metadata •  Get and display document version history •  Replay Doom (You Only Live Once) LoadTestAccuracy
  16. 16. Load Test Risk Factors: UI Complexity LoadTestAccuracy
  17. 17. Load Test Risk Factors: UI Complexity •  Page-Object Pattern: Browsers do what they do Login Event Processor Login Page Dashboard Page Browser processEvent loginAs Send keystrokes Login button click DashboardPage Timed render Wait For: Activities Wait For: My Sites Wait For: My Sites Wait For: My Sites Wait For: My Docs Wait For: etc Increasing Complexity Server x100 LoadTestAccuracy
  18. 18. Load Test Software Alfresco Benchmark Server and Tests
  19. 19. References •  Martin Bergljung’s walkthrough •  Alfresco Wiki: Benchmark_Testing_with_Alfresco •  GitHub: AlfrescoBenchmark TestSoftware
  20. 20. Alfresco Benchmark Suite TestSoftware Benchmark  Server   Tomcat  7   Rest  API   MongoDB   Config  Data   Services   MongoDB   Test  Data   UI   Benchmark  Driver  (xN)   Benchmark  Driver  (xN)   Benchmark  Driver   Tomcat  7   Extras   (Selenium)   Servers  /  APIs   Servers  /  APIs   Load  Balancer   Servers  /  APIs   Test   Services   Rest  API  
  21. 21. Load Tests: /tests/ent-signup •  Create users •  User base reused –  Prerequisite for other tests TestSoftware
  22. 22. Load Tests: /tests/dataload •  Reuse user base •  Create –  Sites –  Site Members •  Site data –  Folders –  Files uploaded using CMIS or TestSoftware
  23. 23. Load Tests: /tests/dataload •  AlfrescoBenchmark/alfresco-text-gen •  Text data generated in server •  Option to push data
 into ContentStore TestSoftware
  24. 24. Load Tests: GitHub benchmark-cmis •  Burst load •  Easy to extend TestSoftware
  25. 25. Load Tests: FTP sample files vs spoofing •  FTP server –  A few thousand samples –  Use for Share tests •  Transformations •  Text extractions •  Renditions •  Spoofing –  Stress SOLR –  document library size •  Search: searchterms.txt TestSoftware
  26. 26. Share Load Testing: /tests/share •  Concurrent Users: –  A new user logs in every N seconds –  The user logs out after spending Y seconds –  Number of concurrent users C = Y/N •  Selenium Driver Overload –  Use Windows –  5s to start a browser on EC2 m3.2xlarge –  Up to 40 browsers per load driver on EC2 m3.2xlarge –  Drivers required D = max( 5s/N, C/40) –  m3.2xlarge Windows à spot price of $0.3/hour –  500 concurrent user à20 drivers à $6/hour TestSoftware
  27. 27. Summary
  28. 28. Summary •  Build up load testing software and skills •  Design tests around existing tools –  Understand risks and where to extend or introduce new tests •  Iterate Summary
  29. 29. Questions
  30. 30. Thank you! Condividi su #AlfrescoDayRoma

×