Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Velocity 2014 nyc WebPagetest private instances

8.245 Aufrufe

Veröffentlicht am

Presentation from Velocity NYC 2014 on setting up private WebPagetest instances

Video: https://www.youtube.com/playlist?list=PLWa0Ky8nXQTaFXpT_YNvLElTEpHUyaZi4

Veröffentlicht in: Technologie
  • Loggen Sie sich ein, um Kommentare anzuzeigen.

Velocity 2014 nyc WebPagetest private instances

  1. 1. WebPagetest Private Instances Patrick Meenan pmeenan@webpagetest.org @patmeenan
  2. 2. The Basics ● Presentation is available on slideshare ○ http://www.slideshare.net/patrickmeenan ○ Mostly contains links to everything I’ll be showing ● Video will be uploaded to YouTube ○ https://www.youtube.com/user/patmeenan ● I’ll send links and announcements of both to Twitter and Google+ ○ https://twitter.com/patmeenan - @patmeenan ○ https://plus.google.com/u/0/+PatrickMeenan/posts
  3. 3. WPT Web Server Sites being tested System Architecture Desktop Agent Node.js Mobile Agent Mobitest Agent
  4. 4. WPT Web Server Sites being tested Agent Data Flow Submit test (UI or API) http://<server>/runtest.php User
  5. 5. Server: - Creates test ID YYMMDD_<hash>_<num> - Creates directory for test results/YY/MM/DD/<hash>/<num> - Writes test job to work directory for location work/jobs/<location>/<ID>.<priority> - Adds test to queue file (lossy, can be re-built from the work directory) tmp/<location hash>.queue - Redirects to result page
  6. 6. WPT Web Server Sites being tested Agent Poll for test status http://<server>/result/<ID>/ User
  7. 7. WPT Web Server Sites being tested Agent User Poll for work for location http://<server>/work/getwork.php
  8. 8. WPT Web Server Sites being tested Agent User - Launch Browser - Navigate to page - Measure/Collect Data
  9. 9. WPT Web Server Sites being tested Agent User Upload large files individually (images, tcpdump, timeline, etc) http://<server>/work/resultimage.php
  10. 10. WPT Web Server Sites being tested Agent User Complete run (or full test if all runs are done) http://<server>/work/workdone.php
  11. 11. WPT Web Server Sites being tested Agent View completed test http://<server>/result/<ID>/ User
  12. 12. Server Config ● Any server that can run php ○ Linux tends to work best ○ Can run on the same PC as test agents if needed ● Copy www/ directory from distribution to site docroot ● Configure rewrites for friendly URLs (optional) ○ .htaccess included for apache ○ nginx.conf can be included into nginx server config ● Fix anything that http://<server>/install/ complains about ○ PHP ini settings: allow_url_fopen, memory and post file size limits ○ PHP modules: GD, zip, zlib, curl, apc, sqlite ○ Command-line utilities: ffmpeg, imagemagick, jpegtran, exiftool ○ Filesystem permissions (and directories)
  13. 13. Nginx config server { listen 80; server_name webpagetest.example.com; root /var/www; location ~ .php$ { fastcgi_pass 127.0.0.1:9000; fastcgi_param SCRIPT_FILENAME /var/www$fastcgi_script_name; fastcgi_param HTTP_MOD_REWRITE On; include fastcgi_params; } include /var/www/nginx.conf; }
  14. 14. /install/
  15. 15. Server WPT Config ● Copy/modify settings/*.sample ○ settings/settings.ini ■ Contact information ■ Maximum number of runs ■ Screen shot quality ■ Show Slow integration ■ Test archiving (covered later) ○ settings/locations.ini ■ Definition for all of the test locations
  16. 16. Locations.ini Grouping Browsers
  17. 17. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” [NYC] browser=Chrome,Firefox,IE 11 label=”New York” [NYC_Mobile] browser=Motorola G - Chrome,Motorola G - Chrome Beta label=”New York” [Group_2] ...
  18. 18. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” [NYC] browser=Chrome,Firefox,IE 11 label=”New York” [NYC_Mobile] browser=Motorola G - Chrome,Motorola G - Chrome Beta label=”New York” [Group_2] ... List of the group IDs in order to be displayed
  19. 19. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” [NYC] browser=Chrome,Firefox,IE 11 label=”New York” [NYC_Mobile] browser=Motorola G - Chrome,Motorola G - Chrome Beta label=”New York” [Group_2] ... “Location ID”s that are part of the group in the order the browsers are to be displayed. String displayed for the “Test Location”
  20. 20. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” [NYC] browser=Chrome,Firefox,IE 11 label=”New York” [NYC_Mobile] browser=Motorola G - Chrome,Motorola G - Chrome Beta label=”New York” [Group_2] ... These are the location ID’s the agents connect to and that are used when submitting tests through the API
  21. 21. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” [NYC] browser=Chrome,Firefox,IE 11 label=”New York” [NYC_Mobile] browser=Motorola G - Chrome,Motorola G - Chrome Beta label=”New York” [Group_2] ... All of the browsers from all of the locations in a group are collected and displayed in the Browser select box.
  22. 22. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” [NYC] browser=Chrome,Firefox,IE 11 label=”New York” [NYC_Mobile] browser=Motorola G - Chrome,Motorola G - Chrome Beta label=”New York” [Group_2] ... All of the agents for a given location ID must support all of the browsers listed
  23. 23. [NYC] browser=Chrome,Firefox label=”New York” [NYC_IE10] browser=IE 10 label=”New York” [NYC_IE11] browser=IE 11 label=”New York” [NYC_Mobile] browser=Motorola G - Chrome,Motorola G - Chrome Beta label=”New York” A single agent can pull from multiple locations (configure it as a comma-separated list). location=NYC,NYC_IE10
  24. 24. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” lat=40.712918 lng=-74.005469 group=North America [NYC] browser=Chrome,Firefox,IE 11 label=”New York” [NYC_Mobile] browser=Motorola G - Chrome,Motorola G - Chrome Beta label=”New York” [Group_2] ... Add map=1 to settings.ini
  25. 25. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” lat=40.712918 lng=-74.005469 group=North America [NYC] browser=Chrome,Firefox,IE 11 label=”New York” [NYC_Mobile] browser=Motorola G - Chrome,Motorola G - Chrome Beta label=”New York” [Group_2] ...
  26. 26. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” [NYC] browser=Chrome,Firefox,IE 11 label=”New York” connectivity="Google Fiber (1Gbps)" key=123456 agents=6 notify=pmeenan@webpagetest.org,admin@webpagetest.org hidden=1 When traffic shaping is not supported. Shows up in the UI as the connectivity information.
  27. 27. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” [NYC] browser=Chrome,Firefox,IE 11 label=”New York” connectivity="Google Fiber (1Gbps)" key=123456 agents=6 notify=pmeenan@webpagetest.org,admin@webpagetest.org hidden=1 Shared key for securing agent communications (optional)
  28. 28. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” [NYC] browser=Chrome,Firefox,IE 11 label=”New York” connectivity="Google Fiber (1Gbps)" key=123456 agents=6 notify=pmeenan@webpagetest.org,admin@webpagetest.org hidden=1 Expected number of agents (for alerting)
  29. 29. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” [NYC] browser=Chrome,Firefox,IE 11 label=”New York” connectivity="Google Fiber (1Gbps)" key=123456 agents=6 notify=pmeenan@webpagetest.org,admin@webpagetest.org hidden=1 Email addresses for alerting when agents stop connecting.
  30. 30. [locations] 1=Group_1 2=Group_2 default=Group_1 [Group_1] 1=NYC 2=NYC_Mobile label=”New York (Chrome, Firefox, IE, Android)” [NYC] browser=Chrome,Firefox,IE 11 label=”New York” connectivity="Google Fiber (1Gbps)" key=123456 agents=6 notify=pmeenan@webpagetest.org,admin@webpagetest.org hidden=1 Hide the location from the UI (still works through the API and from http://<server>/?hidden=1 )
  31. 31. Desktop Agent Config ● Prepare Windows ○ Auto logon ○ Disable screen savers and screen power management (never sleep) ○ Disable UAC ○ Copy agent/* to local disk (c:/webpagetest/ in the examples) ○ Install dummynet (optional for traffic shaping) ○ Create startup shortcut to wptdriver.exe (and urlblast.exe if used) ● Install browsers ○ Firefox can self-install (necessary for updates) ● Configure wptdriver.ini ○ Server URL ○ Location ID ○ Browsers
  32. 32. wptdriver.ini [WebPagetest] url=http://127.0.0.1/ location=Test browser=chrome Time Limit=120 ;key=TestKey123 ;Automatically install and update support software (Flash, Silverlight, etc) software=http://www.webpagetest.org/installers/software.dat [chrome] exe="%PROGRAM_FILES%GoogleChromeApplicationchrome.exe" options='--load-extension="%WPTDIR%extension" --user-data-dir="%PROFILE%" --no-proxy-server' installer=http://www.webpagetest.org/installers/browsers/chrome.dat [Firefox] exe="%PROGRAM_FILES%Mozilla Firefoxfirefox.exe" options='-profile "%PROFILE%" -no-remote' installer=http://www.webpagetest.org/installers/browsers/firefox.dat template=firefox [Safari] exe="%PROGRAM_FILES%SafariSafari.exe" [IE] exe="%PROGRAM_FILES%Internet Exploreriexplore.exe"
  33. 33. EC2 Test Agents ● AMI’s available in all regions ○ IE8-11, Chrome, Firefox, Safari 5 (Windows) ● Pass configuration through user data when launching ○ wpt_server=www.webpagetest.org wpt_loc=EC2_East ● No need to ever touch/login to instance ● Kill/Start as needed ● c1.medium or better recommended
  34. 34. EC2 Test Agents - User Data
  35. 35. EC2 Test Agents - Locations
  36. 36. Sites being tested WPT Web Server Mobile Agent (Node) Node.js Mobile Agent
  37. 37. Mobile Agents - Android ● Android 4.4+ recommended ● Root (Nexus and Moto phones are easiest) ● Install Chrome and Chrome Beta from Play store ● Configure Play to auto-update ● Disable screen lock (security settings) ● Configure developer mode ○ Enable USB debugging ○ Disable screen sleep
  38. 38. Mobile Agents - iOS (alpha) ● Jailbreak ● install/configure iOS webkit debug proxy ● build openurl ipa ○ Needs to be signed with your dev key
  39. 39. Mobile Agents - Tethered host config ● Linux or Windows (Mac may work) ○ Multiple phones per tethered agent ● Node.js ● imagemagick ● Launch wptdriver.bat (or wptdriver.sh) ● Config is passed by command-line ○ Device ID ○ Server URL ○ Location ID ● Windows - consider using adbwatch ○ Kills/restarts adb is it gets hung ○ Launches/restarts agents if they crash
  40. 40. Sites being tested WPT Web Server Mobile Agent (Node) - Cell Network Node.js Mobile Agent
  41. 41. Sites being tested WPT Web Server Mobile Agent (Node) - RNDIS Node.js Mobile Agent IPFW (Optional)
  42. 42. WPT Web Server Sites being tested Mobile Agent (Node) - WiFi Node.js Mobile Agent IPFW Bridge WiFi Access (Optional) Point
  43. 43. Priority Queues ● Tests are pulled from queues based on test priority ● 10 Priority levels (0-9) ○ 0 = highest, 9 = lowest ● Defaults ○ 0 - Tests submitted through the UI ○ 4 - Bulk tests submitted through the UI ○ 5 - Tests submitted through the API ● Queue lengths reported on location check UI ○ http://<server>/getLocations.php
  44. 44. https://www.flickr. com/photos/lunchtimemama/110765169/
  45. 45. API Keys settings/keys.ini (disabled if file is missing) [server] secret=7a5c742c021c414ca0dd1bec42e8f64b key=acdd95754c654042a526847acb96f692 [acdd95754c654042a526847acb96f692] description=Web UI contact=me@example.com limit=0 [6d3a661b7401486b8a5d4f78962f8b80] description=API Key contact=batch@example.com limit=5000 priority=8 [acd5ffd840094cdda054a0b7e57d4546] description=API Key contact=notme@example.com limit=100
  46. 46. API Keys [server] secret=7a5c742c021c414ca0dd1bec42e8f64b key=acdd95754c654042a526847acb96f692 [acdd95754c654042a526847acb96f692] description=Web UI contact=me@example.com limit=0 [6d3a661b7401486b8a5d4f78962f8b80] description=API Key contact=batch@example.com limit=5000 priority=8 [acd5ffd840094cdda054a0b7e57d4546] contact=notme@example.com limit=100 Arbitrary string used for hashing keys sent to browser (I usually use GUIDs)
  47. 47. API Keys [server] secret=7a5c742c021c414ca0dd1bec42e8f64b key=acdd95754c654042a526847acb96f692 [acdd95754c654042a526847acb96f692] description=Web UI contact=me@example.com limit=0 [6d3a661b7401486b8a5d4f78962f8b80] description=API Key contact=batch@example.com limit=5000 priority=8 [acd5ffd840094cdda054a0b7e57d4546] contact=notme@example.com limit=100 Server API key To be used by the Web UI
  48. 48. API Keys [server] secret=7a5c742c021c414ca0dd1bec42e8f64b key=acdd95754c654042a526847acb96f692 [acdd95754c654042a526847acb96f692] description=Web UI contact=me@example.com limit=0 [6d3a661b7401486b8a5d4f78962f8b80] description=API Key contact=batch@example.com limit=5000 priority=8 [acd5ffd840094cdda054a0b7e57d4546] contact=notme@example.com limit=100 Each API key defined as an ini file section
  49. 49. API Keys [server] secret=7a5c742c021c414ca0dd1bec42e8f64b key=acdd95754c654042a526847acb96f692 [acdd95754c654042a526847acb96f692] description=Web UI contact=me@example.com limit=0 [6d3a661b7401486b8a5d4f78962f8b80] description=API Key contact=batch@example.com limit=5000 priority=8 [acd5ffd840094cdda054a0b7e57d4546] contact=notme@example.com limit=100 Contact info shows up in usage stats
  50. 50. API Keys [server] secret=7a5c742c021c414ca0dd1bec42e8f64b key=acdd95754c654042a526847acb96f692 [acdd95754c654042a526847acb96f692] description=Web UI contact=me@example.com limit=0 [6d3a661b7401486b8a5d4f78962f8b80] description=API Key contact=batch@example.com limit=5000 priority=8 [acd5ffd840094cdda054a0b7e57d4546] contact=notme@example.com limit=100 Allowed “Page Loads” per day (tests x runs) x 2 if also recording repeat view
  51. 51. API Keys [server] secret=7a5c742c021c414ca0dd1bec42e8f64b key=acdd95754c654042a526847acb96f692 [acdd95754c654042a526847acb96f692] description=Web UI contact=me@example.com limit=0 [6d3a661b7401486b8a5d4f78962f8b80] description=API Key contact=batch@example.com limit=5000 priority=8 [acd5ffd840094cdda054a0b7e57d4546] contact=notme@example.com limit=100 Forced test priority Defaults to 5 for API (can be specified in request)
  52. 52. API Keys [server] secret=7a5c742c021c414ca0dd1bec42e8f64b key=acdd95754c654042a526847acb96f692 [acdd95754c654042a526847acb96f692] description=Web UI contact=me@example.com limit=0 [6d3a661b7401486b8a5d4f78962f8b80] description=API Key contact=batch@example.com limit=5000 priority=8 [acd5ffd840094cdda054a0b7e57d4546] contact=notme@example.com limit=100 Minimal key configuration
  53. 53. Firewalls... Intranet WPT Web Server Desktop Agent Private Sites Public Desktop Agent Public Sites
  54. 54. Server Lock-down ● Use keys for test locations ● Open access to /work/* for test agents ● Auth access (or IP rules) for rest of the UI
  55. 55. WPT Web Server Firewalls... Private Sites Intranet Desktop Agent Public Desktop Agent Public Sites WPT Relay Server
  56. 56. Relay Server ● Configure locations for public agents ● Configure API keys for internal instance to use for relay ● add headless=1 to settings.ini ○ Disbles UI for submitting tests, requires API key for all tests ● Results are pulled back to internal instance ○ never visible on external
  57. 57. Relay Server - locations.ini ● Configure locations.ini like normal on the relay server ● Local server configures 1:1 mapping of locations ○ forwarded to the relay server [Public_EC2_East] browser=IE 11,Chrome,Firefox label="EC2 East (Public)" relayServer="http://<relay server>/" relayKey=<your API key> relayLocation=EC2_East Config like normal location API Key and Location ID on the Relay Server
  58. 58. Results Storage ● Each test is self-contained in a directory under results/ ● Grouped by year, month and day ○ results/YY/MM/DD/… ● Prune old tests? Delete directories.
  59. 59. Results Archiving ● WPT can seamlessly restore missing tests from “archive” ● Supports archiving to local path or S3-like storage ● Configure archive settings in settings.ini ● Test copied to archive after it completes (as 1 zip file per test) ● To delete old archived tests need to run/schedule script ○ php cli/archive.php ● Local Path: ○ zip files copied to specified directory ○ YY/MM/DD/<hash>/<id>.zip ● S3-like ○ zip files uploaded to bucket with <id>.zip filename ○ Can restore from URL or S3 API
  60. 60. Results Archiving - Local Path ● Settings.ini ○ archive_dir=/archive/ ○ archive_days=2 ● Local Path: ○ zip files copied to specified directory ○ YY/MM/DD/<hash>/<id>.zip ● 2nd Level archiving ○ settings.ini: archive2_dir=/mnt/archive/ ○ archive zip files merged into daily zip files ○ cli/archive2.php archive script
  61. 61. Results Archiving - S3 ● Settings.ini ○ archive_s3_server=s3.amazonaws.com ○ archive_s3_key=<access key> ○ archive_s3_secret=<secret> ○ archive_s3_bucket=<bucket> ● zip files uploaded to bucket with <id>.zip filename ● Can use URL access to restore ○ Defaults to using API ○ archive_s3_url=http://s3.amazonaws.com/
  62. 62. Bulk Testing in the UI (Private Instances)
  63. 63. Bulk Testing in the UI (Private Instances) ● Can be disabled through settings.ini ○ noBulk=1 ● Submit list of URLs to test ● UI options apply to each test ● Scripts can be run against each URL ○ %URL% in script gets replaced with test URL ○ %HOST% in script gets replaced with host from test URL ○ %HOSTR% in script gets replaced with host from URL after redirects ● Aggregate results
  64. 64. API - Submitting a test ● Anything you can do with the UI you can do with the API ○ If a field isn’t documented, just inspect the HTML on the UI to get the field name :-) ● Make sure to encode parameters ○ urlencode if GET ○ form encode if POST
  65. 65. API Response http://www.webpagetest.org/runtest.php?url=www.aol.com&f=json { "statusCode":200, "statusText":"Ok", "data":{ "testId":"140610_FY_N2F", "ownerKey":"04121a1e5b17a59ba9ddd666ae020cf52cee4abe", "jsonUrl":"http://www.webpagetest.org/jsonResult.php?test=140610_FY_N2F", "xmlUrl":"http://www.webpagetest.org/xmlResult/140610_FY_N2F/", "userUrl":"http://www.webpagetest.org/result/140610_FY_N2F/", "summaryCSV":"http://www.webpagetest.org/result/140610_FY_N2F/page_data.csv", "detailCSV":"http://www.webpagetest.org/result/140610_FY_N2F/requests.csv" } }
  66. 66. Node API Wrapper Created by Marcel Duran (@marcelduran) https://github.com/marcelduran/webpagetest-api $ npm install webpagetest -g webpagetest test -k <key> http://twitter.com/marcelduran webpagetest status 140610_F0_T5K webpagetest results 140610_F0_T5K webpagetest test http://twitter.com/marcelduran --poll 5 --timeout 60
  67. 67. CI Integration https://github.com/marcelduran/webpagetest-api/wiki/Test-Specs Match result to provided specs { "median": { "firstView": { "requests": 20, "render": 400, "loadTime": 3000, "score_gzip": { "min": 90 } } } }
  68. 68. CI Integration cont. Uses Mocha with reporters suitable for integration with: ● Jenkins ● Travis-CI ● Drone.io ● Just about anything else you can imagine
  69. 69. grunt-perfbudget Created by Tim Kadlec (@tkadlec) https://github.com/tkadlec/grunt-perfbudget
  70. 70. Results logging ● As each test comes in, full result data is written to log files ○ Page Data (load time, Speed index, requests, custom metrics, etc) ○ Request Data (details for every individual request) ● Each test run or request is on it’s own log file line ● JSON-formatted ● Config information is included ○ Test URL ○ Run # ○ First/Repeat View ○ Test label ○ Location ○ Browser ○ Connectivity ○ Test ID ○ URL to test result
  71. 71. Results logging - integration ● Splunk ● Logster ○ Logster -> StatsD = Arbitrary metrics trending automatically ● Track individual requests across all pages ○ ads JS performance ○ CDN response times ○ Effectiveness of image compression ○ so much more
  72. 72. Benchmarks (Private Instances) ● Still pretty experimental ● Recurring tests run automatically ○ Configuration is a bit ugly but flexible ● Tests are scheduled, run and aggregated automatically ○ Top-level trended aggregate view ○ All-metrics trended aggregate view ○ Per-page trended view ○ Scatter Plot for a given run ○ Filmstrip comparisons
  73. 73. Monitoring the System ● Hourly Agent Monitoring ● getLocations.php ● getTesters.php ● checkTesters.php ● Custom cron jobs/scripts
  74. 74. bit.ly/usingwebpagetest
  75. 75. Feedback Please!
  76. 76. Thank You!

×