SmokeTests - What, Why & How - ConFoo 2019
•
0 gefällt mir•355 views
These are the slides for the talk "SmokeTests - What, Why & How" @ ConFoo Montreal 2019 in Canada. 15th March 2019
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie SmokeTests - What, Why & How - ConFoo 2019
Ähnlich wie SmokeTests - What, Why & How - ConFoo 2019 (20)
Mehr von tech.kartenmacherei
Mehr von tech.kartenmacherei (8)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
SmokeTests - What, Why & How - ConFoo 2019
- 1. SmokeTests The what, why and how
- 2. Who am I? Sebastian Thoss Chapter Lead Backend Better ventures group
- 5. 01 02 03 04 05 Agenda Types of Tests SmokeTests What and Why How do SmokeTest work Recommended Server Architecture SmokeTests as a Service
- 8. Class 3 Class 2 Unit Tests Class 1 Class 4 Mock Mock
- 10. Test Pyramid INTEGRATION TESTS UNIT TESTS
- 11. Class 4Class 3 Integration Tests Class 1 Class 2 Mock Mock
- 12. Test Pyramid ACCEPTANCE TESTS INTEGRATION TESTS UNIT TESTS
- 13. Acceptance Tests Class 1 Class 2 Class 3 Class 4
- 14. Test Pyramid ACCEPTANCE TESTS INTEGRATION TESTS UNIT TESTS ?
- 15. SmokeTests: What & Why?
- 17. What are SmokeTests? In computer programming and software testing, smoke testing is preliminary testing to reveal simple failures severe enough to reject a prospective software release. Source: https://en.wikipedia.org/wiki/Smoke_testing
- 18. What are SmokeTests? 1. … be simple 2. … be fast 3. … test URLs and optional parameters too 4. … cover at least all URLs in google index 5. … be extended by every new URL 6. … never forget a URL 7. … use a (manual) maintained list of URLs SmokeTests should…
- 19. What are SmokeTests? 1. Status code 2. Time to first byte 3. If body is provided 4. Correct server What SmokeTests validate?
- 20. What are SmokeTests? SmokeTest Client HTTP 1.1/200 OK <html> <head> <title>Foo</title> <body> <div id="bar"><span>foobar</span></div> </body> </html> SmokeTests are NOT Acceptance Tests!
- 21. How do SmokeTests work
- 22. How do SmokeTests work? https://confoo.ca/en <html><body>…</body></html> TTFB: 65ms HTTP 1.1/200 OK SmokeTest Client CI Server Application to test Production Server
- 23. How do SmokeTests work? https://confoo.ca/foo <html><body>…</body></html> TTFB: 3ms HTTP 1.1/404 Not Found SmokeTest Client CI Server Production Server Application to test
- 24. class SmokeTest extends TestCase { /** * @dataProvider urlProvider */ public function testWebsiteUrl(Url $url) { $result = $this->sendGetRequest($url); $this->assertSame(200, $result->getStatusCode()); $this->assertNotEmpty($result->getBody()); $this->assertLessThanOrEqual(100, $result->getTimeToFirstByteInMilliseconds()); } public function urlProvider() { $urls = [‘http://www….de’, …]; $urlCollection = UrlCollection::fromStrings($urls); return $urlCollection->asDataProviderArray($urlCollection); } … Logic is in here
- 27. How do SmokeTests work? SmokeTest Client CI Server Application to test Production Server https://www.confoo.ca/baz https://www.confoo.ca/bar https://www.confoo.ca/en
- 28. How do SmokeTests work? https://www.confoo.ca/baz https://www.confoo.ca/bar https://www.confoo.ca/en TTFB: 34ms SmokeTest Client CI Server Application to test Production Server
- 29. How do SmokeTests work? https://www.confoo.ca/foobaz https://www.confoo.ca/bar https://www.confoo.ca/en SmokeTest Client CI Server Application to test Production Server
- 30. How do SmokeTests work? TTFB: 65ms https://www.confoo.ca/foobaz https://www.confoo.ca/bar https://www.confoo.ca/en SmokeTest Client CI Server Application to test Production Server
- 31. How do SmokeTests work? https://www.confoo.ca/123 https://www.confoo.ca/foobaz https://www.confoo.ca/bar SmokeTest Client CI Server Application to test Production Server
- 32. How do SmokeTests work? TTFB: 620ms https://www.confoo.ca/123 https://www.confoo.ca/foobaz https://www.confoo.ca/bar SmokeTest Client CI Server Application to test Production Server
- 34. How do SmokeTests work?
- 35. BUT BUT
- 36. There is a library for it DjThossi/smoke-testing-php
- 37. How do SmokeTests work? DataProvider Single Test Application HTTP Requests HTTP Responses PHPUnit Calls Result[ ] Result
- 38. class SmokeTest extends PHPUnit_Framework_TestCase { use SmokeTestTrait; /** * @dataProvider myDataProvider */ public function testExample(Result $result) { $this->assertSuccess($result); $this->assertTimeToFirstByte(new TimeToFirstByte(100), $result); $this->assertBodyNotEmpty($result); $this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result); } public function myDataProvider() { $urls = ['http://www.kartenmacherei.de', …]; $options = new SmokeTestOptions( UrlCollection::fromStrings($urls), new RequestTimeout(2), new FollowRedirects(true), new Concurrency(3), new BodyLength(500) ); return $this->runSmokeTests($options); }
- 39. class SmokeTest extends PHPUnit_Framework_TestCase { use SmokeTestTrait; /** * @dataProvider myDataProvider */ public function testExample(Result $result) { $this->assertSuccess($result); $this->assertTimeToFirstByte(new TimeToFirstByte(100), $result); $this->assertBodyNotEmpty($result); $this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result); } public function myDataProvider() { $urls = ['http://www.kartenmacherei.de', …]; $options = new SmokeTestOptions( UrlCollection::fromStrings($urls), new RequestTimeout(2), new FollowRedirects(true), new Concurrency(3), new BodyLength(500) ); return $this->runSmokeTests($options); }
- 40. class SmokeTest extends PHPUnit_Framework_TestCase { use SmokeTestTrait; /** * @dataProvider myDataProvider */ public function testExample(Result $result) { $this->assertSuccess($result); $this->assertTimeToFirstByte(new TimeToFirstByte(100), $result); $this->assertBodyNotEmpty($result); $this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result); } public function myDataProvider() { $urls = ['http://www.kartenmacherei.de', …]; $options = new SmokeTestOptions( UrlCollection::fromStrings($urls), new RequestTimeout(2), new FollowRedirects(true), new Concurrency(3), new BodyLength(500) ); return $this->runSmokeTests($options); }
- 41. class SmokeTest extends PHPUnit_Framework_TestCase { use SmokeTestTrait; /** * @dataProvider myDataProvider */ public function testExample(Result $result) { $this->assertSuccess($result); $this->assertTimeToFirstByte(new TimeToFirstByte(100), $result); $this->assertBodyNotEmpty($result); $this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result); } public function myDataProvider() { $urls = ['http://www.kartenmacherei.de', …]; $options = new SmokeTestOptions( UrlCollection::fromStrings($urls), new RequestTimeout(2), new FollowRedirects(true), new Concurrency(3), new BodyLength(500) ); return $this->runSmokeTests($options); }
- 42. class SmokeTest extends PHPUnit_Framework_TestCase { use SmokeTestTrait; /** * @dataProvider myDataProvider */ public function testExample(Result $result) { $this->assertSuccess($result); $this->assertTimeToFirstByte(new TimeToFirstByte(100), $result); $this->assertBodyNotEmpty($result); $this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result); } public function myDataProvider() { $urls = ['http://www.kartenmacherei.de', …]; $options = new SmokeTestOptions( UrlCollection::fromStrings($urls), new RequestTimeout(2), new FollowRedirects(true), new Concurrency(3), new BodyLength(500) ); return $this->runSmokeTests($options); }
- 43. class SmokeTest extends PHPUnit_Framework_TestCase { use SmokeTestTrait; /** * @dataProvider myDataProvider */ public function testExample(Result $result) { $this->assertSuccess($result); $this->assertTimeToFirstByte(new TimeToFirstByte(100), $result); $this->assertBodyNotEmpty($result); $this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result); } public function myDataProvider() { $urls = ['http://www.kartenmacherei.de', …]; $options = new SmokeTestOptions( UrlCollection::fromStrings($urls), new RequestTimeout(2), new FollowRedirects(true), new Concurrency(3), new BodyLength(500) ); return $this->runSmokeTests($options); }
- 44. How do SmokeTests work?
- 46. Webserver (Router) Webserver Application Box A K/V Store DB Webserver Application Box B K/V Store DB
- 47. Webserver (Router) Webserver Application Box A K/V Store DB Webserver Application Box B K/V Store DB active = A
- 48. Webserver (Router) Webserver Application Box A K/V Store DB Webserver Application Box B K/V Store DB active = B
- 50. Webserver Application Box A K/V Store DB Webserver (Router) Build Server Deploy Code active = B
- 51. Webserver Application Box A K/V Store DB Webserver (Router) Build Server Deploy Code Configure active = B
- 52. Webserver Application Box A K/V Store DB Webserver (Router) Build Server Deploy Code Configure Smoke Tests active = B
- 53. Webserver Application Box A K/V Store DB Webserver (Router) Build Server Deploy Code Configure Smoke Tests active = B
- 54. Webserver Application Box A K/V Store DB Webserver (Router) Build Server Deploy Code Configure Smoke Tests Switch to A active = Bactive = A
- 55. Webserver Application Box A K/V Store DB Webserver (Router) Build Server Deploy Code Configure Smoke Tests active = B
- 58. Smest.it
- 59. Smest.it
- 60. Smest.it
- 61. Smest.it
- 62. Smest.it
- 63. Smest.it
- 64. Smest.it Try it for free!