How EVERFI Moved from No Automation to Continuous Test Generation in 9 Months
•
0 gefällt mir•374 views
See and hear about EVERFI's journey to generating targeted tests automatically from changing system schemas in this webinar with Applitools, CircleCI, and Cypress. Greg Sypolt, VP of Quality Engineering, and Sneha Viswalingam, Director of Quality Engineering, share the time and productivity savings achieved through this approach, and how adopting shift-left test generation has shortened the QA cycle. * See the Applitools products used, including the Ultrafast Grid, at https://applitools.info/n3o * Read and download the case study at https://applitools.info/gbi
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie How EVERFI Moved from No Automation to Continuous Test Generation in 9 Months
Ähnlich wie How EVERFI Moved from No Automation to Continuous Test Generation in 9 Months (20)
Mehr von Applitools
Mehr von Applitools (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
How EVERFI Moved from No Automation to Continuous Test Generation in 9 Months
- 2. ABOUTUS @GREGSYPOLT IN/GREGSYPOLT GSYPOLT.GITHUB.IO GREG SYPOLT VICE PRESIDENT OF QUALITY ENGINEERING SNEHA VISWALINGAM DIRECTOR OF QUALITY ENGINEERING @SVISWALINGAM IN/SNEHAVISWALINGAM CODEHERSTORIES
- 3. •Like everything, we have to start at the base to drive positive change early on. Understand our testing problem. •How do we build for the future with model-based testing? How we modeled the system to increase our quality coverage. •The technologies driving the transformation. Learn how we are auto-generating Cypress.io test code and Applitools visual assertions from Curiosity Software model-based testing tool solution. •It's a journey; we are always thinking about how to become more e ff icient, faster, and not the bottleneck. KEYTAKEAWAYS
- 4. QUALITYEVOLUTION THE FUTURE Targeted Generation Cache Generated Tests Improved Branch Strategy Video and Audio Assertions Innovation Never Stops BEGINNING It calls for change, we are committed to answering it by introducing model-based testing. So we started the journey so that we could remove manual testing for 100+ courses with model-based testing. MONTH 6 MONTHS Innovation (Test Generator CLI) and Best Practices HOURS 3 MONTHS Utilizing test generation algorithms will recreate the entire system from a collection of models to generate Cypress and Applitools test code from those visual flowchart models. WEEKS 9 MONTHS Innovation (Model Course Generation CLI) 52 courses are automated by generating test code with seconds. SECONDS
- 6. SETOFPAINPOINTS •Traditional testing methods •Not all sections of the page tested •Not all permutations and combinations of the course were tested •Need to make tests re-usable •Quick learning time for folks from non-coding background •Front end testing is fragile •Ambiguous test cases
- 8. WHATISMODEL-BASEDTESTING(MBT)? • A model is a description of a system behavior (input sequences, actions, conditions, output, etc.) • Reusable, only de f ined once as digital twin of the system behavior • Your quality coverage of the system is documented • Test scripts automatically generated once the models are ready. Model System Executable Tests Is a partial description of Generation Run tests
- 10. WHYMBT WASTHEBESTCHOICEFOREVERFI OtherForms • In a common test script, you would write a test closure (describe , it), with a series of f ixed commands and assertions, that makeup scenarios. • Using page object model typically leads to duplication. • Copy and paste. Model-BasedTesting • Software behavior speci f ied once • Lower maintenance • 1:1 relationship between model and web component • Easier to develop and maintain • Very high coverage • Everything inside your model is validated • Test code from test to test are similar; beforeEach / afterEach hooks and it’s a collection of reusable functions • Requires a shift in mindset and culture • More up front effort and steep learning curve • Maximizes coverage with the least number of tests. The manual calculation is less ef f icient and less accurate than MBT options.
- 12. COMPONENTTESTGENERATION Test Modeller Test Engine JS Keyboard JS A11y JS Visual JS Mouse M K A V M K A V M K A V Parallelism: 12 V V V V V V V V V V V V Chrome 1024x768 Chrome 764x768 Safari 1024x768 Safari 764x768 JS Template Button Accordion Carousel Course Modeled Components Modello
- 13. COURSETESTGENERATION Button Accordion Carousel Video Applitools 80+ Components Test Generation Sequences Page 1 Page 2 Page 3 Page 4 Page 5 Page 1 Page 2 Page 3 Page 4 Test Modeller
- 14. CLOSERLOOKATMODELCODE const numberOfSlides = Cypress.$('.slick-dots > li').length; for(var i = 0; i < numberOfSlides-1; i++){ cy.get(`#hp-${i+1}`).click(); if (Cypress.env('APPLITOOLS_ENABLED')) { cy.eyesOpen({ testName: [testName] + ` ${i}`, }); cy.eyesCheckWindow({ target:'window', fully: true, }); cy.eyesClose(); } cy.get("BUTTON[aria-label='Next']").click(); } cy.get(`#hp-${numberOfSlides}`).click(); if (Cypress.env('APPLITOOLS_ENABLED')) { cy.eyesOpen({ testName: [testName] + ` ${numberOfSlides}`, }); cy.eyesCheckWindow({ target:'window', fully: true, }); cy.eyesClose(); } Carousel Button Modeling Course Page
- 20. HOWCANWEMAKETHISBETTER? •Large set of courses to generate automation •Course map changes •Course automation maintenance •Understanding all the reusable models and f iguring out where they are located in the Test Modeler •How can we shift left
- 25. BUILDANDTESTCOURSEWORKFLOW COURSE ARTIFACT TEST ENGINE UI TEST VISUAL TEST TESTING GRID MODELLO M odello COURSE BUILDER
- 26. MODELLO Parse and Transform to Data Structures Recurse through the course structures Course Pages iterated to create models for each page Sequential Map of Pages and Blocks
- 29. DATAPOINTSALONGOURJOURNEY Number of Courses Speed of Modeling System Quality Coverage Accelerated Transformation Shift Left Boosted Productivity Collaboration Quality of Delivery Cost Savings
- 31. THEFUTUREOFEVOLUTION Targeted Generation Cache Generated Tests Improved Branching Strategy Explore Video and Audio Assertions
- 32. CREDITS,WHOMADEALLTHISHAPPEN Other Credits IMAGES by unsplash.com LOGOS by f latiron.com Engineering and Product Teams dell M Scott Gunther Chris Barnes Subarna Karki Mosab Nazli Mark Nalbach Jennifer Maki James Moten Usman Raja Pallavi Pandey Juan Pablo Durante Sam Black Indu Rajasekaran Yoli Roca Sneha Viswalingam Greg Sypolt Event Sponsors
- 33. @GREGSYPOLT IN/GREGSYPOLT GSYPOLT.GITHUB.IO GREG SYPOLT VICE PRESIDENT OF QUALITY ENGINEERING SNEHA VISWALINGAM DIRECTOR OF QUALITY ENGINEERING @SVISWALINGAM IN/SNEHAVISWALINGAM CODEHERSTORIES