[2024]Digital Global Overview Report 2024 Meltwater.pdf
Â
C:\documents and settings\selvam.mc\my documents\automation testing process
1. Large - Scale Test Automation Production Date :09th Julyâ2010 Place :Chennai Olympia Tech Park, India Anbhu Selvam.MCâŚMS(CS).MBA. (Senior Software Quality Engineer, EDS. India)
2. Common Challenges Automation Technology âMultiplatform (cross-platform) support â˘Ability to interface with multiple platforms â˘Ease of extensibility to new platforms â˘Reusability of test suites/scripts across multiple platforms âOutput/Object validation support â˘High rate-of-change âInterface changes âFunctionality changes
3. Large-scale Automations Method-Centric Automation is Method-Centricâit is more than about the technology; it is about the effective application of technology âMaintainabilityâhandling high rate-of-change âReusabilityâability to reuse common test components without having to program/code âScalabilityâability to automate large volume of tests through reusability (and team-based cost efficiency) âVisibility â˘Tests auditable by management and non-programmer staff â˘Productivity is structurally measurable
4. Efficiency is Key! Efficiency Key Contributors â˘Test design Definition: A test (case) creation activity with intent to serve one or both of the following objectives: 1.Exposing bugs/errors (or show otherwise, it works as intended), and 2.Optimizing maintainability and scalability (especially for automation-readiness). â˘Test automation (large-scale) âMethod-centric, not tool-centric âTeam-based, not individual coders
5. Automation Strategy Must Include Efficient Practices âMethodology â˘High maintainability (low maintenance cost) â˘High reusability âTechnology â˘High extensibility â˘Object validation capability â˘People and process âHigh scalability (large-volume and team-based methodology) âJust-in-time automationâtest engineers are skilled at test design and automation can be done before code-complete and/or concurrent to exploratory testing âHigh manageability (high visibility)
6. "Agile" System Development Establish objectives (global) â˘Develop tests â˘Develop the system âin iterations, one subsystem at the time âtest-driven â˘Release Agile âHome Ground*â â˘Low mission-critical â˘Senior developers â˘Requirements change often â˘Small number of developers â˘Culture that thrives on chaos
7. ReservedRole of Automation Effective automation can allow tests to support agile system development â˘Automation should not dominate. Don't make it into an "agile automation" project â˘A keyword driven method is, in my view, essential for successful test development w/ automation in agile development environment âAutomation is separate from test development âUsually doesnât need user interaction (only some with testers) âAutomation focuses on action-keywords, not on tests
8. DEVELOPMENT Stakeholders Testers Agile Test Development Process AUTOMATION AUTOMATION AUTOMATION System Development Level âII Development
9. Recommendations Have a global test-design scheme âKeep it short and simple: focused on breakdown of the tests âThis global test design can (and usually should) be revisited throughout the project cycle â˘Separate (1) test objectives and (2) test cases âTest objectives are easier to create/describe than test cases â˘Use your stakeholders (specialists, power-users, etc.) wisely âFocus the efforts on the relevant tests âDonât bother them more than needed â˘Avoid involving them in low-and medium-level tests âObserve various types of input: â˘How does this work? â˘What do we need to test? â˘What is interesting to test/How can we break the system? â˘Keep automation separate âTest developers are not consumed by automation âUse keyword-driven approach (e.g., Action Based Testing)
13. Key Takeaways 1. Fully understand automation cost-of-ownership 2.Donât underestimate the challenge of keeping maintenance costs low 3.You need to get efficientâoptimize your volume of test to exceed 50% coverage 4.Efficiency is key, and it will come from excellent test design and automation methodology (e.g., action-driven), and a well architected framework technology 5.Minimize programming tests 6.High scalability comes from high reusability of common âactionsâ and team-based staffing model 7.High maintainability comes from keeping maintenance activities at the lowest level 8.High visibility in your automation program to give you control and measurability, which ultimately leads to manageability 9.Practice just-in-time automation 10.Have a global test-design scheme that separate test objectives from test cases; and automation from test development
15. Automation Cost of Ownership 1.Technology = Tool Licensing + Development/Customization Cost 2.Production Cost = *One-time Cost + **Recurring Cost *NEWâOne-time Cost of Design, Implementation & Execution âTest Case or Test Script Creation ÂťTest line creation ÂťSequencing test logic or order of test lines (test steps) ÂťTesting and debugging test script âFunction or Keyword Creation âInterface Capturing or Mapping âTest Dataset Creation **EXISTINGâRecurring Cost Test Execution & Maintenance Cost âMonitoring test suit execution âInvestigating and troubleshooting false negatives âMaintaining Test Case or Test Script ÂťTest line modification ÂťUpdating test logic or re-sequencing order of test lines (test steps) ÂťTesting and debugging updated test script ÂťFunction or Keyword Modification ÂťInterface Recapturing or Remapping ÂťTest Dataset Modification
16. The Cost of Automation The Cost of Owning an Automated Test Cost of an Automated Test*= **Cost of Ownership / Volume (#) of Tests * All test cases are not equalâa clear and structured definition of a test case is required. Furthermore, this cost will be a moving target over time ** Cost of Ownership = Technology Cost + Production Cost
17. The Need for Large Volume Cost of an Automated Test* =**Cost of Ownership/Volume (#) of Tests You need to get efficient by optimizing the Volume of Tests! * All test cases are not equalâa clear and structured definition of a test case is required. Furthermore, this cost will be a moving target over time ** Cost of Ownership = Technology Cost + Production Cost
18. Rules of Automation Built-to-Last 1.Tests are treated as product asset, along with the source code. 2.Tests, good or bad are dependent on the design. 3.Tests, manual or automated must be optimized for visibility, reusability, scalability and maintainability. 4.Tests must be automation-ready. 5.Tests, if they are worth automating, should follow the 5% rule: No more than 5% of all tests should be executed manually No more than 5% of all efforts around testing should involve automating the tests No more than 5% of coded test scripts against non-coded test scripts.