Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Will Robots Replace Testers

Will tools and automation continue to support testing or will robots replace testers in the future? This talk sets the scene and perhaps a direction for the future of tools and automation in testing.

Right now, the software world is going “bot mad”. It looks like many jobs in the next ten to twenty years will be done by bots and those jobs will effectively disappear as career choices. Inevitably, there has been some talk of testers being replaced by bots and tools. The common response to date has been to say, “Impossible!” But I’m not sure such a kneejerk reaction is sensible.

Futurists might suggest the destination is intelligent robot testers. I’m not sure that is where we are heading. The next steps we take will not require sophisticated AI or Deep Learning. But the next generation of testing tools will force a change of thinking and culture. Our goals with tools will change too and we may then have a clearer view of where we are heading. Tools that use ML may then be part of the tester’s armoury.

This talk suggests how we might make sense of the tools landscape of the near future, where the pressure to modernise processes and automate is greatest, and what a new test process supported by tools might look like.

Ähnliche Bücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Ähnliche Hörbücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen
  • Als Erste(r) kommentieren

Will Robots Replace Testers

  1. 1. Will Robots ReplaceTesters? @paul_gerrard Paul Gerrard paul@gerrardconsulting.com gerrardconsulting.com
  2. 2. Summary • Impact and influence of tools and bots on testing is increasing • What is the direction of travel for tools/bots? • Will the way we test be transformed? • Do we need to prepare for a traumatic change? • How will tools and automation support testing (or potentially replace) testers? This session is based on “The Future of Tools in Testing”: https://tkbase.com/resources/viewResource/14 Intelligent Definition and Assurance Slide 2
  3. 3. Software world goes “bot mad” • Many jobs in the next ten to twenty years will be done by bots and those jobs will effectively disappear as career choices • Some talk of testers being replaced by bots and tools • The common response:“Impossible!” • I’m not so sure anymore • Let’s explore what tools can do for us in a different way than you may be used to. Intelligent Definition and Assurance Slide 3
  4. 4. Robots won’t replace testers for some time • My thesis: new tools that support exploring, thinking, recording and reporting will emerge • Is the destination intelligent robot testers? • The next steps we take will not require sophisticated AI or Deep/Machine Learning – Our goals with tools will change – Different goals force a change of thinking and culture • NextGen tools will pave the way for AI/ML • I am building one. Intelligent Definition and Assurance Slide 4 From now on, Ill use the term Machine Learning or ML to refer to AI and Deep Learning.
  5. 5. A milestone in human achievement? • In March 2016, a computer beat the best human player of Go for the first time • Google’s AlphaGo program beat Lee Sedol the greatest living player, by four games to one. Intelligent Definition and Assurance Slide 5
  6. 6. Rules of Go • Rule 1 (the rule of liberty) Every stone remaining on the board must have at least one open "point" (an intersection, called a "liberty") directly next to it (up, down, left, or right), or must be part of a connected group that has at least one such open point ("liberty") next to it. Stones or groups of stones which lose their last liberty are removed from the board. • Rule 2 (the "ko rule") The stones on the board must never repeat a previous position of stones. Moves which would do so are forbidden, and thus only moves elsewhere on the board are permitted that turn. • All other information about the game is heuristic – learned through experience of play • Chess: 10120 possible moves • Go: 10761 possible moves a mere 10641 times as many. Intelligent Definition and Assurance Slide 6
  7. 7. Why is AlphaGo significant? • There is no possibility of computing all (or even the next few) Go moves by computer • Humans recognise patterns, play by intuition and imagination • Is AlphaGo simulating human intuition and imagination? • Like Go, testing is simple in theory, but is highly complex in practice • Could testing be computerised in the same way? Intelligent Definition and Assurance Slide 7
  8. 8. A recent study*… • Over the next two decades, 47% of jobs in the US may be under threat • It ranks 702 occupations in order of their probability of computerisation – Telemarketers: 99% likely – Recreational therapists: 0.28% likely – Computer programmers: 48% likely • Something significant is going on out there – If programmers have a 50/50 chance of being replaced by robots, we should think seriously about how the same might happen to testers. Intelligent Definition and Assurance Slide 8 * “The future of employment: how susceptible are jobs to computerisation?” http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf
  9. 9. Some systems-related occupations Intelligent Definition and Assurance Slide 9 Occupation Rank (out of 702) Probability of Computeris- ation Computer and Information Research Scientists 69 1.5% Network and Computer Systems Administrators 109 3.0% Computer and Information Systems Managers 118 3.5% Information Security Analysts, Web Developers, and Computer Network Architects 208 21% Computer Occupations, All Other 212 22% Computer Programmers 293 48% Computer Support Specialists 359 65% Computer Operators 428 78% Inspectors, Testers, Sorters, Samplers and Weighers 670 98%
  10. 10. Some observations • The ‘robots are coming’ meme implies that it is ML that is the driver for all this – Much of this is hype, with the industry trying to sell the next big thing to business – Nothing new there • Often, there is little or no need for ML – Inspectors, testers and telesales are likely to be replaced by sensors and data collectors in factories or Interactive Voice Response (IVR) systems – Data is larger and analysed in more sophisticated ways – The human interaction in those occupations isn’t sophisticated. Intelligent Definition and Assurance Slide 10
  11. 11. Test Automation = Mechanical Tools What we REALLY need are THINKING TOOLS
  12. 12. Intelligent Definition and Assurance The term Test Automation misleads • It misleads as a label because the whole of testing cannot be automated • The label is bad, but the scope of Test Automation is what I call ‘Applying’ in the New Model of Testing Slide 12
  13. 13. Test Automation: MechanicalTools • Test execution tools have been around since the 1970s • Other tools in this category are those which perform logistical or practical tasks: – Creation and management of environments and data – Test harnesses – Mocking – Set-up, tear-down and clean-up • These tasks have always been part of the test execution process • Modern tools are slicker but these tools have not evolved – The technical environments have changed but… – All of these tasks could be done ‘manually’ – at least in principle. Intelligent Definition and Assurance Slide 13
  14. 14. Testers need ThinkingTools • There are ten testing activities in the New Model – Test automation tools only support one:‘Applying’ • The remaining nine activities (information gathering, analysis, modelling, challenging, test design and so on) are not well supported • All require some level of thinking and skills • Checking is possible when a system and its purpose are well understood and trusted • Test automation tools are simple in principle… … compared to the rest of the test process. Intelligent Definition and Assurance Slide 14
  15. 15. Requirements for thinking tools • The tasks to be supported include: – Discussing and debating requirements and their sources – Creating predictive models of system behaviour – Identifying knowledge gaps; challenging sources – Creating models of usage, hazards, risks, failure modes, extreme or erroneous behaviour – Deciding when a model is adequate or inadequate – Deciding what to do next from a test outcome – And so on… • These are Human or so called Wicked Problems • For now, tools must focus on the what, not the how. Intelligent Definition and Assurance Slide 15
  16. 16. We can’t solve the Wicked Problem but… • “Testing is an information, intelligence or evidence- gathering activity performed on behalf of (testing) stakeholders to support their decision-making” • Can we create tools to support tester thinking activities and capture that thinking? • Perhaps the best we can do for now: – Support human thinking and collaboration – Look after the paperwork – Integrate with test automation (the easy part). Intelligent Definition and Assurance Slide 16
  17. 17. Two dimensions of tool capability • There are several dimensions of tool capability sophistication perhaps • Let’s start with a two-dimensional perspective 1. Notetaking, data capture and modelling capability which I’ll call the ‘Ability to Capture Knowledge’ 2. The second dimension relates more to knowledge acquisition. Let’s call that the ‘Ability to Investigate’ • I feel a four quadrant model coming on (yes, I hate them too). Intelligent Definition and Assurance Slide 17
  18. 18. Four quadrant model of intelligent test tools Ability to Investigate AbilitytoCaptureKnowledge • Text editors, Screen Shots Models, visualisations, relationships, transformations • Note Takers • Mind Maps • UML/Case Tools Control,imagination,discernment,foresight • Pencil and paper, sketching tools Intelligent Definition and Assurance Slide 18
  19. 19. Ability to Capture Knowledge • Humble text editors and screen shot utilities • Pencil and paper (better than many software tools) – Freehand sketches do not limit your imagination • Dedicated modelling tools using UML are placed highest – They provide a structure, consistency checking to some degree and some transformational capabilities which simple drawing or modelling tools cannot match – But you are limited to the models the tools can manage • We may (or may not) have reached half way up this scale • Tools that give our imagination free reign and perform validation, consistency checking or transformations, do not yet exist. Intelligent Definition and Assurance Slide 19
  20. 20. Ability to Investigate • The lowest capability: – the tester does all the thinking and has complete control • The highest capability: – the tool is capable of asking its own questions, discover its own information, make its own models, judge on the relevance, completeness and accuracy of the information it acquires – The tool does all of the thinking required • Today, all tools are bottom feeders in this respect. Intelligent Definition and Assurance Slide 20
  21. 21. What is this model useful for? • All of the tools I mention are on extreme left, mostly towards the bottom left • Is the model useful for anything? • It’s less about classification of tools; it’s more a suggestion of the roadmap our tools might take • Let’s consider the situation from another perspective – that of the medical profession. Intelligent Definition and Assurance Slide 21
  22. 22. Compare the diagnosis of illnesses to testing • Doctors ask questions, look for symptoms, take measurements • Many ailments can be identified within a few minutes, most within hours • Well defined procedures can be performed by bots* • Doctors won’t be replaced by bots soon because – Patients like dealing with humans – Doctors are a powerful lobby (in the UK at least) • Testers can’t rely on their lobbying power or public support to resist automation of their roles. * “The Robot Will See You Now” http://www.theatlantic.com/magazine/archive/2013/03/the-robot-will-see-you-now/309216/ Intelligent Definition and Assurance Slide 22
  23. 23. Future ofTools What tools can we expect to emerge in the next few years?
  24. 24. Vendors and the tools market • To date, the tool vendors have picked the low- hanging fruit of Mechanical Tools – The market for test automation is crowded – Open source tools are on the march • The unexploited market in tools that support system exploration, collaboration and test design could be much larger than test execution tools at least • All testers need them – (how many testers? 1 million, 2 million, X million?) Intelligent Definition and Assurance Slide 24
  25. 25. Exploration support • Frustration with testers: – testers are unimaginative, working by-rote – constant pressure to cut costs • Productivity of exploratory test approaches is proven • Testers want to explore, but the need for control and documentation constrains them • Testers needs tools that can capture plans and tester activity in real-time • The next generation will be led by tools that support the exploration of sources of knowledge. • These tools might use a “Surveying” metaphor. Intelligent Definition and Assurance Slide 25
  26. 26. A new test process? • The “tester as surveyor” affects the relationship of testing to development • A new style of testing process emerges: – Test documentation not created in a knowledge vacuum – Iterative, incremental knowledge acquisition and capture process closely aligned with the delivery of features • Could this be an Agile test process at last? • At least: it fits the increasingly popular Continuous Delivery, DevOps development approaches. Intelligent Definition and Assurance Slide 26
  27. 27. System Surveying • A System Survey captures features and the architecture of the system from a test perspective – Testers pair with developers and survey features – The knowledge required to design and build systems emerges over time – So do the models produced by testers • Surveys that evolve the System Model/Map are shared • The tester surveys paths through the architecture – Model connections are derived from the paths of exploration • No need for extensive scripts or test procedures! – Heard that one before? – The information required for scripting is in the model. Intelligent Definition and Assurance Slide 27
  28. 28. A scaleable, automatable process • Test process comprises a sequence of parallel actions – Sequence: survey, model refinement then testing – Parallel: small subsets of functionality selected for surveys – These processes are both iterative and incremental as learning proceeds • Scalable: if you survey it, you can test it • Automatable: What you can survey and test, you can probably automate • “Humans make the early maps; tools will follow the trails we make.” • We don’t need Machine Learning to do this: – Simple tools make suggestions that better inform and enrich exploration and testing. Intelligent Definition and Assurance Slide 28
  29. 29. What effect will Machine Learning have on testers? • Tester surveys are the source of data for bots: – Queries, observations, ideas, concerns mapped to the system model are a source of data for analysis – We will need a format and protocol for the information we capture for the bots to work their magic • More likely that developers are affected by ML – In a few years, some component development and unit testing could be wholly automated – It would remove a little of the uncertainty that testers face and may make the tester job a little easier • We’ll have to wait a bit longer for TerminatorTester. Intelligent Definition and Assurance Slide 29
  30. 30. Intelligent Definition and Assurance Slide 30 TERMINATOR TESTER Not Yet!
  31. 31. Will Robots ReplaceTesters? @paul_gerrard Paul Gerrard paul@gerrardconsulting.com gerrardconsulting.com

    Als Erste(r) kommentieren

    Loggen Sie sich ein, um Kommentare anzuzeigen.

  • PavanKumarBoya

    Apr. 19, 2017
  • ananp11

    Apr. 19, 2017

Will tools and automation continue to support testing or will robots replace testers in the future? This talk sets the scene and perhaps a direction for the future of tools and automation in testing. Right now, the software world is going “bot mad”. It looks like many jobs in the next ten to twenty years will be done by bots and those jobs will effectively disappear as career choices. Inevitably, there has been some talk of testers being replaced by bots and tools. The common response to date has been to say, “Impossible!” But I’m not sure such a kneejerk reaction is sensible. Futurists might suggest the destination is intelligent robot testers. I’m not sure that is where we are heading. The next steps we take will not require sophisticated AI or Deep Learning. But the next generation of testing tools will force a change of thinking and culture. Our goals with tools will change too and we may then have a clearer view of where we are heading. Tools that use ML may then be part of the tester’s armoury. This talk suggests how we might make sense of the tools landscape of the near future, where the pressure to modernise processes and automate is greatest, and what a new test process supported by tools might look like.

Aufrufe

Aufrufe insgesamt

309

Auf Slideshare

0

Aus Einbettungen

0

Anzahl der Einbettungen

4

Befehle

Downloads

4

Geteilt

0

Kommentare

0

Likes

2

×