Myth vs Reality: Understanding AI/ML for QA Automation - w/ Jonathan Lipps

Myth vs Reality: Understanding AI/ML for QA Automation
Jonathan Lipps • Founding Principal • Cloud Grey
 
@AppiumDevs • @cloudgrey_io • @jlipps • appiumpro.com
Applitools Webinar · The Internet
January 31, 2020

Founding Principal
Architect, Maintainer
 

@jlipps · cloudgrey.io
AI == BS?
😮

“The core feature of a B.S.-industrial
complex is that every member of the
ecosystem knows about the charade, but is
incentivized to keep shoveling.”
source:https://hackerfall.com/story/ai-bs-industrial-complex-and-its-discontents

How is AI any diﬀerent from any other
software technology?

AI: anything a computer does that seems
smart
(not very helpful)

ML: “field of study that gives computers the
ability to learn without being explicitly
programmed.” - Arthur Samuel
(ok that’s a bit better)

Category of ML Main Idea
Supervised Learning Learn a function based on tagged inputs
Unsupervised Learning Learn classifications and patterns in untagged data
Reinforcement Learning
Learn by trial-and-error in a scenario that generates reward
feedback
Deep Learning A specific take on the use of neural networks

Category of ML Use Cases
Supervised Learning
Classify a new instance of data based on a trained model.
Predict a numeric quantity from new data.
Unsupervised Learning
Find patterns in a dataset that are difficult for human
researchers to spot.
Reinforcement Learning
Develop human-like reasoning in a well-defined task
environment.
Deep Learning Attack problems with very complex input data.

Category of ML Examples
Supervised Learning
Given information about a flower’s petal length, shape, and
other details, predict the species of flower.
Unsupervised Learning
Given a huge library of popular music, find natural
groupings of songs and see if they correspond to human
understandings of genre.
Reinforcement Learning Teach a bot to compete against humans in a video game.
Deep Learning
Given an image of an animal, classify it according to the
animal’s species.

Example ML Algorithms / Approaches Main Idea
Linear Regression
Used in supervised learning to learn a function which can
be applied to new inputs to get a scalar output value.
k-Means Clustering
Used in unsupervised learning to partition an n-dimensional
space based on natural groupings of data.
Neural Networks
Used in a variety of applications. Simulates the operations
of neurons to learn the weight of different values in an
input vector.
Generative Adversarial Networks
Pit two neural networks against each other in a kind of
‘imitation game’ in order to produce fake data that passes
for real data.

source: https://medium.com/machine-learning-for-humans/supervised-learning-740383a2feab

source: https://medium.com/machine-learning-for-humans/unsupervised-learning-f45587588294

source: https://www.youtube.com/watch?v=3lp9eN5JE2A&t=1631s

source: https://towardsdatascience.com/understanding-generative-adversarial-networks-gans-cd6e4651a29

Categories of “AI” solutions in QA Main Idea
AI in marketing only
Intelligently designed software that doesn’t use machine
learning models.
AI/ML in a supporting role
ML models are used to support features, not as a
replacement for test authoring.
AI/ML as the primary driver of automation
Tests are written and bugs found by autonomous bots
acting on pre- or post-trained ML models.

Categories of “AI” solutions in QA Example
AI in marketing only
Scrape production user activity logs to generate test cases.
Capture multiple selectors for elements to increase test
robustness.
AI/ML in a supporting role
Image recognition models to detect visual differences.
Video quality models give feedback on user-perceived
quality.
AI/ML as the primary driver of automation
You hand off the app to the AI with no additional metadata
and it sends you back bug reports.

AI == BS*
(with a few exceptions)

Do you need “AI” in your testing? Why?

Evaluate technologies based on their actual
ROI, not how well they claim the hype of
the zeitgeist.

A handy question to probe a product with:
“What corpus did you use to train your ML
model?”

Prediction: most actual ROI will be from AI/
ML in supporting roles, for a while yet.

Thank You!
Don’t forget to sign up for
Your free weekly Appium newsletter
appiumpro.com
 

Myth vs Reality: Understanding AI/ML for QA Automation - w/ Jonathan Lipps

Myth vs Reality: Understanding AI/ML for QA Automation - w/ Jonathan Lipps

Recomendados

Más contenido relacionado

Was ist angesagt?

Was ist angesagt?(20)

Similar a Myth vs Reality: Understanding AI/ML for QA Automation - w/ Jonathan Lipps

Similar a Myth vs Reality: Understanding AI/ML for QA Automation - w/ Jonathan Lipps(20)

Más de Applitools

Más de Applitools(20)

Último

Último(20)

Myth vs Reality: Understanding AI/ML for QA Automation - w/ Jonathan Lipps