1.
@BagmarAnand
Unlocking the Power
of
ChatGPT and AI in Testing
A Real-World Look
Anand Bagmar
Software Quality Evangelist

2.
ABOUT ME
BagmarAnand
linkedin.com/in/anandbagmar/

3.
Hype in AI

4.
Gartner’s Hype Cycle
for AI, 2021 report
https://www.gartner.com/en/articles/the-4-trends-that-prevail-
on-the-gartner-hype-cycle-for-ai-2021

5.
https://www.gartner.com/en/articles/the-4-trends-that-prevail-on-the-gartner-hype-cycle-for-ai-2021

6.
AGENDA
• What is ChatGPT?
• Using ChatGPT in Testing
• Using ChatGPT in Programming
• Limitations of ChatGPT
• Other AI Tools you can use
• Pitfalls of AI
• Summary
• Q&A

7.
@BagmarAnand
ChatGPT
https://openai.com/blog/chatgpt/

8.
@BagmarAnand
ChatGPT – in its own words

9.
@BagmarAnand
ChatGPT is evolving!

10.
@BagmarAnand
GPT-3
https://en.wikipedia.org/wiki/GPT-3

11.
@BagmarAnand
ChatGPT – Interesting Stats
https://www.demandsage.com/chatgpt-statistics/
• ChatGPT was launched on 30th November 2022.
• The new and improved embedding model of ChatGPT was launched on
15th December 2022.
• ChatGPT has crossed one million users within a week of its launch.
• ChatGPT can only fetch data prior to the year 2021 as its training
stopped in the year 2021.
• Microsoft Azure supports OpenAI and provides them with the
computational power required for running ChatGPT.

12.
ChatGPT in Testing
Example: Testing an eCommerce application – Amazon (USA)

13.
@BagmarAnand
Testing Approach
• Getting started
• Framework
• Design patterns
• Code optimization
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

14.
@BagmarAnand
Testing - Strategy
• Getting started
• Framework
• Design patterns
• Code optimization
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

15.
@BagmarAnand
Testing - Strategy
Question asked:
Create a test strategy for
testing and automating
Amazon USA

16.
@BagmarAnand
Testing - Strategy
Question asked:
Create a test strategy for
testing and automating
Amazon USA

17.
@BagmarAnand
Testing - Strategy
Question asked:
Create a test strategy for
testing and automating
Amazon USA This is not useful.
Seems like a template

18.
@BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case

19.
@BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case

20.
@BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case

21.
@BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case

22.
@BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case This is promising.
A great start!

23.
@BagmarAnand
Testing – Strategy with Risk Mitigation
Next question:
What risks and their
mitigation strategies
should be included in
the above strategy?

24.
@BagmarAnand
Testing – Strategy with Risk Mitigation
Next question:
What risks and their
mitigation strategies
should be included in
the above strategy?

25.
@BagmarAnand
Testing – Strategy with Risk Mitigation
Next question:
What risks and their
mitigation strategies
should be included in
the above strategy?

26.
@BagmarAnand
Testing – Test Design
• Getting started
• The 1st Test
• Design
patterns
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

27.
@BagmarAnand
Test Design – Test Scenarios
Next question:
What are the most
critical test scenarios I
should test for this
website?

28.
@BagmarAnand
Test Design – Test Scenarios
Next question:
What are the most
critical test scenarios I
should test for this
website?

29.
@BagmarAnand
Test Design – Test Scenarios
Next question:
What are the most
critical test scenarios I
should test for this
website?
Great set of regression
scenarios!

30.
@BagmarAnand
Test Design – Test Cases
Next question:
List 10 test cases for
"Product Search and
Filtering" scenario in
order of priority

31.
@BagmarAnand
Test Design – Test Cases
Next question:
List 10 test cases for
"Product Search and
Filtering" scenario in
order of priority

32.
@BagmarAnand
Test Design – Test Cases
Next question:
List 10 test cases for
"Product Search and
Filtering" scenario in
order of priority
Isn’t this amazing?

33.
@BagmarAnand
Test Design – Test Data
Next question:
What test data do I
need to be available
for performing the
above test cases?

34.
@BagmarAnand
Test Design – Test Data
Next question:
What test data do I
need to be available
for performing the
above test cases?

35.
@BagmarAnand
Test Design – Test Data
Next question:
What test data do I
need to be available
for performing the
above test cases?
Well – I was hoping for more
specific, but it’s a start!

36.
@BagmarAnand
Testing - Automation
• Getting started
• The 1st
Test
• Design patterns
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

37.
@BagmarAnand
Automation – Strategy
Next question:
What is a good way to
automate the above
test cases?

38.
@BagmarAnand
Automation – Strategy
Next question:
What is a good way to
automate the above
test cases?

39.
@BagmarAnand
Automation – Getting Started
Next question:
I don't know Selenium.
How should I learn?

40.
@BagmarAnand
Automation – Getting Started
Next question:
I don't know Selenium.
How should I learn?

41.
@BagmarAnand
Automation – Getting Started
Next question:
I don't know Selenium.
How should I learn? This answer took some time
TestAutomationUniversity
(TAU) was missed!

42.
@BagmarAnand
Automation – The 1st Test
Next question:
Implement the 1st test
case using Selenium
Java with Junit

43.
@BagmarAnand
Automation – The 1st Test
Next question:
Implement the 1st test
case using Selenium
Java with Junit

44.
@BagmarAnand
Automation – The 1st Test
Next question:
Implement the 1st test
case using Selenium
Java with Junit

45.
@BagmarAnand
Automation – The 1st Test
Next question:
Implement the 1st test
case using Selenium
Java with Junit

46.
@BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

47.
@BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

48.
@BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

49.
@BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

50.
@BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

51.
@BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

52.
@BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

53.
@BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

54.
@BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

55.
@BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

56.
@BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

57.
@BagmarAnand
Testing – Test Execution
• Getting started
• The 1st Test
• Design
patterns
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

58.
@BagmarAnand
Test Execution – Reports
Next question:
How to generate
reports and metrics for
automated tests?

59.
@BagmarAnand
Test Execution – Reports
Next question:
How to generate
reports and metrics for
automated tests?

60.
@BagmarAnand
Test Execution – Reports
Next question:
How to generate
reports and metrics for
automated tests?

61.
@BagmarAnand
Test Execution – Reports
Next question:
How to generate
reports and metrics for
automated tests?
This is not what I was looking for!

62.
@BagmarAnand
Test Execution – Reports
Follow-up question:
Show an example to
generate JunitReports
for the automated
tests

63.
@BagmarAnand
Test Execution – Reports
Follow-up question:
Show an example to
generate JunitReports
for the automated
tests

64.
@BagmarAnand
Test Execution – Reports
Follow-up question:
Show an example to
generate JunitReports
for the automated
tests

65.
ChatGPT in Programming

66.
@BagmarAnand
ChatGPT in Programming
Writing
code
Refactoring
Debugging
...

67.
@BagmarAnand
Writing code
Writing
code
Refactoring
Debugging
...

68.
@BagmarAnand
Writing code
Question asked:
Implement code for count-
and-say
https://leetcode.com/problems/count-and-say/

69.
@BagmarAnand
Writing code
Question asked:
Implement code for count-
and-say
https://leetcode.com/problems/count-and-say/

70.
@BagmarAnand
Writing code
Question asked:
Implement code for count-
and-say
https://leetcode.com/problems/count-and-say/

71.
@BagmarAnand
Writing code
Question asked:
Implement code for count-
and-say
https://leetcode.com/problems/count-and-say/

72.
@BagmarAnand
Writing code
Follow-up asked:
Give above solution in Java

73.
@BagmarAnand
Writing code
Follow-up asked:
Give above solution in Java

74.
@BagmarAnand
Writing code
Follow-up asked:
Give above solution in Java
This actually works J

75.
@BagmarAnand
Writing code
Follow-up asked:
Use proper variable names
for above code

76.
@BagmarAnand
Writing code
Follow-up asked:
Use proper variable names
for above code

77.
@BagmarAnand
Writing code
Follow-up asked:
Use proper variable names
in the loops

78.
@BagmarAnand
Writing code
Follow-up asked:
Use proper variable names
in the loops

79.
@BagmarAnand
Refactoring
Writing code Refactoring
Debugging
...

80.
@BagmarAnand
Refactor code
Question asked:
Given a link to code in
Github, refactor it

81.
@BagmarAnand
Refactor code
Question asked:
Given a link to code in
Github, refactor it

82.
@BagmarAnand
Refactor code
Question asked:
Given a link to code in
Github, refactor it
I have not verified the
validity of this refactoring

83.
@BagmarAnand
Question asked:
Given a snippet of
code, refactor it
Refactor code

84.
@BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it

85.
@BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it

86.
@BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it

87.
@BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it
Large output pauses abruptly
”continue” proceeds

88.
@BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it

89.
@BagmarAnand
Complex code
Question asked:
Given a complex code,
asked ChatGPT to
refactor the code, and
then reduce the
complexity

90.
@BagmarAnand
Complex code
Question asked:
Given a complex code,
asked ChatGPT to
refactor the code, and
then reduce the
complexity

91.
@BagmarAnand
Complex code
Question asked:
Given a complex code,
asked ChatGPT to
refactor the code, and
then reduce the
complexity

92.
@BagmarAnand
Complex code
Follow-up question:
Reduce complexity of
above code

93.
@BagmarAnand
Complex code
Follow-up question:
Reduce complexity of
above code

94.
@BagmarAnand
Debugging
Writing
code
Refactoring
Debugging
...

95.
@BagmarAnand
Debugging
Question asked:
Given a buggy code, fix it

96.
@BagmarAnand
Debugging
Question asked:
Given a buggy code, fix it

97.
@BagmarAnand
Debugging
Question asked:
Given a buggy code, fix it

98.
@BagmarAnand
Debugging
Question asked:
Given a buggy code, fix it
Not only fixed the code, ChatGPT
also explained the error

99.
Limitations of ChatGPT

100.
@BagmarAnand
Limitations of ChatGPT
https://openai.com/blog/chatgpt/

101.
@BagmarAnand
Limitations based on my observations
• ChatGPT does not know your context – use this as a starting point
• Will not work on internal sites
• Data privacy could become a huge issue
• Will not work for new features that are being developed

102.
AI tools impacting SDLC

103.
@BagmarAnand
Github Copilot
• Github Copilot is not new
• It is super helpful to write clean code, and rapidly!
• It is helpful to implement product code, as well as automation code
• Refer to these links for more information about it:
• https://applitools.com/blog/using-github-copilot-to-automate-tests/
• https://applitools.com/event/make-testing-easy-github-copilot/
• https://github.com/features/copilot

104.
@BagmarAnand
Applitools Visual AI
Applitools Ultrafast Cloud
applitools.info/b8h

105.
@BagmarAnand
Visual AI simplifies functional testing

106.
@BagmarAnand
Visual AI simplifies functional testing

107.
@BagmarAnand
Visual AI handles dynamic content

108.
@BagmarAnand
Visual AI handles dynamic content

109.
@BagmarAnand
Visual AI handles shifting content

110.
@BagmarAnand
Visual AI handles shifting content

111.
@BagmarAnand
Visual AI handles shifting content

112.
@BagmarAnand
Visual AI handles shifting content

113.
@BagmarAnand
Visual AI accelerates
cross browser/device testing
Capture the entire UI with
a single snapshot
DOM & CSS sent to
Applitools
Ultrafast Test Cloud
Applitools Eyes
Visual AI analysis
Ultrafast Grid
renders DOM
snapshots in parallel
Login to Eyes to
view test results
.
Firewall
18.2x faster than traditional cloud execution grids.
Applitools
Ultrafast Test Cloud

114.
@BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

115.
@BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

116.
@BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

117.
@BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

118.
@BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

119.
@BagmarAnand
Automation – With Applitools Visual AI
While this is not the
complete test, it
shows the correct
setup of Applitools
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

120.
@BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

121.
@BagmarAnand
Advantages of AI in Testing
• Reduce mundane tasks – let the machine handle it!
• Reduce the error prone nature of manual testing
• Easier Scaling
• Learn new tools & technologies
• Make work more interesting

122.
Potential Pitfalls of AI

123.
@BagmarAnand
Potential pitfalls of AI
• Thinking AI will solve all problems!
• Results in superficial knowledge
• Data Privacy
https://twitter.com/ProtonPrivacy/status/1619007351750627342

124.
Current Sentiment

125.
@BagmarAnand
https://nypost.com/2023/01/25/chat-gpt-could-make-these-jobs-obsolete/
https://www.semafor.com/article/01/27/2023/openai-has-hired-an-army-of-contractors-to-make-basic-coding-obsolete
Buzz in social media

126.
@BagmarAnand
Are you excited?
https://storage.needpix.com/rsynced_images/excited-2681489_1280.jpg
https://www.lavorareascuola.it/wp-content/uploads/2020/05/thinking-2681494_960_720.jpg
Current Sentiments
Or
Are you worried?

127.
Summary
• ChatGPT and many other tools in the AI-space are solving very
interesting challenges
• Understand how to use them in your context
• Ask the right question, to get a good answer!
• Tools have limitations – choose them wisely
• Human mind & experience is “still” the essential ingredient in
SDLC

128.
Some tips for you
Use this opportunity to:
• Upskill yourself in new tools and technologies
• Evolve your thought process
• Be creative and use new information to come with
unique solutions
• Be innovative and learn how to use tools and
technologies for different purposes

129.
https://twitter.com/paulg/status/1618676092163997696?t=J4XUNYJ8ynidWgrUW02Ing&s=03

130.
And a New Challenge
Have you thought of how will you test these new
AI tools and technologies?

131.
HI + AI = SI
Human Intelligence + Artificial Intelligence = Super Intelligence

132.
QUESTIONS?
BagmarAnand
linkedin.com/in/anandbagmar/

133.
QUESTIONS?
Talk with a Testing Specialist
applitools.info/lbd

134.
TRY APPLITOOLS
Try the only truly AI-driven
Visual Testing Platform with
a free-forever account
applitools.info/xm3

135.
@BagmarAnand
Additional Learning
applitools.info/cdf

136.
@BagmarAnand
Coming in March
applitools.info/7mp
TAU Conference
March 8-9
Let the Engineers Speak
March 16

137.
@BagmarAnand
• https://applitools.com/blog/ai-the-magical-helping-hand-in-testing/
• https://applitools.com
• https://openai.com/blog/chatgpt/
• https://github.com/features/copilot
Resources

138.
Thank You!
Join us in the lounge for additional networking