Search test robot presentation in Milan QE Conference

1. Search Test Robot – a Search automation Solution
-Basavaraj M
-Vertical Intent
Search
-Innovation in
Automation
1
08/14/12

2. Problem Statement
Yahoo’s search motto – Answers, not links!
Showing results directly on search page – for number of user query
intentions, for huge number of queries.
2
08/14/12

3. Problem Statement
3
08/14/12

4. Concept
http://search.yahoo.com/search?p=cricket+score
$[query]
http://search.yahoo.com/search?p=restaurants+in+london
http://search.yahoo.com/search?p=hotels+in+london
http://search.yahoo.com/search?p=places+to+see+in+london
$[type] + $[destination]
http://shine.yahoo.com/channel/food/recipes/mixed-mushroom-pizza-534026/
http://shine.yahoo.com/channel/food/recipes/root-vegetable-stew-with-herbed-
dumplings-1900207/
$[channelType]/$[topic]/$[article]
Pattern in the URLs + data ====> Test cases
Validations ==Depend on==> data inputs
Data inputs ===Have===> parameters, with one-one / one-many relationships
4
08/14/12

5. Solution & Applicability
5
08/14/12

6. Solution & Applicability
An example to illustrate url generation inputs for robot :
url=http://$[host]/search?p=$[query_part1]+and+$[query_part2]%20%20$
[query_part3]....
host =host1
query_part1=file::dataFile1.txt,file::dataFile2.txt
query_part2=file::dataFile3.csv
query_part3=value1,value2,value3,value4....
# Mapping Relationship between validators and input data
Validator1=dataFile1.txt
Validator2=dataFile2.txt,dataFile3.csv
6
08/14/12

7. Solution & Applicability
7
08/14/12

8. Case Study
• A case study, in travel intent search test team -
•
8
08/14/12

9. Demo
9
08/14/12

10. What are the BENEFITS?
• It is a generic framework – can be used to automate front end or
back end system cases. It has been used for both FE and BE in
VIS.
• EXTENSIBILITY – Can be extended to automate any FE / BE
system automation.
• It can be used for automation of other sites than search also.
Example : yahoo properties – shine, shopping…
• Huge time saved both in automation design and automation
runtimes.
• Even though no Browser/UI involved, for failure cases, the html
file is saved, for later debugging.
• The utility of comparing any two automation runs to generate a
delta report.
• Independent of browsers. 10
08/14/12

11. Can I use this for my project?
• Is the SUT is data intensive? Is there a need to consider large scale
inputs?
• Are there limited set of functionalities, but correspondingly huge test data?
i.e. Data driven test automation is what you are looking at?
• Is the SUT depends on multiple other components whose integration
makes the whole system work? This requires to measure the system
functionality, between any two automation runs.
• This approach, this test robot is the best solution.
11
08/14/12

12. Limitations
• The current implementation doesn’t support appealing reporting
mechanism. It generates a detailed text reports. HTML reporting has to be
implemented.
• The current implementation doesn’t support java script execution. With the
integration of HTTP Unit in executor, this can be achieved. With HTTPUnit it
can be extended for functional test cases automation too.
• The current implementation doesn’t support the persistence of test state
and updation of reports at runtime. This will help manage the interruptions in
automation runs.
• The multi thread support, HTML parsing, reflection callbacks are being
implemented.
12
08/14/12

13. Summary
•It is not feasible to go with traditional UI automation for search
domain projects / data intensive projects.
•Search Test Robot – is definitely a smarter way of automation
reducing the automation design and automation times by ~80%.
13
08/14/12

14. References
• Gray box testing - http://en.wikipedia.org/wiki/Software_testing -
Grey_box_testing
• HTTP Unit - http://httpunit.sourceforge.net/
• http://twiki.corp.yahoo.com/view/QABlr/YSearchTestRobotProductBacklog
• http://twiki.corp.yahoo.com/view/QABlr/YSearchTestRobot
•Q&A
14
08/14/12

15. Q&A
Thank You!
15
08/14/12