by 李志和 (Bevis Lee)

1. STAF 在自動化測試上的延伸應用 –
TMSTAF (TrendMicro STAF)
李志和 (Bevis Lee)
Sr. Engineer, R&D Dept.
2012/06/10
1 6/12/2012 | Copyright 2012 Trend Micro Inc.

2. Agenda
• Brief Introduction of STAF
• What‟s TMSTAF?
• TMSTAF vs. QA automation
• Q&A
6/12/2012 | Copyright 2012 Trend Micro Inc. 2

3. Brief Introduction of STAF
6/12/2012 | Copyright 2012 Trend Micro Inc. 3

4. • History
– Created at IBM Austin in 1998
– Open-sourced (on SourceForge) in 2001
• http://staf.sourceforge.net/
6/12/2012 | Copyright 2012 Trend Micro Inc. 4

5. • STAF runs as a daemon process, called STAFProc
6/12/2012 | Copyright 2012 Trend Micro Inc. 5

6. What‟s TMSTAF?
6/12/2012 | Copyright 2012 Trend Micro Inc. 6

7. TMSTAF = Trend Micro STAF
6/12/2012 | Copyright 2012 Trend Micro Inc. 7

8. TMSTAF Architecture
6/12/2012 Confidential | Copyright 2012 Trend Micro Inc. 8

9. 6/12/2012 | Copyright 2012 Trend Micro Inc. 9

10. File Structure
6/12/2012 | Copyright 2012 Trend Micro Inc. 10

11. TMSTAF Features
6/12/2012 Confidential | Copyright 2012 Trend Micro Inc. 11

12. Testware concept included:
Testware configurations can be customized
Test suite, test case defined in Python syntax
Test process included: Setup, Test and Teardown
Execution log for tracing
XML test report w/ email notification
Command line interface
Cross-platform execution
6/12/2012 | Copyright 2012 Trend Micro Inc. 12

13. Testing Report
6/12/2012 | Copyright 2012 Trend Micro Inc. 13

14. Email Notification of Testing Result
6/12/2012 | Copyright 2012 Trend Micro Inc. 14

15. TMSTAF vs. QA‟s Automation
6/12/2012 | Copyright 2012 Trend Micro Inc. 15

16. QA‟s Automation Process
• Automation construction
Test Test cases Automation
Automation
cases convert to script pre-
environment
creation automation run and
setup
scripts debug
• Automation online
Integrate
Build
with build
quality
release
verification
process
6/12/2012 | Copyright 2012 Trend Micro Inc. 16

17. Test Cases Creation
6/12/2012 Confidential | Copyright 2012 Trend Micro Inc. 17

18. • Based on concept of „test suite‟, test case can be
created for both „manual‟ and „automation‟ use
6/12/2012 | Copyright 2012 Trend Micro Inc. 18

19. Automation
Environment Setup
6/12/2012 Confidential | Copyright 2012 Trend Micro Inc. 19

20. • Automation environment can be deployed easily
Automation Environment
TMSTAF
framework
Python 2.6 STAF 3.x
6/12/2012 | Copyright 2012 Trend Micro Inc. 20

21. Test Cases Convert to
Automation Scripts
6/12/2012 Confidential | Copyright 2012 Trend Micro Inc. 21

22. • Convert test case to automation scripts structurally
Automation
Test step
Module Test case
(automation
script)
Automation
Test step
Module
Automation
Module
6/12/2012 | Copyright 2012 Trend Micro Inc. 22

23. 6/12/2012 | Copyright 2012 Trend Micro Inc. 23

24. 6/12/2012 | Copyright 2012 Trend Micro Inc. 24

25. 3 Weapons…
6/12/2012 Confidential | Copyright 2012 Trend Micro Inc. 25

26. I - Scenario Mechanism
test0010.py
var1 = 111
var2 = 222
func = PythonFunction('moduleATester', 'testFunc1')
r = self.addTest(func, [var1,var2])
r.failUnlessEqual(0)
test0020.py
var1 = 333
var2 = 444
func = PythonFunction('moduleATester', 'testFunc1')
r = self.addTest(func, [var1,var2])
r.failUnlessEqual(0)
6/12/2012 | Copyright 2012 Trend Micro Inc. 26

27. test0010.py
var1 = 111
var2 = 222
self.setScenario('_scenario01.py')
test0020.py
var1 = 333
var2 = 444
self.setScenario('_scenario01.py')
_scenario01.py
func = PythonFunction('moduleATester', 'testFunc1')
result = self.addTest(func, [var1,var2])
result.failUnlessEqual(0)
6/12/2012 | Copyright 2012 Trend Micro Inc. 27

28. II - Setup, Teardown and Test procedures
Setup steps
Test Case Test steps
Teardown steps
6/12/2012 | Copyright 2012 Trend Micro Inc. 28

29. • Setup = addSetup()
• Test = addTest()
• Teardown = addTeardown()
6/12/2012 | Copyright 2012 Trend Micro Inc. 29

30. III - Result Verification
• Execution result of each steps (includes setup and
teardown) can be verified easily
Passed if ‘result’ = 0
6/12/2012 | Copyright 2012 Trend Micro Inc. 30

31. Script Pre-run and
Debug
6/12/2012 Confidential | Copyright 2012 Trend Micro Inc. 31

32. • More efficiency development of automation
Script
execution
Refinement Debugging
of script of script
6/12/2012 | Copyright 2012 Trend Micro Inc. 32

33. • Script execution from command line interface
6/12/2012 | Copyright 2012 Trend Micro Inc. 33

34.  “-h” Online Help
 Easy to Integrate
 Run Target Case
 Run Target Suite
6/12/2012 | Copyright 2012 Trend Micro Inc. 34

35. • Execution log for tracing/debugging
6/12/2012 | Copyright 2012 Trend Micro Inc. 35

36. 6/12/2012 | Copyright 2012 Trend Micro Inc. 36

37. Automation Online
6/12/2012 Confidential | Copyright 2012 Trend Micro Inc. 37

38. Build Release Flow
6/12/2012 | Copyright 2012 Trend Micro Inc. 38

39. 1
3
2
4
5
6/12/2012 | Copyright 2012 Trend Micro Inc. 39

40. Build Quality
Verification
6/12/2012 Confidential | Copyright 2012 Trend Micro Inc. 40

41. Mail Notifications
6/12/2012 | Copyright 2012 Trend Micro Inc. 41

42. 6/12/2012 | Copyright 2012 Trend Micro Inc. 42

43. 6/12/2012 | Copyright 2012 Trend Micro Inc. 43

44. Detail Testing Report
6/12/2012 | Copyright 2012 Trend Micro Inc. 44

45. 6/12/2012 | Copyright 2012 Trend Micro Inc. 45

46. 6/12/2012 | Copyright 2012 Trend Micro Inc. 46

47. 6/12/2012 | Copyright 2012 Trend Micro Inc. 47

48. 6/12/2012 | Copyright 2012 Trend Micro Inc. 48

49. 6/12/2012 | Copyright 2012 Trend Micro Inc. 49

50. 6/12/2012 | Copyright 2012 Trend Micro Inc. 50

51. 6/12/2012 | Copyright 2012 Trend Micro Inc. 51

52. So, with TMSTAF,
you can…
6/12/2012 Confidential | Copyright 2012 Trend Micro Inc. 52

53. Test
Test case can be used for both ‘manual’ and
cases creation
‘automation’ use, no extra effort needs
Automation
Easy to setup automation environment for
environment
development and execution
setup
Test cases
convert to Implement test cases with structural and flexible
automation mechanism
scripts
Automation script More efficiency development and trouble-shooting
pre-run and debug of automation
Integrate with
build release Easy integrate with standard build release process
process
Build quality Useful information in testing report can help to
verification identify issue more quickly
6/12/2012 | Copyright 2012 Trend Micro Inc. 53

54. Q&A
6/12/2012 | Copyright 2012 Trend Micro Inc. 54