1. Random Stability Testing
Yu, Yucai
2013/11/05

2. 2
Agenda
• Why we do this?
• What problem we want to resolve?
• Model based solution
• Demo
• Deep dive

3. 3
Why we do this?
- Request from Toyota, part of IVI system testing
“Continue using for an extended period (48 hours)”
- No available solution for random stability test
- Leverage fMBT and iGAS
- Enhancement for flexibility, usability, portability

4. 4
What problem we should resolve?
- Do what user usually do, to cover kinds of activities
- Randomly switch among applications with some probability
- Identify Pass/Fail, handle failures
- Easy for issue analysis with context, get trend and statistics
- Portability for different platforms

5. 5
Model based solution
Action
Trigger
Result
Analysis
Exception
Handling
Modeling Implementing Executing Reporting

6. 6
Feedbacks
We found bugs:
• TIVI-1965 PA crashes after long time operations
• TIVI-1819 Web apps process still alive after launch/close many times
• TIVI-1864 weston will crash after random and continuous operation
• TIVI-1996 Click app has no response after switching app for many times
• TIVI-1988 The screen of all app list flashes sometimes after system boot up
Feedback from Jaska(murphy key developer):
• This is excellent for policy use case testing and when to open?

7. 7
We can do more…
Multi device control:
• Mobiles(Tizen, android) call each other
• Tizen IVI HFP call
• Dual OS

8. 8
Demo

9. 9
Deep dive
• Modeling
• Implementation
• Image recognition with iGAS
• Deal with failure

10. 10
Modeling
Using fMBT to create the model and drive test execution

11. 11

12. 12
Implement test steps
• Method 1: invoke external command(middleware scripts, iGAS etc.)
• Method 2: programming to gain great flexibility

13. 13
Programming test steps
fMBT’s device class library:
• Support several platforms: Tizen, android, windows, general Linux.
• Support function: Screenshot, tapBitmap(tapItem), verifyBitmap, Shell
etc.
Our compliment: event record and playback

14. 14
Event record and playback
Two benefit:
1. Solve some scenario that gfx recognition way not work well.
• Complex gesture
• Terminal operation
• App theme Change
• Record children style’s random operation
• Deal with sensors
2. Help create tests quickly(third party apps etc.)

15. 15
Image recognition with iGAS
• fMBT’s graphics engine is effective, but not very “smart”.
• Improve with iGAS’s smart image recognition engine: SIFT, pHASH etc.
• Improve with iGAS’s camera solution.

16. 16
How to deal with failure
Collect visual action logs
Collect app logs and system status
Monitor system resource: CPU, memory, process etc.
Collect crashes
Reboot and restart testing

17. Thanks