Motorola developed a new product and used machine learning to automatically analyze logs from users and customers to identify software bugs and suspicious behaviors. The system was trained over multiple weeks to improve its ability to accurately predict bugs and filter out false positives and negatives. Keywords were identified that helped determine whether issues were actual bugs or non-bugs, with some keywords like "tethering" being 92% accurate predictors of bugs.

2. ●
○
○
○
●
○
■
●
○

3. Motorola
Employees
CustomersNew Product

4. New Product
Generate logs
Filter suspicious
behaviours
Manual analysis
and triage
Fix

5. ●
●
●
●

6. ●
○
○
●
○

7. ●
○
○
○
○
○
○
●
○
○
○
○

8. ●
●
●
●
Train Predict
Time TimeWeek 1
Train Predict
Week 2
Train Predict
Week 3

9. ●
○
○
○
○
○
●
○
●
○

10. ●
○
○
●
○
○
○
●
○
○
○

11. ●
○
○
Bug thresholdNon-bug threshold

12. ●
○
○
■
■
●
○
○
○
Not Bug Bug
Not Bug 7,961
(90.86%)
801
(9.14%)
Bug 106
(4.35%)
2,329
(95.65%)
Predicted
Actual

13. 10% FP threshold
5% FN thresholdFN
FP

14. ●
●
○
○
●
○
○

15. ●
○
●
○
●
Keyword AUC
tethering 92%
signal 89%
radio 88%
pkey 88%
duplicate 86%
wakelock 83%
bluetooth 83%
screen 81%
kernel 81%
streaming 80%
gps 80%

16. ●
○
●
○
●
○