This document summarizes a presentation on automated testing of multimodal applications. The presentation covers technical communication methods like buttons, graphics and touchscreens. It discusses fields of use for multimodal interfaces in logistics, entertainment, customer service and home automation. It also addresses VoiceXML as a standard for multimodal interaction and requirements for testing interfaces. The presentation includes demonstrations of testing approaches and code examples. It concludes with a question and answer section.

1. Automated Testing of
Multimodal Applications
Raphael Groner
ISTQB® Certified Tester
Bachelor of Science for Production Management
IT Specialist (IHK) - Application Development
http://www.xing.com/profile/Raphael_Groner
 voice-mediator @ email.de
http://www.jvoicexml.org
20.10.2014 M2M Summit 2014, Düsseldorf Sheet # 1

2. Init: What is it all about?
20.10.2014 M2M Summit 2014, Düsseldorf Sheet # 2

3. Agenda
1.Technical Communication
2.Fields of Use
3.MMI as a Standard
4.Requirements
5.Testing
6.Demo
7. Q&A
20.10.2014 M2M Summit 2014, Düsseldorf Sheet # 3

4. 1.Technical Communication
* Mechanical buttons
19-20th century
B. Preim, Development of Interactive Systems, 1999
Graphics and Touch
21st century
Apple Siri, iPhone6, 2014
+ Functions through Interaction +
20.10.2014 M2M Summit 2014, Düsseldorf Sheet # 4

5. 2.Fields of Use for MMI
• Logistics: „Pick by Voice“
 Commissioning & Inventory
 Voice-based Briefing
 Controlling via Identification
• Entertainment
 Multimedia & Games
 Social Networks
 Microsoft Kinect
• Inquiry desk and agency
 Travel, Stocks, Weather, …
 Call Center: „FAQ“
 Traffic Navigation
 Monitoring of Animals
• Home Automation
 DIY Automation Project
20.10.2014 M2M Summit 2014, Düsseldorf Sheet # 5

6. 3.MMI as a Standard
• Why?’ Theory of Communication
Naturally spoken and Formalised Languages
Interaction between People & Machine
How interact Machines among each other?
• VoiceXML is an extra W3C and XML standard
Version 2.1 is Current
Version 3.0 is Draft
Deployment like HTML
Editors, Generators
VoiceXML Forum
20.10.2014 M2M Summit 2014, Düsseldorf Sheet # 6

7. 4.Requirements
 Text interface available (VoiceXML)
 Automation & Scripting enabled
 Validation of Inputs und Outputs
 Input done with Keys or Spoken
 Output with Emphasis and Idioms or Dialects
 Output checked for correct Parts
 Combination of Conditions
 Delay of Inputs meaning „Timeout“
 …
20.10.2014 M2M Summit 2014, Düsseldorf Sheet # 7

8. 5.a)Test in a Process
http://www.scrum-kompakt.de
20.10.2014 M2M Summit 2014, Düsseldorf Sheet # 8

9. 5.b)Test Approaches
• Javascript’ Voxeo Phono
• C/C++’ OpenVXI, publicVoiceXML, …
• Stress/Penetration’ Voice Prover, PAGI, …
• „Flow objects“’ SpeakRight
• Java’ JVoiceXML with JUnit
20.10.2014 M2M Summit 2014, Düsseldorf Sheet # 9

10. 6.Demo
<?xml version="1.0" encoding="UTF-8"?>
<vxml xmlns="http://www.w3.org/2001/vxml" version="2.1">
<form>
<field name="answer">
<grammar src="yesno.srgs" type="application/srgs+xml"/>
<prompt>Do you like this example?</prompt>
<filled>
<if cond="answer=='yes'">
You like this example.
<else/>
You do not like this example.
</if>
</filled>
</field>
</form>
</vxml>
20.10.2014 M2M Summit 2014, Düsseldorf Sheet # 10

11. 7.Q&A
20.10.2014 M2M Summit 2014, Düsseldorf Sheet # 11