1.
MINT-Composer – A Toolchain for the Model- based Specification of Post-WIMP Interactors Sebastian FeuerstackDr.-Ing. Sebastian FeuerstackUniversidade Federal de São CarlosDepartamento de Computacão 18. Oktober 2012 Sebastian Feuerstack I Sebastian@Feuerstack.org

2.
Basic Question How to model and run multimodal Post-WIMP interactions?18. Oktober 2012 2

3.
Post-WIMP Interacton• WIMP interaction style: Window, Icon, Menu, Pointing device• Post-WIMP Interfaces ”…attempt to appear more natural and less obtrusive. Ideally this kind of post-WIMP interaction feels as familiar and fluid as using simple nondigital artifacts” [Jetter 2012] 18. Oktober 2012 3

4.
Interactive music sheet18. Oktober 2012 4

5.
Why modeling? • Visual language that – is declarative and precise to discuss & execute – Is located between a tool and source code – Supports different forms of interaction and flexible prototyping of multimodal interfaces18. Oktober 2012 5

6.
How to model? AUI Widget Mapping CUI CUI W W Modality D1• Models – Interactors (Abstract & Concrete Media, Mode) • Static: class diagram, Behavior: state-chart – Mappings • Custom Notation 18. Oktober 2012 6

7.
Development Process18. Oktober 2012 7

8.
Interactor Modeling 18. Oktober 2012 8

9.
Mode Modeling IR:Head NoHead discover lost Head Captured right_tilted center right centered left left_tilted center18. Oktober 2012 9

10.
Demonstration Home Network MINT- Head Platform Recognition App Download WebServer Home Gateway End-Device WebServer Webbrowser18. Oktober 2012 The Augmented “Drag-and-Drop 10

11.
Conclusions & Future Work What’s the advantage?• Detailed Modeling of Multimodal Interactions including behavior• Existing tools• No gap between design- and runtime 18. Oktober 2012 11

12.
Whats next?Focus on: – Fusion – Paradigm Design – Formalization – Tools – W3C Standardization http://www.multi-access.de All papers, Videos, MINT Framework as Open Source 18. Oktober 2012 Design of Multimodal Interaction 12

13.
18. Oktober 2012 13