hcid2011 - Gesture Based Interfaces: Jacques chueke (HCID, City University London)
•Als PPTX, PDF herunterladen•
1 gefällt mir•3,806 views
One of the many talks at hcid2011 held at City University London on 19th April 2011
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (20)
Ähnlich wie hcid2011 - Gesture Based Interfaces: Jacques chueke (HCID, City University London)
Ähnlich wie hcid2011 - Gesture Based Interfaces: Jacques chueke (HCID, City University London) (20)
Mehr von City University London
Mehr von City University London (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
hcid2011 - Gesture Based Interfaces: Jacques chueke (HCID, City University London)
- 1. Beyond Mouse & Keyboard: Post-WIMP and Novel Forms of Interaction Jacques Chueke Open Day, Centre for HCI Design London, UK, April 2011 Master in Design, PUC-Rio, RJ, Brazil PhD Researcher at the Centre for HCI Design Faculty of Informatics, City University London 1
- 2. “The most profound technologies are those that disappear. They wave themselves into the fabric of everyday life until they are indistinguishable from it.” Weiser (1991) Is this possible? Is this desirable? 2
- 3. It’s not supposed to be like this… Gmail Motion, April 2011 3
- 9. (…) The notion of the invisible interface correctly identifies the inflexible obtrusiveness of conventional interfaces as problematic. Embodied interaction provides some conceptual tolls for understanding how the interface might move into the background without disappearing altogether. Dourish (2005: 202-203) 10
- 13. Perceptible Affordances in Digital Art Digital Art [INTERFACE LAYER] OUTPUT - No Prior-Learning, besides knowledge of the world - Reactive SYMBOLIC / SYNESTHESIC / ABSTRACT TUI/NUI/RBI/VR/AR/BMI/OUI [MODE OF INTERACTION LAYER] INPUT 14
- 15. We propose a view that identifies some fraction of a user interface as based on the Post-WIMP theme (1) plus some other fraction that provides computer-only functionality (2) that is not realistic. As a design approach or metric, the goal would be to make the first category as large as possible and use the second only as necessary, highlighting the tradeoff explicitly. Jacob at al, 2008 15
- 16. Bibliography Beaudouin-Lafon, M. (November 2000). "Instrumental Interaction: An Interaction Model for Designing Post-WIMP User Interfaces". CHI '00: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. The Hague, The Netherlands: ACM Press. pp. 446–453. doi:10.1145/332040.332473. ISBN 1-58113-216-6. http://www.daimi.au.dk/CPnets/CPN2000/download/chi2000.pdf. Breeze, James. Eye Tracking: Best Way to Test Rich App Usability. UX Magazine, access on 25 November 2010. (http://www.uxmag.com/technology/eye-tracking-the-best-way-to-test-rich-app-usability) Buxton, W. (2001). Less is More (More or Less), in P. Denning (Ed.), The Invisible Future: The seamless integration of technology in everyday life. New York: McGraw Hill, 145–179 ITU Internet Reports 2005: The Internet of Things – Executive Summary. Dam, A. (February 1997). "POST-WIMP User Interfaces". Communications of the ACM (ACM Press) 40 (2): pp. 63–67. doi:10.1145/253671.253708. Dourish, P. Where the Action Is: The Foundations of Embodied Interaction. A Bradford Book: The MIT Press, USA, 2004. Gaver, W. Technology Affordances. Copyright 1991 ACM 0-89791-383-3/91/0004/0079. Gentner, D. and Nielsen, J. (April 1993). "The Anti-Mac Interface". Communications of the ACM (ACM Press) 39 (8): pp. 70–82. http://www.useit.com/papers/anti-mac.html. Jacob, R. et al. (2008). "Reality-Based Interaction: A Framework for Post-WIMP Interfaces". CHI '08: Proceedings of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems. Florence, Italy: ACM. pp. 201–210. doi:http://doi.acm.org.ezproxy.lib.ucf.edu/10.1145/1357054.1357089. ISBN 978-1-60558-011-1. 16
- 17. Bibliography McGrenere, J., Ho, W. (2000). Affordances: Clarifying and Evolving a Concept. Procs. of Graphic Interfaces 2000, Montreal, May 2000. McNaughton, J. Utilizing Emerging Multi-touch Table Designs. Technology Enhanced Learning Research Group - Durham University. TR-TEL-10-01. Nielsen, J. (April 1993). "Noncommand User Interfaces". Communications of the ACM (ACM Press) 36 (4): pp. 83–99. doi:10.1145/255950.153582. http://www.useit.com/papers/noncommand.html. Norman, D. (1999). Affordance, Conventions and Design. In ACM Interactions, (May + June, 1999), 38-42. Picard, R. Affective Computing. The MIT Press, Cambridge, Massachusetts. London, England, 1998. Ramduny-Ellis, D.; Dix, A.; Hare, J.; Gill, S. Physicality: Towards a Less-GUI Interface (Preface). Procs. Third International Workshop on Physicality. Cambridge, England, 2009. Sorensen, M. Making a Case for Biological and Tangible Interfaces. Proceedings of the Third International Workshop on Physicality. Cambridge, England, 2009. Sternberg, R. Cognitive Psychology. Wadsworth, Cengage Learning. Belmont, CA, USA, 2009, 2006. Vyas, D., Chisalita, C. Veer, G. Affordance in Interaction. ECCE '06 Proceedings of the 13th Eurpoean conference on Cognitive ergonomics: trust and control in complex socio-technical systems. ACM New York, NY, USA ©2006 ISBN: 978-3-906509-23-5 17
Hinweis der Redaktion
- Welcome to my session, entitled ‘’. My name is Jacques. I’m certain we are about to have an interesting chat about a subject that I love, which is the evolution of digital interfaces and novel forms of interaction. If you’re wondering what accent is this, it’s Brazilian portuguese. I’m a PhD researcher at the Centre for HCI Design. I’m a teacher in Brazil, teaching Usability for web and software on Postgraduation degrees and graphic design at PUC-Rio university. It’ll be a great pleasure to share with you some thought about my research. Shall we start?
- It’s happening for sure. There’s an obvious interest on novel forms of interaction within the industry, games (Wii, Playstation Move, X-Box 360-Kinect), Ubicomp, PCs, Arts.Mark D. Weiser (July 23, 1952 – April 27, 1999) was a chief scientist at Xerox PARC in the United States. Weiser is widely considered to be the father of ubiquitous computing, a term he coined in 1988.While Weiser worked for a variety of computer related startups, his seminal work was in the field of ubiquitous computing while leading the computer science laboratory at PARC, which he joined in 1987. His ideas were significantly influenced by his father's reading of Michael Polanyi's "The Tacit Dimension". He became head of the computer science laboratory in 1988 and chief technology officer in 1996, authoring more than eighty technical publications.
- Igonnaletyou decide what’swrongwiththispicture.
- 1980 EPITOME of Desktop MetaphorAbstract Tasks x RBI (physics, inertia, etc)RBI: Based on physical skills, limbs extension, gaze, notion of depth/perspective, knowledge of the real world (physics, inertia). 3D interactions for instance. More Organic.Jacob (2008): ‘A useful interface will rarely entirely mimic the real world, but will necessarily include some unrealistic or artificial features and commands. In fact, much of the power of using computers comes from this multiplier effect—the ability to go beyond a precise imitation of the real world. For example, in a GUI, one might want to go beyond realistically pointing to and dragging individual files to more abstract commands like Archive all files older than 180 days or Delete all files that contain the text string “reality-based”. (FILTER)
- REALISTIC X NON-REALISTIC / ABSTRACTION X REALISTICAndries "Andy" van Dam (born 8 December 1938, Groningen) is a Dutch-born American professor of computer science and former Vice-President for Research at Brown University in Providence, Rhode Island. Together with Ted Nelson he contributed to the first hypertext system, HES in the late 1960s. He co-authored Computer Graphics: Principles and Practice along with J.D. Foley, S.K. Feiner, and John Hughes. He also co-founded the precursor of today's ACM SIGGRAPH conference.Currently, Professor van Dam is teaching at Brown University. He teaches Introduction to Computer Graphics, as well as one first-year course every fall. He is also serving on the Technical Board of Microsoft Research, as Chairman of the Rhode Island Governor's Science and Technology Advisory Council (STAC), and as Chairman of the IEEE James H. Mulligan, Jr. Education Medal committee. In 1994 he was inducted as a Fellow of the Association for Computing Machinery,[6] and a chaired professorship was recently endowed in his honor at Brown University.
- Very interested on this moment of first interaction with Post-WIMP/NUI without prior learning. How the interface should react? What should display? How to design to maximize efficiency and reduce learning/memory/attention (cognitive issues)?How to signal properly what can be clicked on and what can be done, even before any interaction takes place.That justifies the use of eye tracking/eye gaze.Although technology of interaction is evolving what is being presented on the ‘screen’ is not. If what is being displayed on the screen does not match the possibilities brought by new technologies for interaction, we might not experience the full embodied condition and the easiest and intuitive mode of interaction that could lighten our cognitive burden/learning process. There are more objects of interest than meets the eye.Personal computing has 3-5 small tasks in order to access a larger application to perform a major task. WIMP: Window, Icon, Menu, Pointing deviceWYSIWYG:what you see is what you get. The term is used in computing to describe a system in which content displayed during editing appears very similar to the final output,[2] which might be a printed document, web page, or slide presentation.Direct Manipulation. Shneiderman, 1983HCI: Plans, Procedures, Tasks and GoalsGOMS: Goals, Operators, Methods, and Selection rules (Idealized goals...)HCD: Human Centered Design: The tool dictates the activities.“In the early 1980s Xerox launched Star, the first commercial system with a Graphical User Interface (GUI) and the first to use the “desktop” metaphor to organize a user’s interactions with the computer. Despite the perception of huge progress, from the perspective of design and usage models, there has been precious little progress in the intervening years. In the tradition of Rip van Winkle, a Macintosh user from 1984 who just awoke from a 17-year sleep would have no more trouble operating a “modern” PC than operating a modern car.” (Buxton, 2001)"Rip Van Winkle" is a short story by the American author Washington Irving published in 1819: Rip is told that he has apparently been away from the village for twenty years.“Skill” in these computational modes is like remembering a recipe in order to use an application for a desired outcome. GUI models are performatory by nature, rather than exploratory based upon the senses they use. In addition, GUI has no intended specialization, therefore usability is hampered even more. Sorensen (2009)“Technological systems set up barriers between things – barriers between applications, barriers between files, barriers between activities, barriers between media, barriers between users, and so on. Applications on a PC exist in different worlds; even those that are integrated into “suites” still maintain barriers between different forms of content and different forms of interaction with that content.” Dourish (2005: 197)"The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information" is one of the most highly cited papers in psychology.[1][2][3] It was published in 1956 by the cognitive psychologistGeorge A. Miller of Princeton University's Department of Psychology in Psychological Review. It supposedly argues that the number of objects an average human can hold in working memory is 7 ± 2. This is frequently referred to as Miller's Law (not to be confused with his theory of communication: Miller's Law).Recent research has demonstrated that not only is the "law" based on a misinterpretation of Miller's paper, but that the correct number is probably around three or four.In his article, Miller discussed a coincidence between the limits of one-dimensional absolute judgment and the limits of short-term memory.
- We view the affordances of an artefact as the possibilities (for both: thinking and doing) that are signified by the users during their interaction with the artefact. Acknowledging the work of Baerentsen & Trettvik, we propose an interaction-centered view of affordance, which we call Affordance in Interaction. From this view, affordances of an artefact are not the properties of the artefact but a relationship that is socially and culturally constructed between the users and the artefact in the lived world. This view strongly suggests that affordance emerges during a user’s interaction with the environment. In addition, the affordance in interaction view focuses on the ‘active interpretations’ of the users interacting with the artefact. From this view, users are actively participating in the interaction with the artefact and continuously interpreting the situation and constructing and re-building meanings about the artefact. We suggest that affordances can be better understood as an interpretative relationship between users and the artefact. Vyas et al (2006) SEMIOTICS SIGN: Charles Morris (Sintatic – Semantic – Pragmatic) Peirce (Representamen – Object – Interpretant x Icon – Index - Symbol)
- I made a distinction for the purpose of better clarification on how Perceptible Aff operate in Post-WIMP. As a Bridge between the interface layer (visual, acoustic, haptic) + mode of interaction.The trick is to show what is really possible to be done rather than what is apparently possible.‘In today’s screen design sometimes the cursor shape changes to indicate the desired action (e.g., the change from arrow to handshape in a browser), but this is a convention, not an affordance. After all, the user can still click anywhere, whatever the shape of the cursor. Now if we locked the mouse button when the wrong cursor appeared, that would be a real affordance, although somewhat ponderous. The cursor shape is visual information: it is a learned convention. When you learn not to click unless you have the proper cursorform, you are following a cultural constraint. Norman (1999)PERCEIVED AFFORDANCE x (2) CULTURAL CONSTRAINT = CONVENTION + SYMBOLIC COMMUNICATION (SYMBOLIC MEANING ARBRITARY – LEARNED CONVENTION)EXAMPLE OF PERCEPTIBLE AFFORDANCE: SLIDER/BUTTON
- The creation of novel visual metaphors/reactive interface should be tested in order to verify if their Perceptible Affordances efficiently convey possible interactions within Post-WIMP/NUI.
- Tangible and social computing both reflect this central concern with embodiment. The tangible computing work attempts to capitalize on our physical skills and our familiarity with real world objects. It also tries to make computation manifest to us in the world in the same way as we encounter other phenomena, both as a way of making computation fit more naturally with everyday world and as a way of enriching our experiences with the physical. It attempts to move computation and interaction out of the world of abstract cognitive processes and into the same phenomenal world as our other sorts of interactions. Pg. 102-103.Dourish (2004)So the idea of the invisible interface is too simplistic. It frames interface interaction as an all-or-nothing issue. In arguing against the tyranny of complex interfaces that interfere with the job of getting things done, it misidentifies the problem, demonizes the interface, and abandons altogether the idea that the interface might mediate user action. (…) The notion of the invisible interface correctly identifies the inflexible obtrusiveness of conventional interfaces as problematic. Embodied interaction provides some conceptual tolls for understanding how the interface might move into the background without disappearing altogether. Dourish (2005: 202-203)
- The most important achievement on TUI is bridging the gap between input and output by displaying outputs and inputs on the same surface, helping to integrate perception and action seamlessly into one environment. Sorensen (2009), apudSharlin (2004)MS Surface (2007): Specialized Applications.Activity Centered Design: The tool is the way (Norman, 2005)Addictive rather than transformative quality to the GUI. When a tool is physically acted upon, the result is twofold: causality is instantly observed and time is inherently felt. This coupling between user and tool allows for embodiment. ACD models and TUIs alternatively are more open-ended in application, requiring the user to explore rather than CONFORM or ADAPT to the tool. ‘In other words, as we act through technology that has become ready-to-hand, the technology itself disappears from our immediate concerns. We are caught up in the performance of the work; our mode of being is one of “absorbed coping.” The equipment fades into the background. This unspoken background against which our actions are played out is at the heart of Heidegger’s view of being-in-the-world. Dourish (2004:109)PHYSICAL EXPLORATION + VISUAL EXPLORATION (FAULTY…) = REACTIVE INTERFACEMENTION MEZATOP – MEIRION WILLIAMSMartin Heidegger (September 26, 1889 – May 26, 1976; German pronunciation: [ˈmaɐ̯tiːn ˈhaɪdɛɡɐ]) was an influential German philosopher known for his existential and phenomenological explorations of the "question of Being."[3] His best-known book, Being and Time, is considered to be one of the most important philosophical works of the 20th century [4] and he has been influential beyond philosophy, in literature,[5] psychology,[6] and artificial intelligence.I think, therefore I am – had reflected a doctrine that we “occupy” two different and separate worlds, the world of physical reality and the world of mental existence. This doctrine, called Cartesian dualism, holds that mind and body are quite different; thinking and being are two different sets of phenomena. …So Heidegger rejected the dualism of mind and body altogether. He argued that thinking and being are fundamentally intertwined. …From his perspective, the meaningfulness of everyday experience lies not in the head, but in the world. Pg. 107Dourish (2004)
- Accessibility: the technology conveys opportunities for people with special needs to interface with digital devices (multi-sensory).MENTION GReAT (Gesture Recognition in Aphasia Therapy) SAM MUCROSOFT WORK
- I’ve created this installation at the University of Plymouth, i-DAT Centre. Will include Eye Gaze in the future.Accidental Interactions: help initiate participation, legitimizes participation, invites people in. Evaluation apprehension: effects amplified by the screen.Generalized rather than specific gesture enable expressiveness. Gesture as performance. Live Wall: The installation "Live Wall" is intended to engage a wider public with emerging scientific, philosophical and technological issues in instructive, engaging and memorable ways. New insights into the practical and philosophical meaning of technology are achieved through the participation of the user at the deepest level of cognitive experience and natural behaviours as form of action/input/interaction - generating reaction/output/response.MENTION COLLECTIVE SEEPER: EVAN GRANT
- Let’s revisit the concept of Perceptible Aff x Installations (Digital Art). Here I substituted the Perceptible Aff with SYMBOLIC?SYNESTHESIS?ABSTRACT interaction.- No SEMANTICS or specific language. Purely Reactive.The trick is to show what is really possible to be done rather than what is apparently possible.‘In today’s screen design sometimes the cursor shape changes to indicate the desired action (e.g., the change from arrow to handshape in a browser), but this is a convention, not an affordance. After all, the user can still click anywhere, whatever the shape of the cursor. Now if we locked the mouse button when the wrong cursor appeared, that would be a real affordance, although somewhat ponderous. The cursor shape is visual information: it is a learned convention. When you learn not to click unless you have the proper cursorform, you are following a cultural constraint. Norman (1999)(1) PERCEIVED AFFORDANCE x (2) CULTURAL CONSTRAINT = CONVENTION + SYMBOLIC COMMUNICATION (SYMBOLIC MEANING ARBRITARY – LEARNED CONVENTION)
- The way we perceive things is changing. We need to re-interpret the shift we’re living and review the language itself that would better convey the message of Post-WIMP/NUI and encompass the experience of the very interaction itself.The interface should change to encompass TUI and NUI, rather than just co-exist with addictive features in an already exceeded GUI. Research about the different feedback (multi-sensorial) should take place in order to encompass more effcientlythe possibilitities of interfacing with eyes, gestures, voice, touch, emotions and the very mind itself. GUI additions such as Natural User Interfaces, Microsoft’s Surface Computer, eye-tracking and other Haptic interfaces are not transforming the underlying problems created with the GUI.Sorensen (2009)