Socially intelligent sensing - Hayley Hung - TU Delft - Behavior Design AMS
•
2 gefällt mir•2,971 views
Tenure Track Assistant Professor gave this presentation at the first Behavior Design AMS meetup on Thursday, September 19th in Amsterdam.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie Socially intelligent sensing - Hayley Hung - TU Delft - Behavior Design AMS
Ähnlich wie Socially intelligent sensing - Hayley Hung - TU Delft - Behavior Design AMS (20)
Mehr von Behavior Design AMS
Mehr von Behavior Design AMS (11)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Socially intelligent sensing - Hayley Hung - TU Delft - Behavior Design AMS
- 1. 1 Hayley Hung, TUDelft Socially Intelligent Sensing Hayley Hung Pattern Recognition and Bioinformatics Group Intelligent Systems Department
- 2. 2 Hayley Hung, TUDelft What is social behaviour?
- 3. 3 Hayley Hung, TUDelft What is Human Face-to-face interaction for ? Makes our lives easier: Relationships Trust Co-operation Persuade/influence others Information sharing How could technology help us to understand/interpret/ socially relevant behaviour? How could this help to influence/enhance our experience?
- 4. 4 Hayley Hung, TUDelft Research Mission Statement To develop algorithms that can model and understand non- verbal human social behaviour in real life situations. And through this, to understand how to build systems that can enhance people's quality of life by behaving with socially aware intelligence. Develop algorithms that are perceptive to human social behaviour: Social Signal Processing, Machine Learning Enhancing people's quality of life: Human Machine Interaction, Ambient Intelligent Environments, Design, Architecture. Socially aware intelligence: Social and Behavioural Psychology, Ethnography.
- 5. Hayley Hung, TUDelft What can you say about this picture?
- 6. Hayley Hung, TUDelft What can you say about this picture? Relaxed postureGestures Vocal Behaviour Mutual Gaze Interpersonal Distance
- 7. 7 Hayley Hung, TUDelft Current Research Frontier Person detection Person tracking Gaze detection Body pose estimation Group detection Social and Behavioural Pscychology, Ethnography Activity modelling Action recognition Attraction Estimation Rapport Estimation Role Recognition Personality estimation Dominance Estimation
- 8. 8 Hayley Hung, TUDelft Current Research Frontier Relationship intimacy estimation Conversation quality estimation Person detection Person tracking Gaze detection Body pose estimation Group detection Conversational event estimation Personality estimation Relationship quality estimation Social and Behavioural Pscychology, Ethnography Activity modelling Action recognition New Problem Definitions
- 9. 9 Hayley Hung, TUDelft How to model social behaviour Sensor Data Feature and Cue Extraction Data Annotation Social Behaviour Modelling and Classification Model Performance Evaluation
- 10. 10 Hayley Hung, TUDelft Task 1: Estimating Attraction Source: http://catinbag.blogspot.nl/2010/07/fatal-attraction.html Veenstra and Hung, “Do They Like Me? Using Video Cues to Predict Desires during Speed-dates” in ICCV Workshops 2011
- 11. 11 Hayley Hung, TUDelft Speed Dating, Non-verbal cues and Attraction Can proximity-related video cues be used to automatically predict attraction in speed-dates?
- 12. 12 Hayley Hung, TUDelft Automated Position Extraction - =
- 13. 13 Hayley Hung, TUDelft Automated Position Extraction
- 14. 14 Hayley Hung, TUDelft A Sped up Speed Date:
- 15. 15 Hayley Hung, TUDelft Speed Dating Results Predicting attraction Variance in position is best feature predictor for women (70%). Variance in position of the women and synchrony both perform well (70%) for men. Fusion of all synchrony features Fusion of all movement features
- 16. 16 Hayley Hung, TUDelft Speed Date Experiments : Conclusion The video channel can indeed be a source of valuable information in speed-dates Results differ per gender: Movement synchrony information is more important for males than females. For females, information on the movement of their male counterpart gives good results
- 17. 17 Hayley Hung, TUDelft Task 2: Classifiying Social Actions using a Single Wearable Accelerometer Hung, Englebienne, Kools, “Estimating Social Actions”, Ubicomp 2013
- 18. 18 Hayley Hung, TUDelft Modelling Human Social Behaviour in Dense Crowds How can we model instantaneous social behaviour in extremely large crowds?
- 19. 19 Hayley Hung, TUDelft Our Goal To develop methods to automatically measure socially relevant behaviour and moods in dense crowds using just a single tri-axial accelerometer First step: detect socially relevant actions Speaking; Laughing; Gesturing; Stepping; Drinking
- 20. 20 Hayley Hung, TUDelft Our Goal Use insights from Social Psychology: Speakers move more than listeners (McNeill 2000) Laughter and joking correlated with sudden bursts of motion (Kendon 1990) Synchronised motion during conversation (Kendon 1990,Chartrand and Bargh 1999)
- 21. 21 Hayley Hung, TUDelft Data: The Scenario 32 volunteers (mostly mutual strangers) Experiment Stages: Briefing; Meeting and Mingling; Team formation (groups of 4); Quiz; Award Giving Prizes for top 3 teams 5mx6m recording area
- 22. 22 Hayley Hung, TUDelft Data: The Scenario 32 volunteers (mostly mutual strangers) Experiment Stages: Briefing; Meeting and Mingling; Team formation (groups of 4); Quiz; Award Giving Prizes for top 3 teams 5mx6m recording area
- 23. 23 Hayley Hung, TUDelft Data: The Scenario 32 volunteers (mostly mutual strangers) Experiment Stages: Briefing; Meeting and Mingling; Team formation (groups of 4); Quiz; Award Giving Prizes for top 3 teams 5mx6m recording area Each participant wore a sensing badge
- 24. 24 Hayley Hung, TUDelft Classifying social actions Social Actions: speaking, laughing, gesturing, drinking, or stepping (%) Gesture Step Drink Laugh Speech Precision 59 100 100 100 64 Recall 24 21 21 38 82 F- measure 34 35 35 56 72
- 25. 25 Hayley Hung, TUDelft Social Action Conclusion It is possible to detect socially relevant behaviour. Can we detect when people are in the same conversational group? Could we even detect personality traits quality of people's interaction? Quality of people's relationships? ...etc
- 26. 26 Hayley Hung, TUDelft Looking to the Future... The way we behave socially can exhibit strong detectable patterns, which are robust to noise. How simple can the extracted features be? How could socially aware systems benefit better design?
- 27. 27 Hayley Hung, TUDelft Potential Applications: Human Robot Interaction
- 28. 28 Hayley Hung, TUDelft Applications: Urban Planning Behaviour in Public Spaces How can we measure statistically generaliseable changes as a result of interventions?
- 29. 29 Hayley Hung, TUDelft Potential Applications: Organisational behaviour
- 30. 30 Hayley Hung, TUDelft Acknowledgements/Collaborators Arno Veenstra Gwenn Englebienne Jeroen Kools Ben Krose Maarten van Steen Matt Dobson Claudio Martella Domenic Vossen
- 31. 31 Hayley Hung, TUDelft Workshop at ACM Multmedia (acmmm13.org/) Barcelona , October 22 Human Behaviour Understanding for the Interactions in Arts, Creativity, Entertainment and Edutainment Albert Ali Salah (Bogazici University, Turkey) Oya Aran (Idiap Research Institute, Switzerland) Hatice Gunes (Queen Mary University of London, UK)
- 32. 32 Hayley Hung, TUDelft Data:Sensors Each participant wore: indoor positioning device Proximity and Acceleration sensor 12 participants wore wireless microphone (for annotation) 3 fish eye cameras (for annotation) 3 accelerometer readings failed (software bug)