Literature Review On: ”Speech Emotion Recognition Using Deep Neural Network”
Roman10 mazzei
1. The FACE of Autism Daniele Mazzei University of Pisa IEEE Ro-Man 2010 Viareggio 12th-15th sept 2010
2. Mindblindness People with autism are known to have great difficulties in inferring and analysingmental states, both their own and of others. S. Baron-Cohen has defined mindblindness as the inability to develop an awareness of what is in the mind of another human
3. ASD and Robotics In recent years, robots have been proposed for the treatment and rehabilitation of people with cognitive disorders. Human-like robots which embody emotional states expression, empathy and non-verbal communication have also been proposed for autism therapy
4. Our Hypothesis Therapy in which a robot is able to adapt its behaviour in response to a patient’s postulated mental states can help ASD subjects to improve their social skills. We call this adaptive therapy.
5. FACET platform FACE android Gaze tracking hat Sensorized T-Shirt Face Control Unit (Bodymap and brain) Face tracking and facial expression recognition Inertial platform Room motorized cameras Face configurator and control interface Environmental microphone
6.
7.
8. Sensorized Shirt Based on e-textiles Gathers, computes and transmits HR, HRV, RR, skin conductance, skin temperature and respiratory rate All this signals can be correlated with subject emotional states Made by SmartexSrl, Prato, Italy
9. HATCAM Eye Tracker Wearable and specifically designed to investigate early attention disorders in infants Hat or Band mounted system fast and easy calibration procedure Mirror allows to use images of both eyes (3D gaze point identification) A.Armato, et al. GNB 2010 Patent: TO2008A000862 2008
10.
11. The Bodymap: A sort of black board used to send and receive messages
12. The Roblets: The appendix of the system, as the parts of our nervous system, read data from the environment and convert the Brain signal into an actionEach acquisition or actuation system is an independent roblet
13. The Joker Effect The Joker effect consists in the generation of unsuitable expressions for the therapeutic context and could induce fear or discomfort in the subject. It is a emphatic misalignment typical of sociopaths We need to modulate in intensity the expressions
14. Happy +3 Surprised +2 Sad +1 Neutral 0 Angry -1 Disgusted -2 Fearful -3 Adaptive Control Paradigm +1 0 -1 FACE mood state FACET Signal processing algorithm Subject physiological signals Base line signals Expressions selection and intensity calculation algorithm FACE operator GUI Happiness +3 Surprise +2 Sadness +1 Neutral 0 Anger -1 Disgust -2 Fear -3 0 +1 FACE terapist GUI FACE Android
15. Happy +3 Surprised +2 Sad +1 Neutral 0 Angry -1 Disgusted -2 Fearful -3 Adaptive Control Paradigm +1 0 -1 FACE mood state FACET Signal processing algorithm Subject physiological signals Base line signals Expressions selection and intensity calculation algorithm FACE operator GUI Happiness +3 Surprise +2 Sadness +1 Neutral 0 Anger -1 Disgust -2 Fear -3 0 +1 FACE terapist GUI FACE Android
16. Happy +3 Surprised +2 Sad +1 Neutral 0 Angry -1 Disgusted -2 Fearful -3 Adaptive Control Paradigm +1 0 -1 FACE mood state FACET Signal processing algorithm Subject physiological signals Base line signals Expressions selection and intensity calculation algorithm FACE operator GUI Happiness +3 Surprise +2 Sadness +1 Neutral 0 Anger -1 Disgust -2 Fear -3 0 +1 FACE terapist GUI FACE Android
17. Happy +3 Surprised +2 Sad +1 Neutral 0 Angry -1 Disgusted -2 Fearful -3 Adaptive Control Paradigm +1 0 -1 FACE mood state FACET Signal processing algorithm Subject physiological signals Base line signals Expressions selection and intensity calculation algorithm FACE operator GUI Happiness +3 Surprise +2 Sadness +1 Neutral 0 Anger -1 Disgust -2 Fear -3 0 +1 FACE terapist GUI FACE Android
18. Expressions Creation 3D simulator of FACE allows to study and design facial expressions The final expression intensity is modulated merging the motor position of the selected expression with that of the neutral expression
19. Preliminary results At the moment the therapy/paradigm was tested with 10 ASD subjects and 5 controls No subjects showed discomfort or fear during the therapy Preliminary signal analysis showed a correlation between heart rate variability and intensity of interaction
20. Conclusions and Future Works FACET system could have a wide range of applications in the study of non-verbal communication and expression. On going experiments on control and ASD subjects More tests of the adaptive control algorithm and of the FACET hardware/software infrastructure Using of the FACET platform to perform generic studies on the human-robot empathic link
