Skawina (2013-2016) University of Hamburg – Germany
Modeling affective mechanisms from humans in computer systems is a very difficult task, even with the recent advances of??deep neural networks. This project focus on two different mechanisms of affective computing: perception and intrinsic modeling.?? On the perception side, one of the crucial problems is the subjectivity of emotion description, as different persons can express and perceive emotions differently, depending on several contextual factors. On the intrinsic modeling side, the constraints on emotion representation and its modulation by perception and action must be taken into consideration. This project proposes a neural framework for dealing with emotion perception, modeling, and modulation over different affective mechanisms with the use of deep and self-organizing networks.
pizzicato Data & Code
Gesture Commands for Robot inTeracton (GRIT)
cheap sunglasses lyrics Media
Barros, P., Strahl, E., Wermter, S. The iCub Chronicles – Attention to Emotions! AAAI Conference on Artificial Intelligence, Arizona, USA, 2016
Spontaneous Emotion Recognition
Real-time Gesture Recognition
Publications associated with this project
- Ph.D. Thesis:??Alves de Barros, P. V. (2017). Modeling Affection Mechanisms using Deep and Self-Organizing Neural Networks.
- Elfaramawy, N., Barros, P., Parisi, G. I., & Wermter, S. (2017, October).??Emotion Recognition from Body Expressions with a Neural Network Architecture.??In??Proceedings of the 5th International Conference on Human Agent Interaction??(pp. 143-149). ACM.
- Barros, P., Parisi, G. I., Weber, C., & Wermter, S. (2017).??Emotion-modulated attention improves expression recognition: A deep learning model.??Neurocomputing,??253, 104-114.
- Barros, P., & Wermter, S. (2017, May).??A self-organizing model foraffective memory. In??Neural Networks (IJCNN), 2017 International Joint Conference on??(pp. 31-38). IEEE.
- Barros, P., & Wermter, S. (2016).??Developing crossmodal expression recognition based on a deep neural model.??Adaptive behavior,??24(5), 373-396.
- Hinz, T., Barros, P., & Wermter, S. (2016, September). The Effects of Regularization on Learning Facial Expressions with Convolutional Neural Networks.??In??International Conference on Artificial Neural Networks??(pp. 80-87). Springer International Publishing.
- Barros, P., Weber, C., & Wermter, S. (2016, July).??Learning auditory neural representations for emotion recognition. In??Neural Networks (IJCNN), 2016 International Joint Conference on??(pp. 921-928). IEEE.
- Mousavi, N., Siqueira, H., Barros, P., Fernandes, B., & Wermter, S. (2016, July).??Understanding how deep neural networks learn face expressions. In??Neural Networks (IJCNN), 2016 International Joint Conference on??(pp. 227-234). IEEE.
- Barros, P., Jirak, D., Weber, C., & Wermter, S. (2015).??Multimodal emotional state recognition using sequence-dependent deep hierarchical features.??Neural Networks,??72, 140-151.
- Barros, P., Weber, C., & Wermter, S. (2015, November).??Emotional expression recognition with a cross-channel convolutional neural network for human-robot interaction.??In??Humanoid Robots (Humanoids), 2015 IEEE-RAS 15th International Conference on??(pp. 582-587). IEEE.
- Barros, P., & Wermter, S. (2015, September).??Recognizing complex mental states with deep hierarchical features for Human-Robot Interaction.??In??Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on??(pp. 4065-4070). IEEE.
- Hamester, D., Barros, P., & Wermter, S. (2015, July).??Face expression recognition with a 2-channel convolutional neural network.??In??Neural Networks (IJCNN), 2015 International Joint Conference on??(pp. 1-8). IEEE.
- Barros, P., Parisi, G. I., Jirak, D., & Wermter, S. (2014, November).??Real-time gesture recognition using a humanoid robot with a deep neural architecture.??In??Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on??(pp. 646-651). IEEE.
- Barros, P., Magg, S., Weber, C., & Wermter, S. (2014, September).??A multichannel convolutional neural network for hand posture recognition.??In??International Conference on Artificial Neural Networks??(pp. 403-410). Springer, Cham.