Towards Real-Time Multimodal Emotion Recognition among Couples
METADATA ONLY
Loading...
Author / Producer
Date
2020-10
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Researchers are interested in understanding the emotions of couples as it relates to relationship quality and dyadic management of chronic diseases. Currently, the process of assessing emotions is manual, time-intensive, and costly. Despite the existence of works on emotion recognition among couples, there exists no ubiquitous system that recognizes the emotions of couples in everyday life while addressing the complexity of dyadic interactions such as turn-taking in couples? conversations. In this work, we seek to develop a smartwatch-based system that leverages multimodal sensor data to recognize each partner's emotions in daily life. We are collecting data from couples in the lab and in the field and we plan to use the data to develop multimodal machine learning models for emotion recognition. Then, we plan to implement the best models in a smartwatch app and evaluate its performance in real-time and everyday life through another field study. Such a system could enable research both in the lab (e.g. couple therapy) or in daily life (assessment of chronic disease management or relationship quality) and enable interventions to improve the emotional well-being, relationship quality, and chronic disease management of couples.
Permanent link
Publication status
published
External links
Editor
Book title
ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction
Journal / series
Volume
Pages / Article No.
748 - 753
Publisher
Association for Computing Machinery
Event
22nd ACM International Conference on Multimodal Interaction (ICMI 2020) (virtual)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Emotion Recognition; Multimodal Fusion; Couples; Smartwatches; Machine Learning; Deep Learning; Transfer Learning
Organisational unit
Notes
Due to the Coronavirus (COVID-19) the conference was conducted virtually
Funding
166348 - Measuring the Impact of Social Support and Common Dyadic Coping on Couple's Dyadic Management of Type II Diabetes by a Novel Ambulatory Assessment Application for the Open Source Behavioral Intervention Platform MobileCoach (SNF)