HOME Board


Hit 797
Subject [MMM 2020] Facial Εxpression Sentence Generation (by Joanna) is published in MMM 2020
Date 2020-01-16
Face Tells Detailed Εxpression: Generating Comprehensive Facial Εxpression Sentence through Facial Action Units
Authors: Joanna Hong, Hong Joo Lee, Yelin Kim, Yong Man Ro
Human facial expression plays the key role in the understanding of the social behavior. Many deep learning approaches present facial emotion recognition and automatic image captioning considering human sentiments. However, most current deep learning models for facial expression analysis do not contain comprehensive, detailed information of a single face. In this paper, we newly introduce a text-based facial expression description using several essential components describing comprehensive facial expression: gender, facial action units, and corresponding intensities. Then, we propose comprehensive facial expression sentence generating model along with facial expression recognition model for a single facial image to verify the effectiveness of our text-based dataset. Experimental results show that the proposed two models are supporting each other improving their performances: the text-based facial expression description provides comprehensive semantic information to the facial emotion recognition model. Also, the visual information from the emotion recognition model guides the facial expression sentence generation to produce a proper sentence describing comprehensive description.