• Regressão múltipla com redes neurais convolucionais para estimativa conjunta da intensidade de Action Units in SIBGRAPI'18

    Abstract: Este trabalho apresenta uma rede neural convolucional (CNN - Convolutional Neural Network) para efetuar a estimativa conjunta da intensidade de Action Units (AUs) em imagens de faces. A estimativa da intensidade de AUs é essencial durante a análise de expressões faciais. Os métodos existentes não levam em consideração a possibilidade de estimativa conjunta para vários AUs em uma face e precisam de um modelo para cada AU; métodos que fazem uso dessa informação precisam reestruturar o problema como aprendizado estruturado com grafos, aumentando a complexidade. Portanto, este trabalho propõe um modelo de regressão múltipla para realizar essa estimativa conjunta e permitir a otimização de um modelo end-to-end. O modelo proposto foi avaliado na base BP4D (Binghamton-Pittsburgh 3D Dynamic Spontaneous Facial Expression Database), utilizada no Facial Expression Recognition and Analysis Challenge (FERA) 2015, que possui anotações da intensidade para cinco AUs em imagens com ambiente controlado. Os resultados obtidos, na média das cinco AUs, superam os baselines propostos e são similares ao estado-da-arte, superando-o em uma das AUs.



  • Face Analysis in the Wild in SIBGRAPI'17

    Abstract: With the global demand for extra security systems, and the growing of human-machine interaction, facial analysis in unconstrained environments (in the wild) became a hot-topic in recent computer vision research. Unconstrained environments include surveillance footage, social media photos and live broadcasts. This type of images and videos include no control over illumination, position, size, occlusion, and facial expressions. Successful facial processing methods for controlled scenarios are unable to pledge with challenging circumstances. Consequently, methods tailored for handling those situations are indispensable for the face analysis research progress. This work presents a comprehensive review of state-of-the-art methods, drawing attention to the complications derived from in the wild scenariosa nd the behavior differences when applied to the controlled images. The main topics to be covered are: (1) face detection; (2) facial image quality; (3) head pose estimation; (4) face alignment; (5) 3D face reconstruction; (6) gender and age estimation; (7) facial expressions and emotions; and (8) face recognition. Finally, available code and applications for in the wild face analysis are presented, followed by a discussion on future directions.

  • AUMPNet: Simultaneous Action Units Detection and Intensity Estimation on Multipose Facial Images Using a Single Convolutional Neural Network in FG'17

    Abstract: This paper presents an unified convolutional neural network (CNN), named AUMPNet, to perform both Action Units (AUs) detection and intensity estimation on facial images with multiple poses. Although there are a variety of methods in the literature designed for facial expression analysis, only few of them can handle head pose variations. Therefore, it is essential to develop new models to work on non-frontal face images, for instance, those obtained from unconstrained environments. In order to cope with problems raised by pose variations, an unique CNN, based on region and multitask learning, is proposed for both AU detection and intensity estimation tasks. Also, the available head pose information was added to the multitask loss as a constraint to the network optimization, pushing the network towards learning better representations. As opposed to current approaches that require ad hoc models for every single AU in each task, the proposed network simultaneously learns AU occurrence and intensity levels for all AUs. The AUMPNet was evaluated on an extended version of the BP4D-Spontaneous database, which was synthesized into nine different head poses and made available to FG 2017 Facial Expression Recognition and Analysis Challenge (FERA 2017) participants. The achieved results surpass the FERA 2017 baseline, using the challenge metrics, for AU detection by 0.054 in F1-score and 0.182 in ICC(3, 1) for intensity estimation.



  • Landmark-free smile intensity estimation in SIBGRAPI'16

    Abstract: Facial expression analysis is an important field of research, mostly because of the rich information faces can provide. The majority of works published in the literature have focused on facial expression recognition and so far estimating facial expression intensities have not gathered same attention. The analysis of these intensities could improve face processing applications on distinct areas, such as computer assisted health care, human-computer interaction and biometrics. Because the smile is the most common expression, studying its intensity is a first step towards estimating other expressions intensities. Most related works are based on facial landmarks, sometimes combined with appearance features around these points, to estimate smile intensities. Relying on landmarks can lead to wrong estimations due to errors in the registration step. In this work we investigate a landmark-free approach for smile intensity estimation using appearance features from a grid division of the face. We tested our approach on two different databases, one with spontaneous expressions (BP4D) and the other with posed expressions (BU-3DFE); results are compared to state-of-the-art works in the field. Our method shows competitive results even using only appearance features on spontaneous facial expression intensities, but we found that there is still need for further investigation on posed expressions.