Facial expression recognition plays a significant part in artificial intelligence and computer vision. However, most of facial expression recognition methods have not obtained satisfactory results based on low-level features. The existed methods used in facial expression recognition encountered the major issues of linear inseparability, large computational burden, and data redundancy. To obtain satisfactory results, we propose an innovative deep learning model using the kernel entropy component analysis network and directed acyclic graph support vector machine. (...) We use the KECANet in the feature extraction stage. In the stage of output, binary hashing and blockwise histograms are adopted. We sent the final output features to the DAGSVM classifier for expression recognition. We test the performance of our proposed method on three databases of CK+, JAFFE, and CMU Multi-PIE. According to the experiment results, the proposed method can learn high-level features and provide more recognition information in the stage of training, obtaining a higher recognition rate. (shrink)

Emotional facial expressions can inform researchers about an individual's emotional state. Recent technological advances open up new avenues to automatic Facial Expression Recognition. Based on machine learning, such technology can tremendously increase the amount of processed data. FER is now easily accessible and has been validated for the classification of standardized prototypical facial expressions. However, applicability to more naturalistic facial expressions still remains uncertain. Hence, we test and compare performance of three different FER systems with human emotion (...) class='Hi'>recognition for standardized posed facial expressions and for non-standardized acted facial expressions. For the standardized images, all three systems classify basic emotions accurately and they are mostly on par with human raters. For the non-standardized stimuli, performance drops remarkably for all three systems, but Azure still performs similarly to humans. In addition, all systems and humans alike tend to misclassify some of the non-standardized emotional facial expressions as neutral. In sum, emotion recognition by automated facial expression recognition can be an attractive alternative to human emotion recognition for standardized and non-standardized emotional facial expressions. However, we also found limitations in accuracy for specific facial expressions; clearly there is need for thorough empirical evaluation to guide future developments in computer vision of emotional facial expressions. (shrink)

Previous meta-analyses support a female advantage in decoding non-verbal emotion (Hall, 1978, 1984), yet the mechanisms underlying this advantage are not understood. The present study examined whether the female advantage is related to greater female attention to the eyes. Eye-tracking techniques were used to measure attention to the eyes in 19 males and 20 females during a facial expression recognition task. Women were faster and more accurate in their expression recognition compared with men, and women looked (...) more at the eyes than men. Positive relationships were observed between dwell time and number of fixations to the eyes and both accuracy of facial expression recognition and speed of facial expression recognition. These results support the hypothesis that the female advantage in facial expression recognition is related to greater female attention to the eyes. (shrink)

This study was designed to test three competing hypotheses (impaired configural processing; impaired Theory of Mind; atypical amygdala functioning) to explain the basic facial expression recognition profile of adults with autism spectrum disorders (ASD). In Experiment 1 the Ekman and Friesen (1976) series were presented upright and inverted. Individuals with ASD were significantly less accurate than controls at recognising upright facial expressions of fear, sadness and disgust and their pattern of errors suggested some configural processing difficulties. Impaired (...) class='Hi'>recognition of inverted facial expressions suggested some additional difficulties processing the facial features. Unexpectedly, the clinical group misidentified fear as anger. In Experiment 2 feature processing of facial expressions was investigated by presenting stimuli in a piecemeal fashion, starting with either just the eyes or the mouth. Individuals with ASD were impaired at recognising fear from the eyes and disgust from the mouth; they also confused fearful eyes as being angry. The findings are discussed in terms of the three competing hypotheses tested. (shrink)

State-of-the-art facial expression methods outperform human beings, especially, thanks to the success of convolutional neural networks. However, most of the existing works focus mainly on analyzing an adult’s face and ignore the important problems: how can we recognize facial expression from a baby’s face image and how difficult is it? In this paper, we first introduce a new face image database, named BabyExp, which contains 12,000 images from babies younger than two years old, and each image is with (...) one of three facial expressions. To the best of our knowledge, the proposed dataset is the first baby face dataset for analyzing a baby’s face image, which is complementary to the existing adult face datasets and can shed some light on exploring baby face analysis. We also propose a feature guided CNN method with a new loss function, called distance loss, to optimize interclass distance. In order to facilitate further research, we provide the benchmark of expression recognition on the BabyExp dataset. Experimental results show that the proposed network achieves the recognition accuracy of 87.90% on BabyExp. (shrink)

Micro-expression is an extremely quick and uncontrollable facial movement that lasts for 40–200 ms and reveals thoughts and feelings that an individual attempts to cover up. Though much more difficult to detect and recognize, ME recognition is similar to macro-expression recognition in that it is influenced by facial features. Previous studies suggested that facial attractiveness could influence facial expression recognition processing. However, it remains unclear whether facial attractiveness could also influence ME recognition. Addressing (...) this issue, this study tested 38 participants with two ME recognition tasks in a static condition or dynamically. Three different MEs at two attractiveness levels. The results showed that participants recognized MEs on attractive faces much quicker than on unattractive ones, and there was a significant interaction between ME and facial attractiveness. Furthermore, attractive happy faces were recognized faster in both the static and the dynamic conditions, highlighting the happiness superiority effect. Therefore, our results provided the first evidence that facial attractiveness could influence ME recognition in a static condition or dynamically. (shrink)

This article discusses how research on the image-based analysis of facial images has informed this debate by demonstrating that a single representational system for facial identity and facial expression is not only computationally viable, but can simulate existing cognitive data demonstrating apparent dissociable processing of these two facial properties. It discusses the increasing number of cognitive studies that provide support for this view. Neuropsychological case studies of brain-injured patients and provide limited evidence for separate visual routes processing facial identity (...) and facial expression. It deals with neuroimaging studies addressing both facial identity and facial expression processing which shows that the fusiform gyrus contributes to the recognition of both facial properties. Research continues to reinforce the proposal that a common visual framework supports the representation of at least some aspects of facial identity and facial expression. (shrink)

There are many ways to assess face perception skills. In this study, we describe a novel task battery FEAST (Facial Expression Action Stimulus Test) developed to test recognition of identity and expressions of human faces as well as stimulus control categories. The FEAST consists of a neutral and emotional face memory task, a face and object identity matching task, a face and house part-to-whole matching task, and a human and animal facial expression matching task. The identity and (...) part-to-whole matching tasks contain both upright and inverted conditions. The results provide reference data of a healthy sample of controls in two age groups for future users of the FEAST. (shrink)

The results of a previous study (Craig & Lipp, 2018) into the effects of multiple social category cues (face race and face sex) on facial emotion recognition indicate that face sex dominates face race, and moreover, participant sex differences contribute little to the observed effects. Here, I modelled the same dataset (https://osf.io/rsmxb/) using the ex-Gaussian, a distribution that is 1) well suited to RT data and 2) separates slow from relatively fast influences. Corroborating recent results (Tipples, 2022) current results (...) show larger effects of face sex (for the faces of White individuals) for female participants. Further novel interaction effects were revealed. For example, results support a different time course for the influence of face sex on expression for the faces of Black compared to White individuals. (shrink)

This article presents the analysis of the main Spanish political candidates for the elections to be held on April 2019. The analysis focuses on the Facial Expression Analysis, a technique widely used in neuromarketing research. It allows to identify the micro-expressions that are very brief, involuntary. They are signals of hidden emotions that cannot be controlled voluntarily. The video with the final interventions of every candidate has been post-processed using the classification algorithms given by the iMotions's AFFDEX platform. We (...) have then analyzed these data. Firstly, we have identified and compare the basic emotions showed by each politician. Second, we have associated the basic emotions with specific moments of the candidate's speech, identifying the topics they address and relating them directly to the expressed emotion. Third, we have analyzed whether the differences shown by each candidate in every emotion are statistically significant. In this sense, we have applied the non-parametric chi-squared goodness-of-fit test. We have also considered the ANOVA analysis in order to test whether, on average, there are differences between the candidates. Finally, we have checked if there is consistency between the results provided by different surveys from the main media in Spain regarding the evaluation of the debate and those obtained in our empirical analysis. A predominance of negative emotions has been observed. Some inconsistencies were found between the emotion expressed in the facial expression and the verbal content of the message. Also, evidences got from statistical analysis confirm that the differences observed between the various candidates with respect to the basic emotions, on average, are statistically significant. In this sense, this article provides a methodological contribution to the analysis of the public figures' communication, which could help politicians to improve the effectiveness of their messages identifying and evaluating the intensity of the expressed emotions. (shrink)

Deficits in recognition of facial expressions of emotions are considered to be an important factor explaining impairments in social functioning and affective reactions of schizophrenic patients. Many studies confirmed such deficits while controversies remained concerning the emotion valence and modality. The aim of the study was to explore the process of recognizing facial expressions of emotion in the group of schizophrenic patients by analyzing the role of emotion valence, modality and gender of the model. Results of the group of (...) 35 patients and 35 matched controls indicate that while schizophrenic patients show general impairment in recognizing facial expressions of both positive and the majority of negative emotions, there are differences in deficits for particular emotions. Expressions also appeared to be more ambiguous for the patients while variables connected with gender were found less significant. (shrink)

The current study investigated how sad expressions during encoding affected recognition of facial identity in visual working memory and its electrophysiological correlates. Event-related poten...

Micro-expressions, as fleeting facial expressions, are very important for judging people’s true emotions, thus can provide an essential behavioral clue for lie and dangerous demeanor detection. From embodied accounts of cognition, we derived a novel hypothesis that facial feedback from upper and lower facial regions has differential effects on micro-expression recognition. This hypothesis was tested and supported across three studies. Specifically, the results of Study 1 showed that people became better judges of intense micro-expressions with a duration of (...) 450 ms when the facial feedback from upper face was enhanced via a restricting gel. Additional results of Study 2 showed that the recognition accuracy of subtle micro-expressions was significantly impaired under all duration conditions (50 ms, 150 ms, 333 ms, and 450 ms) when facial feedback from lower face was enhanced. In addition, the results of Study 3 also revealed that blocking the facial feedback of lower face, significantly boosted the recognition accuracy of subtle and intense micro-expressions under all duration conditions (150 ms and 450 ms). Together, these results highlight the role of facial feedback in judging the subtle movements of micro-expressions. (shrink)

Emotions are multimodal processes that play a crucial role in our everyday lives. Recognizing emotions is becoming more critical in a wide range of application domains such as healthcare, education, human-computer interaction, Virtual Reality, intelligent agents, entertainment, and more. Facial macro-expressions or intense facial expressions are the most common modalities in recognizing emotional states. However, since facial expressions can be voluntarily controlled, they may not accurately represent emotional states. Earlier studies have shown that facial micro-expressions are more reliable than facial (...) macro-expressions for revealing emotions. They are subtle, involuntary movements responding to external stimuli that cannot be controlled. This paper proposes using facial micro-expressions combined with brain and physiological signals to more reliably detect underlying emotions. We describe our models for measuring arousal and valence levels from a combination of facial micro-expressions, Electroencephalography (EEG) signals, galvanic skin responses (GSR), and Photoplethysmography (PPG) signals. We then evaluate our model using the DEAP dataset and our own dataset based on a subject-independent approach. Lastly, we discuss our results, the limitations of our work, and how these limitations could be overcome. We also discuss future directions for using facial micro-expressions and physiological signals in emotion recognition. (shrink)

There are complex posture changes in dance movements, which lead to the low accuracy of dance movement recognition. And none of the current motion recognition uses the dancer’s attributes. The attribute feature of dancer is the important high-level semantic information in the action recognition. Therefore, a dance movement recognition algorithm based on feature expression and attribute mining is designed to learn the complicated and changeable dancer movements. Firstly, the original image information is compressed by the (...) time-domain fusion module, and the information of action and attitude can be expressed completely. Then, a two-way feature extraction network is designed, which extracts the details of the actions along the way and takes the sequence image as the input of the network. Then, in order to enhance the expression ability of attribute features, a multibranch spatial channel attention integration module based on an attention mechanism is designed to extract the features of each attribute. Finally, using the semantic inference and information transfer function of the graph convolution network, the relationship between attribute features and dancer features can be mined and deduced, and more expressive action features can be obtained; thus, high-performance dance motion recognition is realized. The test and analysis results on the data set show that the algorithm can recognize the dance movement and improve the accuracy of the dance movement recognition effectively, thus realizing the movement correction function of the dancer. (shrink)

Most research on cross-cultural emotion recognition has focused on facial expressions. To integrate the body of evidence on vocal expression, we present a meta-analysis of 37 cross-cultural studies...

We used multiple methods to examine two questions about emotion and culture: (1) Which facial expressions are recognised cross-culturally; and (2) does the “forced-choice” method lead to spurious findings of universality? Forty participants in the US and 40 in India were shown 14 facial expressions and asked to say what had happened to cause the person to make the face. Analyses of the social situations given and of the affect words spontaneously used showed high levels of recognition for most (...) of the expressions. A subsequent forced-choice task using the same faces confirmed these findings. Analysis of the pattern of magnitude, discreteness, and similarity of responses across cultures and expressions led to the conclusion that there is no neat distinction between cross-culturally recognisable and nonrecognisable expressions. Results are better described as a gradient of recognition. (shrink)

ABSTRACTRecognising identity and emotion conveyed by the face is important for successful social interactions and has thus been the focus of considerable research. Debate has surrounded the extent...

Research on human motion perception shows that people are highly adept at inferring emotional states from body movements. Yet, this process is mediated by a number of individual factors and experie...

Background: This study developed a photo and video database of 4-to-6-year-olds expressing the seven induced and posed universal emotions and a neutral expression. Children participated in photo and video sessions designed to elicit the emotions, and the resulting images were further assessed by independent judges in two rounds. Methods: In the first round, two independent judges, experts in the Facial Action Coding System, firstly analysed 3,668 emotions facial expressions stimuli from 132 children. Both judges reached 100% agreement regarding 1,985 (...) stimuli, which were then selected for a second round of analysis between judges 3 and 4. Results: The result was 1,985 stimuli were produced from 124 participants. A Kappa index of 0.70 and an accuracy of 73% between experts were observed. Lower accuracy was found for emotional expression by 4-year-olds than 6-year-olds. Happiness, disgust and contempt had the highest agreement. After a sub-analysis evaluation of all four judges, 100% agreement was reached for 1,381 stimuli which compound the ChildEFES database with 124 participants and 51% induced photographs. The number of stimuli of each emotion were: 87 for neutrality, 363 for happiness, 170 for disgust, 104 for surprise, 152 for fear, 144 for sadness, 157 for anger 157, and 183 for contempt. Conclusions: The findings show that this photo and video database can facilitate research on the mechanisms involved in early childhood recognition of facial emotions in children, contributing to the understanding of facial emotion recognition deficits which characterise several neurodevelopmental and psychiatric disorders. (shrink)

We used multiple methods to examine two questions about emotion and culture: (1) Which facial expressions are recognised cross-culturally; and (2) does the “forced-choice” method lead to spurious findings of universality? Forty participants in the US and 40 in India were shown 14 facial expressions and asked to say what had happened to cause the person to make the face. Analyses of the social situations given and of the affect words spontaneously used showed high levels of recognition for most (...) of the expressions. A subsequent forced-choice task using the same faces confirmed these findings. Analysis of the pattern of magnitude, discreteness, and similarity of responses across cultures and expressions led to the conclusion that there is no neat distinction between cross-culturally recognisable and nonrecognisable expressions. Results are better described as a gradient of recognition. (shrink)

ABSTRACTLonely individuals show increased social monitoring and heightened recognition of negative facial expressions. The current study investigated whether this pattern extends to other nonverbal...

The topic of recognition has come to occupy a central place in debates in social and political theory. Developed by George Herbert Mead and Charles Taylor, it has been given expression in the program for Critical Theory developed by Axel Honneth in his book The Struggle for Recognition. Honneth's research program offers an empirically insightful way of reflecting on emancipatory struggles for greater justice and a powerful theoretical tool for generating a conception of justice and the good (...) that enables the normative evaluation of such struggles. This 2007 volume offers a critical clarification and evaluation of this research program, particularly its relationship to the other major development in critical social and political theory; namely, the focus on power as formative of practical identities proposed by Michel Foucault and developed by theorists such as Judith Butler, James Tully, and Iris Marion Young. (shrink)

The ideas of expression and expressiveness have been central to Stanley Cavell’s writing from the beginning, joining themes from his more strictly philosophical writing to the role of human expression as projected in cinema. This paper explores a thread running through several different parts of his writing, relating claims he makes about the photographic medium of film and its implications for the question of expression and expressivity in film There is an invocation of notions of necessity and (...) control in the context of cinema that should be understood in the context of related ideas in his writings on Wittgenstein and others. The paper pursues some thoughts about the power of the camera, the themes of activity and passivity in expression, and the human face as the privileged field of such activity and passivity. (shrink)

ABSTRACTThe recognition of emotional facial expressions is often subject to contextual influence, particularly when the face and the context convey similar emotions. We investigated whether spontaneous, incidental affective theory of mind inferences made while reading vignettes describing social situations would produce context effects on the identification of same-valenced emotions as well as differently-valenced emotions conveyed by subsequently presented faces. Crucially, we found an effect of context on reaction times in both experiments while, in line with previous work, we found (...) evidence for a context effect on accuracy only in Experiment 1. This demonstrates that affective theory of mind inferences made at the pragmatic level of a text can automatically, contextually influence the perceptual processing of emotional facial expressions in a separate task even when those emotions are of a distinctive valence. Thus, our novel findings suggest that language acts as a contex... (shrink)

This article analyzes different pathologies of social affirmation and examines the grounds of social recognition from the point of view of the concept of expression. The red thread of the text is provided by Tove Jansson’s fictional works, and the focus will be on three cases in particular. The article sets out from the phenomenological distinction between the sensible expression, on the one hand, and the expressed content, on the other. By focusing on the three cases, the (...) article distinguishes and analyses the fundamental structures of communal life and explicates different ways in which social affirmation can be one-sided or distorted. (shrink)

The ideas of expression and expressiveness have been central to Stanley Cavell’s writing from the beginning, joining themes from his more strictly philosophical writing to the role of human expression as projected in cinema. This paper explores a thread running through several different parts of his writing, relating claims he makes about the photographic medium of film and its implications for the question of expression and expressivity in film There is an invocation of notions of necessity and (...) control in the context of cinema that should be understood in the context of related ideas in his writings on Wittgenstein and others. The paper pursues some thoughts about the power of the camera, the themes of activity and passivity in expression, and the human face as the privileged field of such activity and passivity. (shrink)