Abstract: We as humans use words with accompanying visual and facial cues to communicate effectively. Classifying facial emotion using computer vision methodologies has been an active research area in the computer vision field. In this paper, we propose a simple method for facial expression recognition that enhances accuracy. We tested our method on the FER-2013 dataset that contains static images. Instead of using Histogram equalization to preprocess the dataset, we used Unsharp Mask to emphasize texture and details and sharpened the edges. We also used ImageDataGenerator from Keras library for data augmentation. Then we used Convolutional Neural Networks (CNN) model to classify the images into 7 different facial expressions, yielding an accuracy of 69.46% on the test set. Our results show that using image preprocessing such as the sharpening technique for a CNN model can improve the performance, even when the CNN model is relatively simple.
Abstract: Recently, detecting liars and extracting features which distinguish them from truth-tellers have been the focus of a wide range of disciplines. To the author’s best knowledge, most of the work has been done on facial expressions and body gestures but only few works have been done on the language used by both liars and truth-tellers. This paper sheds light on four axes. The first axis copes with building an audio corpus for deceptive and truthful speech for Egyptian Arabic speakers. The second axis focuses on examining the human perception of lies and proving our need for computational linguistic-based methods to extract features which characterize truthful and deceptive speech. The third axis is concerned with building a linguistic analysis program that could extract from the corpus the inter- and intra-linguistic cues for deceptive and truthful speech. The program built here is based on selected categories from the Linguistic Inquiry and Word Count program. Our results demonstrated that Egyptian Arabic speakers on one hand preferred to use first-person pronouns and present tense compared to the past tense when lying and their lies lacked of second-person pronouns, and on the other hand, when telling the truth, they preferred to use the verbs related to motion and the nouns related to time. The results also showed that there is a need for bigger data to prove the significance of words related to emotions and numbers.
Abstract: The field of automatic facial expression analysis has been an active research area in the last two decades. Its vast applicability in various domains has drawn so much attention into developing techniques and dataset that mirror real life scenarios. Many techniques such as Local Binary Patterns and its variants (CLBP, LBP-TOP) and lately, deep learning techniques, have been used for facial expression recognition. However, the problem of occlusion has not been sufficiently handled, making their results not applicable in real life situations. This paper develops a simple, yet highly efficient method tagged Local Binary Pattern-Histogram of Gradient (LBP-HOG) with occlusion detection in face image, using a multi-class SVM for Action Unit and in turn expression recognition. Our method was evaluated on three publicly available datasets which are JAFFE, CK, SFEW. Experimental results showed that our approach performed considerably well when compared with state-of-the-art algorithms and gave insight to occlusion detection as a key step to handling expression in wild.
Abstract: The number of school-aged children with autism in Indonesia has been increasing each year. Autism is a developmental disorder which can be diagnosed in childhood. One of the symptoms is the lack of communication skills. Music therapy is known as an effective treatment for children with autism. Music elements and structures create a good space for children with autism to express their feelings and communicate their thoughts. School-aged children are expected to be able to communicate non-verbally very well, but children with autism experience the difficulties of communicating non-verbally. The aim of this research is to analyze the significance of music therapy treatment to improve non-verbal communication tools for children with autism. This research informs teachers and parents on how music can be used as a media to communicate with children with autism. The qualitative method is used to analyze this research, while the result is described with the microanalysis technique. The result is measured specifically from the whole experiment, hours of every week, minutes of every session, and second of every moment. The samples taken are four school-aged children with autism in the age range of six to 11 years old. This research is conducted within four months started with observation, interview, literature research, and direct experiment. The result demonstrates that music therapy could be effectively used as a non-verbal communication tool for children with autism, such as changes of body gesture, eye contact, and facial expression.
Abstract: People express emotions through different modalities.
Integration of verbal and non-verbal communication channels creates
a system in which the message is easier to understand. Expanding
the focus to several expression forms can facilitate research on
emotion recognition as well as human-machine interaction. In this
article, the authors present a Polish emotional database composed of
three modalities: facial expressions, body movement and gestures,
and speech. The corpora contains recordings registered in studio
conditions, acted out by 16 professional actors (8 male and 8 female).
The data is labeled with six basic emotions categories, according to
Ekman’s emotion categories. To check the quality of performance,
all recordings are evaluated by experts and volunteers. The database
is available to academic community and might be useful in the study
on audio-visual emotion recognition.
Abstract: This paper has critically examined the use of Machine Learning procedures in curbing unauthorized access into valuable areas of an organization. The use of passwords, pin codes, user’s identification in recent times has been partially successful in curbing crimes involving identities, hence the need for the design of a system which incorporates biometric characteristics such as DNA and pattern recognition of variations in facial expressions. The facial model used is the OpenCV library which is based on the use of certain physiological features, the Raspberry Pi 3 module is used to compile the OpenCV library, which extracts and stores the detected faces into the datasets directory through the use of camera. The model is trained with 50 epoch run in the database and recognized by the Local Binary Pattern Histogram (LBPH) recognizer contained in the OpenCV. The training algorithm used by the neural network is back propagation coded using python algorithmic language with 200 epoch runs to identify specific resemblance in the exclusive OR (XOR) output neurons. The research however confirmed that physiological parameters are better effective measures to curb crimes relating to identities.
Abstract: One of the main aims of current social robotic research
is to improve the robots’ abilities to interact with humans. In order
to achieve an interaction similar to that among humans, robots
should be able to communicate in an intuitive and natural way
and appropriately interpret human affects during social interactions.
Similarly to how humans are able to recognize emotions in other
humans, machines are capable of extracting information from the
various ways humans convey emotions—including facial expression,
speech, gesture or text—and using this information for improved
human computer interaction. This can be described as Affective
Computing, an interdisciplinary field that expands into otherwise
unrelated fields like psychology and cognitive science and involves
the research and development of systems that can recognize and
interpret human affects. To leverage these emotional capabilities
by embedding them in humanoid robots is the foundation of
the concept Affective Robots, which has the objective of making
robots capable of sensing the user’s current mood and personality
traits and adapt their behavior in the most appropriate manner
based on that. In this paper, the emotion recognition capabilities
of the humanoid robot Pepper are experimentally explored, based
on the facial expressions for the so-called basic emotions, as
well as how it performs in contrast to other state-of-the-art
approaches with both expression databases compiled in academic
environments and real subjects showing posed expressions as well
as spontaneous emotional reactions. The experiments’ results show
that the detection accuracy amongst the evaluated approaches differs
substantially. The introduced experiments offer a general structure
and approach for conducting such experimental evaluations. The
paper further suggests that the most meaningful results are obtained
by conducting experiments with real subjects expressing the emotions
as spontaneous reactions.
Abstract: The problem of emotion recognition is a challenging problem. It is still an open problem from the aspect of both intelligent systems and psychology. In this paper, both voice features and facial features are used for building an emotion recognition system. A Support Vector Machine classifiers are built by using raw data from video recordings. In this paper, the results obtained for the emotion recognition are given, and a discussion about the validity and the expressiveness of different emotions is presented. A comparison between the classifiers build from facial data only, voice data only and from the combination of both data is made here. The need for a better combination of the information from facial expression and voice data is argued.
Abstract: This paper presents a facial expression recognition system. It performs identification and classification of the seven basic expressions; happy, surprise, fear, disgust, sadness, anger, and neutral states. It consists of three main parts. The first one is the detection of a face and the corresponding facial features to extract the most expressive portion of the face, followed by a normalization of the region of interest. Then calculus of curvelet coefficients is performed with dimensionality reduction through principal component analysis. The resulting coefficients are combined with two ratios; mouth ratio and face edge ratio to constitute the whole feature vector. The third step is the classification of the emotional state using the SVM method in the feature space.
Abstract: Face and facial expressions play essential roles in
interpersonal communication. Most of the current works on the facial
expression recognition attempt to recognize a small set of the
prototypic expressions such as happy, surprise, anger, sad, disgust
and fear. However the most of the human emotions are
communicated by changes in one or two of discrete features. In this
paper, we develop a facial expressions synthesis system, based on the
facial characteristic points (FCP's) tracking in the frontal image
sequences. Selected FCP's are automatically tracked using a crosscorrelation
based optical flow. The proposed synthesis system uses a
simple deformable facial features model with a few set of control
points that can be tracked in original facial image sequences.
Abstract: Facial expression analysis plays a significant role for
human computer interaction. Automatic analysis of human facial
expression is still a challenging problem with many applications. In
this paper, we propose neuro-fuzzy based automatic facial expression
recognition system to recognize the human facial expressions like
happy, fear, sad, angry, disgust and surprise. Initially facial image is
segmented into three regions from which the uniform Local Binary
Pattern (LBP) texture features distributions are extracted and
represented as a histogram descriptor. The facial expressions are
recognized using Multiple Adaptive Neuro Fuzzy Inference System
(MANFIS). The proposed system designed and tested with JAFFE
face database. The proposed model reports 94.29% of classification
accuracy.
Abstract: The genre of fantasy depicts a world of imagine that triggers popular interest from a created view of world, and a fantasy is defined as a story that illustrates a world of imagine where scientific or horror elements are stand in its center. This study is not focused on the narrative of the fantasy, i.e. not on the adventurous story, but is concentrated on the image of the fantasy to work on its relationship with intended themes and differences among cultures due to meanings of materials. As for films, we have selected some films in the 2000's that are internationally recognized as expressing unique images of fantasy containing the theme of love in them. The selected films are 5 pieces including two European films, Amelie from Montmartre (2001) and The Science of Sleep (2005) and three Asian films, Citizen Dog from Thailand (2004), Memories of Matsuko from Japan (2006), and I'm a Cyborg, but That's OK from Korea (2006). These films share some common characteristics to the effect that they give tiny lessons and feelings for life with expressions of fantasy images as if they were fairy tales for adults and that they lead the audience to reflect on their days and revive forgotten dreams of childhood. We analyze the images of fantasy in each of the films on the basis of the elements of Mise-en-Scène (setting and props, costume, hair and make-up, facial expressions and body language, lighting and color, positioning of characters, and objects within a frame).
Abstract: A biophysically based multilayer continuum model of the facial soft tissue composite has been developed for simulating wrinkle formation. The deformed state of the soft tissue block was determined by solving large deformation mechanics equations using the Galerkin finite element method. The proposed soft tissue model is composed of four layers with distinct mechanical properties. These include stratum corneum, epidermal-dermal layer (living epidermis and dermis), subcutaneous tissue and the underlying muscle. All the layers were treated as non-linear, isotropic Mooney Rivlin materials. Contraction of muscle fibres was approximated using a steady-state relationship between the fibre extension ratio, intracellular calcium concentration and active stress in the fibre direction. Several variations of the model parameters (stiffness and thickness of epidermal-dermal layer, thickness of subcutaneous tissue layer) have been considered.
Abstract: Face authentication for access control is a face
membership authentication which passes the person of the incoming
face if he turns out to be one of an enrolled person based on face
recognition or rejects if not. Face membership authentication belongs
to the two class classification problem where SVM(Support Vector
Machine) has been successfully applied and shows better performance
compared to the conventional threshold-based classification. However,
most of previous SVMs have been trained using image feature vectors
extracted from face images of each class member(enrolled
class/unenrolled class) so that they are not robust to variations in
illuminations, poses, and facial expressions and much affected by
changes in member configuration of the enrolled class
In this paper, we propose an effective face membership
authentication method based on SVM using class discriminating
features which represent an incoming face image-s associability with
each class distinctively. These class discriminating features are weakly
related with image features so that they are less affected by variations
in illuminations, poses and facial expression.
Through experiments, it is shown that the proposed face
membership authentication method performs better than the threshold
rule-based or the conventional SVM-based authentication methods and
is relatively less affected by changes in member size and membership.
Abstract: Facial recognition and expression analysis is rapidly
becoming an area of intense interest in computer science and humancomputer
interaction design communities. The most expressive way
humans display emotions is through facial expressions. In this paper
skin and non-skin pixels were separated. Face regions were extracted
from the detected skin regions. Facial expressions are analyzed from
facial images by applying Gabor wavelet transform (GWT) and
Discrete Cosine Transform (DCT) on face images. Radial Basis
Function (RBF) Network is used to identify the person and to classify
the facial expressions. Our method reliably works even with faces,
which carry heavy expressions.
Abstract: Avoiding learning failures in mathematics e-learning environments caused by emotional problems in students with autism has become an important topic for combining of special education with information and communications technology. This study presents an adaptive emotional adjustment model in mathematics e-learning for students with autism, emphasizing the lack of emotional perception in mathematics e-learning systems. In addition, an emotion classification for students with autism was developed by inducing emotions in mathematical learning environments to record changes in the physiological signals and facial expressions of students. Using these methods, 58 emotional features were obtained. These features were then processed using one-way ANOVA and information gain (IG). After reducing the feature dimension, methods of support vector machines (SVM), k-nearest neighbors (KNN), and classification and regression trees (CART) were used to classify four emotional categories: baseline, happy, angry, and anxious. After testing and comparisons, in a situation without feature selection, the accuracy rate of the SVM classification can reach as high as 79.3-%. After using IG to reduce the feature dimension, with only 28 features remaining, SVM still has a classification accuracy of 78.2-%. The results of this research could enhance the effectiveness of eLearning in special education.
Abstract: The study of the interaction between humans and
computers has been emerging during the last few years. This
interaction will be more powerful if computers are able to perceive
and respond to human nonverbal communication such as emotions. In
this study, we present the image-based approach to emotion
classification through lower facial expression. We employ a set of
feature points in the lower face image according to the particular face
model used and consider their motion across each emotive expression
of images. The vector of displacements of all feature points input to
the Adaptive Support Vector Machines (A-SVMs) classifier that
classify it into seven basic emotions scheme, namely neutral, angry,
disgust, fear, happy, sad and surprise. The system was tested on the
Japanese Female Facial Expression (JAFFE) dataset of frontal view
facial expressions [7]. Our experiments on emotion classification
through lower facial expressions demonstrate the robustness of
Adaptive SVM classifier and verify the high efficiency of our
approach.
Abstract: Detection of human emotions has many potential applications. One of application is to quantify attentiveness audience in order evaluate acoustic quality in concern hall. The subjective audio preference that based on from audience is used. To obtain fairness evaluation of acoustic quality, the research proposed system for multimodal emotion detection; one modality based on brain signals that measured using electroencephalogram (EEG) and the second modality is sequences of facial images. In the experiment, an audio signal was customized which consist of normal and disorder sounds. Furthermore, an audio signal was played in order to stimulate positive/negative emotion feedback of volunteers. EEG signal from temporal lobes, i.e. T3 and T4 was used to measured brain response and sequence of facial image was used to monitoring facial expression during volunteer hearing audio signal. On EEG signal, feature was extracted from change information in brain wave, particularly in alpha and beta wave. Feature of facial expression was extracted based on analysis of motion images. We implement an advance optical flow method to detect the most active facial muscle form normal to other emotion expression that represented in vector flow maps. The reduce problem on detection of emotion state, vector flow maps are transformed into compass mapping that represents major directions and velocities of facial movement. The results showed that the power of beta wave is increasing when disorder sound stimulation was given, however for each volunteer was giving different emotion feedback. Based on features derived from facial face images, an optical flow compass mapping was promising to use as additional information to make decision about emotion feedback.
Abstract: This paper presents a new color face image database
for benchmarking of automatic face detection algorithms and human
skin segmentation techniques. It is named the VT-AAST image
database, and is divided into four parts. Part one is a set of 286 color
photographs that include a total of 1027 faces in the original format
given by our digital cameras, offering a wide range of difference in
orientation, pose, environment, illumination, facial expression and
race. Part two contains the same set in a different file format. The
third part is a set of corresponding image files that contain human
colored skin regions resulting from a manual segmentation
procedure. The fourth part of the database has the same regions
converted into grayscale. The database is available on-line for
noncommercial use. In this paper, descriptions of the database
development, organization, format as well as information needed for
benchmarking of algorithms are depicted in detail.
Abstract: A new approach for facial expressions recognition based on face context and adaptively weighted sub-pattern PCA (Aw-SpPCA) has been presented in this paper. The facial region and others part of the body have been segmented from the complex environment based on skin color model. An algorithm has been proposed to accurate detection of face region from the segmented image based on constant ratio of height and width of face (δ= 1.618). The paper also discusses on new concept to detect the eye and mouth position. The desired part of the face has been cropped to analysis the expression of a person. Unlike PCA based on a whole image pattern, Aw-SpPCA operates directly on its sub patterns partitioned from an original whole pattern and separately extracts features from them. Aw-SpPCA can adaptively compute the contributions of each part and a classification task in order to enhance the robustness to both expression and illumination variations. Experiments on single standard face with five types of facial expression database shows that the proposed method is competitive.