Improvement of Thai speech emotion recognition using face feature analysis

Abstract

Emotions are not usually expressed in the Thai language, because emotional stress would interfere with the speaker's meaning, which makes emotion recognition using Thai speech difficult. Humans use audio cues to recognize fear, anger and disgust, while happiness and surprise seem to be strong "visual" emotions. Our experiments prove this assumption and we present a simple way to combine speech and facial features. The proposed Thai emotion recognition system augments the speech emotion recognition process with face feature analysis via an audiovisual Thai emotion database. Our speech emotion recognizer is based on calculating mel-frequency cepstral coefficients, zero crossing rate, and energy from short-time speech signals. Our face feature analysis is composed of several newly proposed approaches, includes the use of a reference point and middle frames in image sequences, and fully automatic facial landmark point extraction. It produces very good facial expression recognition results and improves the accuracy of the overall system and reduces errors.The combination of speech and facial expressions show that both vision and hearing play an important part in expressing and recognizing emotions. © 2012 Praise Worthy Prize S.r.l. - All rights reserved.