DescriptionSeventy years ago, psychologist categorized the facial expression into seven categories: angry, disgust, fear, happiness, sadness, surprise and neutral. Through analyzing the expression, psychologists want to predict the emotions behind the expression. Due to all kinds of potential applications on human emotion analysis, automatical analysis of human affective expressions has attracted increasing attention from researchers in psychology, computer science, linguistics, neuroscience, and related disciplines. Researchers have done lots of works on this topic in the past thirty years, and proposed lots of promising approaches. Although many works have been done on this topic, these existing methods typically handle deliberately displayed and exaggerated expression of prototypical emotions. There are still some hard problems not solved well for the real system to handle naturally occurring emotions such as exploring discriminative features, time wrapping, and expression intensity estimation. Our work focuses on these real problems, analyzes the challenges in the real system and proposes the sounded solutions for advancing human affect sensing technology.