Classifying emotions via analysis of facial physiological response without relying on expressions

Yitzhak Yitzhaky; Shaul Shvimmer; Shlomi Talala; Rotem Simhon; Michael Gilad

doi:10.1117/12.3034867

13 November 2024 Classifying emotions via analysis of facial physiological response without relying on expressions

Yitzhak Yitzhaky, Shaul Shvimmer, Shlomi Talala, Rotem Simhon, Michael Gilad

Proceedings Volume 13206, Artificial Intelligence for Security and Defence Applications II; 132060I (2024) https://doi.org/10.1117/12.3034867
Event: Security + Defence, 2024, Edinburgh, United Kingdom

Abstract

Assessing a person’s emotional state may be relevant to security in situations where it may be beneficial to assess one’s intentions or mental state. In various situations, facial expressions that often indicate emotions, may not be communicated or may not necessarily correspond to the actual emotional state. Here we review our study, in which we classify emotional states from very short facial video signals. The emotion classification process does not rely on stereotypical facial expressions or contact-based methods. Our raw data are short facial videos obtained at some different known emotional states. A facial video includes a component of diffused light from the facial skin, affected by the cardiovascular activity that might be influenced by the emotional state. From the short facial videos, we extracted unique spatiotemporal physiological-affected features employed as input features into a deep-learning model. Results show average emotion classification accuracy of about 47.36%, compared to 20% chance accuracy given 5 emotion classes, which can be considered high for the cases where expressions are hardly observed.

Citation Download Citation

Yitzhak Yitzhaky, Shaul Shvimmer, Shlomi Talala, Rotem Simhon, and Michael Gilad "Classifying emotions via analysis of facial physiological response without relying on expressions", Proc. SPIE 13206, Artificial Intelligence for Security and Defence Applications II, 132060I (13 November 2024); https://doi.org/10.1117/12.3034867

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available