Sparse conditional mixture model: late fusion with missing scores for multimedia event detection

Ramesh Nallapati; Eric Yeh; Gregory Myers

doi:10.1117/12.2007463

7 March 2013 Sparse conditional mixture model: late fusion with missing scores for multimedia event detection

Ramesh Nallapati, Eric Yeh, Gregory Myers

Proceedings Volume 8667, Multimedia Content and Mobile Devices; 866706 (2013) https://doi.org/10.1117/12.2007463
Event: IS&T/SPIE Electronic Imaging, 2013, Burlingame, California, United States

Abstract

The problem of event detection in multimedia clips is typically handled by modeling each of the component modalities independently, then combining their detection scores in a late fusion approach. One of the problems of a late fusion model in the multimedia setting is that the detection scores may be missing from one or more components for a given clip; e.g., when there is no speech in the clip; or when there is no overlay text. Standard fusion techniques typically address this problem by assuming a default backoff score for a component when its detection score is missing for a clip. This may potentially bias the fusion model, especially if there are many missing detections from a given component. In this work, we present the Sparse Conditional Mixture Model (SCMM) which models only the observed detection scores for each example, thereby avoiding making any assumptions about the distributions of the scores that are made by backoff models. Our experiments in multi-media event detection using the TRECVID-2011 corpus demonstrates that SCMM achieves statistically significant performance gains over standard late fusion techniques. The SCMM model is very general and is applicable to fusion problems with missing data in any domain.

Citation Download Citation

Ramesh Nallapati, Eric Yeh, and Gregory Myers "Sparse conditional mixture model: late fusion with missing scores for multimedia event detection", Proc. SPIE 8667, Multimedia Content and Mobile Devices, 866706 (7 March 2013); https://doi.org/10.1117/12.2007463

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available