Camera assisted multimodal user interaction

Jari Hannuksela; Olli Silvén; Sami Ronkainen; Sakari Alenius; Markku Vehviläinen

doi:10.1117/12.839449

27 January 2010 Camera assisted multimodal user interaction

Jari Hannuksela, Olli Silvén, Sami Ronkainen, Sakari Alenius, Markku Vehviläinen

Proceedings Volume 7542, Multimedia on Mobile Devices 2010; 754203 (2010) https://doi.org/10.1117/12.839449
Event: IS&T/SPIE Electronic Imaging, 2010, San Jose, California, United States

Abstract

Since more processing power, new sensing and display technologies are already available in mobile devices, there has been increased interest in building systems to communicate via different modalities such as speech, gesture, expression, and touch. In context identification based user interfaces, these independent modalities are combined to create new ways how the users interact with hand-helds. While these are unlikely to completely replace traditional interfaces, they will considerably enrich and improve the user experience and task performance. We demonstrate a set of novel user interface concepts that rely on built-in multiple sensors of modern mobile devices for recognizing the context and sequences of actions. In particular, we use the camera to detect whether the user is watching the device, for instance, to make the decision to turn on the display backlight. In our approach the motion sensors are first employed for detecting the handling of the device. Then, based on ambient illumination information provided by a light sensor, the cameras are turned on. The frontal camera is used for face detection, while the back camera provides for supplemental contextual information. The subsequent applications triggered by the context can be, for example, image capturing, or bar code reading.

Citation Download Citation

Jari Hannuksela, Olli Silvén, Sami Ronkainen, Sakari Alenius, and Markku Vehviläinen "Camera assisted multimodal user interaction", Proc. SPIE 7542, Multimedia on Mobile Devices 2010, 754203 (27 January 2010); https://doi.org/10.1117/12.839449

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available