An unconscious study of audiovisual semantic consistency based on printed words

Dongjie Lang; Kun Zhou; Keling Huang; Weiqiang Peng; Ran Huo

doi:10.1117/12.2680948

8 June 2023 An unconscious study of audiovisual semantic consistency based on printed words

Dongjie Lang, Kun Zhou, Keling Huang, Weiqiang Peng, Ran Huo

Proceedings Volume 12707, International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2023); 1270746 (2023) https://doi.org/10.1117/12.2680948
Event: International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2023), 2023, Changsha, China

Abstract

Chen & Spence found in a previous study the time-courses and categorical specificity of cross-modal semantic congruency effects in pictures and printed words. To explore whether the time-courses and semantic consistency of audio and pictures can affect human’s visual processing unconsciously, we developed a Python-based audio-visual semantic consistency program to study the impact of auditory cues on the breakthrough timing of printed words under the Two-Alternative Forced Choice continuous flash suppression (2AFC-CFS) paradigm. Specifically, auditory cues were presented at 5 different (-1000ms, -750ms, -500ms, -250ms, 0ms) stimulus onset asynchronies (SOAs) with respect to the visual targets. In addition, there were 5 different match types in our auditory cues and printed words: congruent, incongruent, correlated, noisy and no-sound. The results of the study shows that SOA and congruency have the main effect in the unconscious condition. At the same time, spoken words produced greater facilitation than naturalistic sounds auditory stimuli. When leading by 500ms or more, spoken words induced a slowly emerging congruency effect in the congruent. In all cases, however, auditory stimuli sped up recognition compared to no sound. These results therefore suggests that the neural representations associated with visual and auditory stimuli can interact in a shared semantic system.

Citation Download Citation

Dongjie Lang, Kun Zhou, Keling Huang, Weiqiang Peng, and Ran Huo "An unconscious study of audiovisual semantic consistency based on printed words", Proc. SPIE 12707, International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2023), 1270746 (8 June 2023); https://doi.org/10.1117/12.2680948

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
5 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Semantics

Visualization

Printing

Information visualization

Consciousness

Brain

Displays

Show All Keywords

Keywords/Phrases

Search In:

Publication Years