Non-normality in neural networks

Ankit Kumar; Kristofer Bouchard

doi:10.1117/12.2613472

2 March 2022 Non-normality in neural networks

Ankit Kumar, Kristofer Bouchard

Proceedings Volume 12019, AI and Optical Data Sciences III; 120190A (2022) https://doi.org/10.1117/12.2613472
Event: SPIE OPTO, 2022, San Francisco, California, United States

Abstract

A central goal of neuroscience is to link synaptic connectivity of neural circuits to produced dynamics and computations. Anatomical and functional connectivity within neural systems is asymmetric, which upon linearization gives rise to non-normal dynamics. Particular linear combinations of neurons that are involved in circuit function are canonically identified in systems neuroscience via PCA, which seeks subspaces which maximize variance. We have recently proposed Dynamical Components Analysis (DCA), which seeks subspaces of activity in which the mutual information between past and future activity (i.e., ‘the dynamic memory’) is largest. Here, we show that the presence of non-normality leads to a divergence between these subspaces and consequently, the importance of single neurons that are identified by each method. Applied to in-vivo electrophysiology recordings from diverse brain areas, subspaces of past-future mutual information are better able to predict animal and human behavior than subspaces of high variance. Finally, we discuss possible consequences of non-normality for the training and function of in-silico recurrent neural networks.

Conference Presentation

Citation Download Citation

Ankit Kumar and Kristofer Bouchard "Non-normality in neural networks", Proc. SPIE 12019, AI and Optical Data Sciences III, 120190A (2 March 2022); https://doi.org/10.1117/12.2613472

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available