The brain is one of the most complex material structures in the universe. Information is represented in three-dimensional (3-D) interconnections between neurons, arranged with sufficient complexity to represent all that can be understood as reality. Neurons functioning at the lowest structural level give rise to emergent behaviors at the level of the integrated living organism, turning on the cognitive switch in sentient beings. Without this causal relationship between structure at various scales, the brain would simply be an inanimate composite of corporeal matter. Neural imaging attempts to span this entire scale—from brain structure at the individual neuron to the vast interconnects at larger scales that create cognitive functionality. The brain, however, is challenging to image, posing an unusual problem for optics engineering. It requires light focusing through scattering tissue at the microscopic scale but using optical structures at the mesoscopic scale. With optogenetics, genetically engineered neurons can be made sensitive to light. This is a two-way challenge—light can be used both to read and write neural activity by detection, excitation or inhibition of action potentials. These methods approach the final frontier in distilling the emergence of conscious experience to its most fundamental, constituent, functional elements.

A panel of five postdoctoral researchers and doctoral candidates from neurophysiologist Hillel Adesnik’s lab in the Department of Molecular and Cell Biology and from optical scientist Laura Waller’s lab in the Department of Electrical Engineering and Computer Sciences conceived, chaired, and organized the conference. For Sculpted Light in the Brain 2017, they invited world-leading experts in neuroscience, computer science, and optical engineering to UC Berkeley’s Stanley Hall auditorium on a sunny day in early June. The conference was facilitated by a seed grant from QB3-MCB initiative, as well as private funding from corporate sponsors.

The impetus behind Sculpted Light in the Brain was to bring together researchers from fields that are often working independently on similar research goals to foster collaboration and share ideas and experience.

Similar to how brain structure and functionality must both be understood to reverse engineer the human neural code, neuroscientists, optical engineers, and computer scientists will all be necessary to make the next great discoveries in brain research. SLB2017 is not only a meeting of three fields attempting to “solve the brain” but also a gathering of fields that understand they will not be able to make a dent in this monumental challenge unless they collaborate closely. Experts from these communities, including pioneers of the BRAIN initiative, gathered for this daylong workshop, milling in the sunshine filtering through the generous alcoves of Stanley Hall.

In order to observe the brain live and awake, our scientific “windows” need certain properties: optical probes that can overcome scattering and indicate brain activity at the spatial scale of an individual neuron and at the temporal scale of a single action potential, for instance, functional calcium indicators (e.g. Gcamp). In the drive toward single-neuron resolution, near-IR stimulation is often used for two-photon neural imaging excitation while green light is emitted by the active, tagged neurons. Despite the opacity of the brain at visible wavelengths, researchers persist with them due to the spatial resolution afforded by the visible spectrum. While conventional one photon optical imaging is simpler to implement in terms of hardware complexity, nonlinear two photon imaging is eventually needed to compensate for the scattering of visible light in brain tissue. In either case, imaging in the visible regime hence feeds a self-referential loop where the brain tries to see itself at spectral ranges that the brain’s optical detection system — the eye — is most sensitive.

As scientists and researchers, we know that we can consciously compute, even create organized systems that can reliably perform logical operations, hence offloading the computation to external systems. At a deep fundamental level, the structural study of the brain begs the question: Can we compute consciousness?

The neural imagers hope to find the answer. This mechanistic picture needs two steps: neural activation in three dimensions to stimulate consequent behavior and the ability to probe neural signal patterns for a given behavior. Once the neural patterns are known for particular behaviors, correlations between the stochastic, highly correlated neural networks and the emergent mind — logical but also unpredictable — will allow characterizing and engineering consciousness from its structural building blocks.

“Concepts without data are empty, data without concepts, blind.” — Immanuel Kant

What is the level at which meaning, control, or logically connected patterns emerge in the brain? Is this at the neural level? Optogenetics tries to probe this by identifying which neurons are activated in response to stimuli. Or is it at the synaptic level, with the neurons simply acting as simple nodes in a representative network? If this is the case, true information would only be contained in the interconnections between neurons.

The brain is a highly parallel network, with only a few sequential steps connecting the correlated, simultaneous firing patterns that represent the interaction of thought and memory. However, it is also difficult to image. At the optical scales that can probe individual neurons, the brain is notoriously scattering, but at the scales at which it is electromagnetically permeable (for example at EEG frequencies), resolution is coarse. Hence a key factor for unlocking the structural mechanism of cognition is imaging at optical wavelength through the scattering in cerebral tissue.

The first few talks weaved through perspectives spanning biology, optics, and chemistry. The first session had Na Ji, new faculty at UC Berkeley, talking about using light screws for imaging deep into the tissue. Two photon microscopy enables greater depth penetration with high-power pulsed lasers, focal plane selectivity due to the quadratic depth activation of the Ca indicator tagging the neuron, but still needs focal plane scanning to image volumetrically. However, using self-healing spiral Bessel beams that are resilient to scattering allows imaging of the entire hundred-micron thick volume.

Ehud Isacoff discussed “quantal imaging” and optical control of neurons, showing selectivity in neural activation using precise indicators. “Flies are inside-out people,” he stated as he presented experiments imaging neural activity in drosophila. A flashy (literally) video of quantal analysis showed a constricted slug, its neurons firing and emitting light as it thought and squirmed, imaged directly through its semi-permeable skin.

Tommaso Fellini described neural imaging approaches as either statistical — measuring neural firing patterns and correlating with behavior — or as interventional-monitoring changes with artificial stimulation. The brain being amazingly plastic, however, often evades pinpointing of causality at the imaging scale of each neuron. Fellini talked about using different parts of the optical spectra for simultaneous stimulation and readout of neural activity. He argued that this could be the basic tool for probing the brain as a black box at the neural level.

Hillel Adesnik is a UC Berkeley faculty and one of the conference faculty sponsors. Claiming to be more an electrophysiologist by training — “I’m not really an optics guy” — Adesnik demonstrated a comprehensive approach for stimulating individual with two photon holography. His lab not only designs new opsins with good temporal response for precisely timed photostimulation, but the technology also stimulates multiple neurons simultaneously in 3-D. For this his team has a new holography technique with 3-D temporal focusing (3D-SHOT) that achieves resolution gains by splitting pulses of light in separate colors, and a spatial light modulator to recombine pulses inside targeted neurons. The idea of transcending scales is again crucial to his approach, as he stated that in future experiments, many functionally defined, randomly distributed neurons will have to be simultaneously activated to induce complex macroscopic behavior.

This set of talks is already describing the most current tools for neural imaging and stimulation. Once fully developed, they will allow looking at confined sets of neurons, firing in coordination, stimulating new patterns, a domino of activity that computational neuroscientists are currently investigating to reverse engineer the mind from the large, noisy datasets brain imaging technologies have already made accessible.

Lastly, the coupling of light to the functioning of neurons is constantly improved by biochemists and genetic engineers who design optically active tags for imaging and opsins for stimulation. Evan Miller described the fluorescence mechanism for chemically tagged neurons — photo-induced electron transfer, the intricate mechanism of the fluorophores in lipids, their polarity and activation, and how to engineer their mobility and reliable firing using sidegroups to stabilize their orientation in lipid membranes.

“Where is the singularity,

Where consciousness emerges?

Inside the neurons,

Or in between?” — Anonymous, coffee break

Michale Hausser introduced the closed loop optical interrogation technique, where probing and imaging simultaneously allows for understanding correlation behaviors of synapses. Here was also introduced the very insightful Hebb’s postulate, fundamental to the nature of correlation based computation in neural networks:

“Cells that fire together, wire together.” — Donald Hebb

Using closed loop monitoring and stimulation of pairs of neurons, Hausser showed that there are “yoke” behaviors between two neurons — the more they fire together via stimulation, the more correlated they become to each other’s individual firing behavior. In experiments they showed this neural weighting lasts on the order of days for rat brains.

The optical scientists distribute through the sessions — Rafael Piestun and our cohost Laura Waller talking about computational methods for imaging through scattering. These include control and inversion of optical patterns for activation and imaging. Some of the common themes include wavefront shaping, point spread function (PSF) engineering for depth selectivity where depth dependent PSFs allow for single-shot volumetric imaging, focusing through scattering media with characterization of the linear transmission matrix, its implementation with phase SLMs, and the converse, i.e. imaging through the scattering using inversion of the transmission matrix. Valentina Emiliani introduced techniques for focusing in 3-D volumes using multiple SLMs, enabling multiplication of the degrees of freedom for 3-D light shaping. Other tricks involved time-space exchange. For instance, temporal multiplexing builds up the depth variable, as long as the cameras are faster than the neural firing, allowing for building volumetric datasets in real-time.

The final speaker, Rafael Yuste from Columbia, is one of the key figures in inspiring the White House to sponsor the BRAIN initiative — a call for researchers to image entire neural ensembles in their functional habitat. What he described as the imaging and manipulation of neural circuits, the firing of neurons in groups, paraphrased in his talk title as the neural piano.

“Neurons are the mother of all emergent systems, meaning embedded in interactions.”

Neurons working together, in correlated networks, are super sticky — single cell photo-stimulation triggers embedded neural-ensembles, forming the basis of memory formation. Pathways that are most used are most reinforced — this is the foundation of mental conditioning, habit formation, possibly underlying the endless urge to find patterns that match pre-existing patterns in the brain. The plasticity of the brain, along with the tools of imaging with specificity at single cell resolution, enables imaging of entire neural ensembles as they fire across an organism’s psyche, ripples of will and thought, patterns making memory and intention. Only a subset of these neurons within a larger ensemble, when activated together, can activate the entire ensemble, determining the organism’s emergent behavior. For instance, in a lick test, tickling the right set of neurons in a mouse’s brain can determine which probe the mouse chooses to lick. Questions remain — imaging the patterns of neural activity, and conversely controlling them to replicate behaviors, does not quite imply any understanding of the mechanism governing the evolution of cognition and thought, of the models of cause and effect that drive will and intention. Is it possible to start at initial conditions of the neural circuit and predict subsequent states and, hence, behavioral patterns? What constitutes a “thought,” a “memory”? At which level of interaction is neural imaging congruous to mind reading?

Where and when in the physical brain does the light of the conceptual mind switch “on”? At what point is an organism holistically conscious — does the creative fountainhead of the mind exist at the smallest scale, at wave-functions fluctuating at sub neuron scales, or is it the interaction of many neurons that causes the emergence of conscious experience, the creation of the “mind” as seated in the “brain”?

The future is constantly here, so are the visionaries chasing it — scientists pursuing the ultimate mystery, the brain, our first and final frontier, showing its knobs but from behind the elusive veil of self-perception, from behind the fundamental uncertainty of a system trying to understand itself. Yet there is progress, understanding evolves, and researchers continue to illuminate with knowledge and perseverance our path through the tunnel of time. Stay tuned for #slb2018.