Skip to content
Gallant Lab

A laboratory of cognitive, systems & computational neuroscience

Learn

The mammalian cerebral cortex is a multi-scale biological computing device consisting of billions of neurons, arranged in layered, local circuits. Connections within and between these circuits are both convergent and divergent, and both feed-forward and feed-back connections are typical. The result is a hierarchical, parallel, highly interconnected network of areas that tile the cerebral cortex.

The information represented in each cortical area reflects a nonlinear sum of the information in all areas that provide input to it. Thus, each area represents information explicitly that is only implicit in the input. The central goal of our research program is to discover how the mammalian brain represents information about the world and about its own mental states, by identifying and characterizing these cortical maps.

To address this problem, our laboratory makes heavy use of an inductive scientific approach called system identification, which is the foundation of the Voxelwise Encoding Model framework described below. Most of our data come from functional magnetic resonance imaging (fMRI) studies collected under very general conditions: while subjects watch long movies, listen to audio stories, or play video games. When necessary, we supplement this approach with targeted experiments optimized to test very specific hypotheses about brain function.

We then use classical statistical tools, Bayesian analysis and machine learning to fit computational models to the brain data. (These are usually called “encoding models”, because they describe how information about the world is encoded in the brain.) We evaluate several competing models to find the one that most accurately predicts responses, using a separate data set that was not used to estimate the model. The resulting encoding models describe what specific information is represented explicitly at each location across the cerebral cortex. These models are fit using himalaya, an open source Python package developed in our lab that estimates encoding models efficiently on both CPU and GPU hardware. A hands-on tutorial walks through the entire pipeline, from defining features to visualizing results on flattened cortical maps.

Our experiments generate large, rich data sets that cannot be summarized in a single number, so we develop software optimized for visualizing our results. Most notably, we developed the open source pycortex software for visualizing functional brain data (paper). Our brain viewers provide several demonstrations of this software.

Encoding models that accurately predict brain activity are the gold standard of systems and cognitive neuroscience, but they also have many practical uses. They provide a new tool for neurological evaluation and diagnosis, and a critical foundation for developing therapies to repair brain damage. They can also be inverted by means of a Bayesian framework in order to decode brain activity, which provides a direct and principled way to do “brain decoding”, and to build brain-machine interfaces (BMI) and neural prosthetics.

If you would like to know more about the general philosophy of the lab, please listen to this Freakonomics podcast interview with Jack Gallant or to these OHBM discussions between Peter Bandettini and Jack Gallant (discussion 1, discussion 2).

Voxelwise Encoding Model review paper

This review paper provides the first comprehensive guide to the Voxelwise Encoding Model (VEM) framework. The VEM framework is a framework for fitting encoding models to fMRI data. This framework is currently the most sensitive and powerful approach available for modeling fMRI data. It can be used to fit dozens of distinct models simultaneously, each model having up to several thousand distinct features. The Voxelwise Encoding Model framework also conforms to all best practices in data science, which maximizes sensitivity, reliability and generalizability of the resulting models.

Voxelwise Encoding Model tutorials

These Python tutorials show how to fit, evaluate, and interpret voxelwise encoding models on one of our public available datasets. We are providing these online tutorials here as a service to the fMRI community.

Theory paper focusing on feature space selection and banded ridge regression.

2022-12-01

Encoding models identify the information represented in brain recordings, but fitting multiple models simultaneously presents several challenges. This paper (Dupré la Tour et al., Neuroimage, 2022) describes how banded ridge regression can be used to solve these problems. Furthermore, several methods are proposed to address the computational challenge of fitting banded ridge regressions on large numbers of voxels and feature spaces. All implementations are released in an open-source Python package called Himalaya.

System identification review for sensory neuroscience

This review paper (Wu, David & Gallant, Annual Review of Neuroscience, 2006) provides a comprehensive introduction to system identification methods for sensory neuroscience. The paper shows how many different algorithms used in sensory neurophysiology can be viewed as variants of a single statistical inference problem: maximum a posteriori (MAP) estimation. It covers practical issues including stimulus selection, model estimation, regularization, visualization, and prediction-based validation. This paper provides the theoretical foundation for the encoding model framework used in the Gallant Lab.

The Gallant Lab in context

This interactive figure places the Gallant Lab's research in the context of the broader field. It traces a progression from low- and intermediate-level visual encoding and image reconstruction, through cortex-wide semantic mapping of vision and language, to how attention and task warp those representations during active naturalistic behavior — alongside the modeling toolkit that makes this work possible. Click any node to see the paper it represents, with its citation, summary, and a link to the original publication.