Parcellation of the brain (click on image to download).
Parcellation of the human brain based on responses to natural movies. Brain activity evoked by natural movies was measured using functional MRI. Volumetric fMRI signals were processed to identify brain locations that responded similarly to the movies. For visualization the volumetric fMRI signals were projected onto a flattened representation of the neocortical sheet. Each point in the flat map was then colored to reflect the similarity measurements. In this Figure the large maps are the flattened left and right hemisphere of one observer. Known functional areas are identified by white outlines, and major cortical sulci are identified by long black lines. (For reference the insets at lower left and lower right show postero-medial views of the inflated hemispheres.) Similar colors on these maps indicate brain regions that responded similarly to the movies. Natural movies tend to elicit similar responses from early and intermediate visual areas, and motor regions associated with eye movements (bright purple and red). Higher visual areas, areas associated with the default network and some regions of frontal cortex appear to form a separate functional cluster (yellow and orange). [Attribution: Shinji Nishimoto, Alex G. Huth, An Vu and Jack L. Gallant, UC Berkley, 2011.]
Images from 2011 Current Biology paper (click on image to download).
The left panel shows a segment of a Hollywood movie trailer that the subject viewed while in the magnet. The right panel shows the reconstruction of this segment from brain activity measured using fMRI. (In this static figure only the first frame of each segment is shown.) The procedure is as follows: [1] Record brain activity while the subject watches several hours of movie trailers. [2] Build dictionaries (regression model) to translate between the shapes, edges and motion in the movies and measured brain activity. A separate dictionary is constructed for each of several thousand points in the brain at which brain activity was measured. (For experts: our success here in building a movie-to-brain activity encoding model was one of the keys of this study) [3] Record brain activity to a new set of movie trailers that will be used to test the quality of the dictionaries and reconstructions. [4] Build a random library of ~18,000,000 seconds of video downloaded at random from YouTube (that have no overlap with the movies subjects saw in the magnet). Put each of these clips through the dictionaries to generate predictions of brain activity. Select the 100 clips whose predicted activity is most similar to the observed brain activity. Average those clips together. This is the reconstruction. [Attribution: Shinji Nishimoto & Jack L. Gallant, UC Berkeley, 2011.]
The the movie that each subject viewed while in the magnet is shown at upper left. (In this static figure only the first frame of each segment is shown.) Reconstructions for three subjects are shown in the three rows at bottom. All these reconstructions were obtained using only each subject's brain activity and a library of 18 million seconds of random YouTube video that did not include the movies used as stimuli. (In brief, the algorithm processes each of the 18 million clips through the brain model, and identifies the clips that would have produced brain activity as similar to the measured activity as possible. The clips used to fit the model, the clips used to test the model and the clips used to reconstruct the stimulus were entirely separate.) The reconstruction at far left is the Average High Posterior (AHP). The reconstruction in the second column is the Maximum a Posteriori (MAP). The other columns represent less likely reconstructions. The AHP is obtained by simply averaging over the 100 most likely movies in the reconstruction library. These reconstructions show that the process is very consistent, though the quality of the reconstructions does depend somewhat on the quality of brain activity data recorded from each subject. [Attribution: Shinji Nishimoto, An T. Vu, Thomas Naselaris, Yuval Benjamini, Bin Yu & Jack L. Gallant. Reconstructing dynamic visual experiences from brain activity evoked by natural movies. Current Biology, http://dx.doi.org/10.1016/j.cub.2011.08.031.]