One of the major research areas in our laboratory concerns “brain reading”. Many people are interested in brain reading, so we answer common questions about this topic here.
There are two main reasons that scientists conduct research on brain reading. First, many scientists and engineers seek to develop brain-machine or brain-computer interfaces that can be used to solve problems in the real world. For example, many different labs are trying to develop brain-machine interfaces that can be used to control prosthetic limbs. However, practical brain reading devices would have wide applicability in many other fields, including in communication, medical diagnosis and entertainment. The second motivation for brain reading research is that it can serve as a useful adjunct to basic research in cognitive neuroscience. One way to test whether some part of the brain contains a specific type of information is to see whether that information can be decoded from brain activity.
It is useful to distinguish between three types of brain reading: classification, identification and reconstruction. In classification we try to determine which one of several possible brain states a person is in. For example, one can try to classify whether a person has seen a horizontal or a vertical grating. In identification we try to determine which specific stimulus a subject has seen, when the stimulus has been drawn from a list of known stimuli. For example, imagine that someone has drawn a photograph at random from a large stack of photos. If we record this person’s brain activity while they view the photo, can we search through the entire stack of photos and guess which one they saw? Reconstruction is the most ambitious form of brain reading. In reconstruction we try to reconstruct the stimulus that a person saw while they were being scanned, even if the stimulus is completely novel and has never been seen before. For example, if someone watched a movie while in the scanner, we could try to recreate the movie that they were watching. All three of these types of brain reading have been demonstrated in using visual stimuli such as gratings, geometric patterns and natural images.
At this time non-invasive approaches to brain reading include functional MRI (fMRI), magneto-encephelography (MEG), electro-encephelography (EEG) or near infra-red spectroscopy (NIRS). However, these techniques vary substantially in the quality of results that can be achieved. Functional MRI can recover a wide range of information from the brain, MEG somewhat less, EEG relatively little and NIRS still less. Although fMRI provides the best non-invasive measures of brain activity (and so the best decoding), it is an extremely expensive technique that requires a very large magnet housed in a shielded room. Therefore, future brain reading devices that are both portable and non-invasive will be based on other technologies, such as MEG. Note that in theory brain reading could be performed invasively by placing electrodes between the skull and the surface of the brain, or by inserting them directly into the brain. However, these invasive techniques are unlikely to be used except in rare medical situations where an invasive surgery might provide significant improvement in life quality. Therefore, much current work focuses on non-invasive techniques for brain reading that are likely to have wider applicability.
As noted elsewhere in this FAQ there are several different approaches to brain reading, and the procedures differ. Our laboratory typically uses a two step procedure. The first step of the process is to construct a set of encoding models that describes how visual stimuli are represented in the pattern of activity across visual cortex. The assumption here is that the activity in visual cortex is systematically related to the particular visual stimulus that is being viewed at any point in time. The goal is to build encoding models that can take any visual stimulus and predict the pattern of activity that the image would produce in visual cortex. The second step of the process is to use these encoding models to solve the image reconstruction problem. One way to describe how we approach this problem is to think of all possible images as potential reconstructions. To determine if a particular candidate is an accurate reconstruction, we use the encoding models to determine the likelihood that it would evoke the brain activity we actually measured. We also use prior information about what natural images look like to determine if the candidate is an accurate reconstruction. Both of these sources of information are used to converge on a reconstruction that is both structurally and semantically accurate.
Decoding performance depends on the quality of brain activity measurements. At this time the best way to measure brain activity is with functional MRI (fMRI). However, fMRI does not actually measure the activity of neurons. Instead, it measures blood flow consequent to neural activity. Many studies have shown that the blood flow signals measured using fMRI are generally correlated with neural activity. However, fMRI has relatively modest spatial and temporal resolution, so much of the information contained in the underlying neural activity is lost when using this technique. fMRI measurements are also quite variable from trial to trial. Both of these factors limit the amount of information that can be decoded from fMRI measurements. Decoding also depends critically on our understanding of how the brain represents information, because this will determine the quality of the computational model. If the encoding model is poor (i.e., if it does a poor job of prediction) then the decoder will be inaccurate. While our computational models of some cortical visual areas perform well, they do not perform well when used to decode activity in other parts of the brain. A better understanding of the processing that occurs in parts of the brain that mediate more abstract cognitive processes will be required before it will be possible to decode other aspects of human experience.
There are many potential applications of devices that can decode brain activity. In addition to its value as a basic research tool, it could be used to aid in diagnosis of diseases (e.g., stroke, dementia); to assess the effects of therapeutic interventions (drug therapy, stem cell therapy); or as the computational heart of a neural prosthesis. It could also be used to build a brain-machine interface, as described below.
Neuroscientists generally assume that all mental processes have a concrete neurobiological basis. Under this assumption, as long as we have good measurements of brain activity and good computational models of the brain, it should be possible in principle to decode the visual content of mental processes like dreams, memory, and imagery. However, current computational models of visual processing have been developed to account for visual perception of natural scenes. The accuracy of these models for decoding subjective states such as dreaming and imagination will depend on how similar those processes are to normal visual perception. This is an active topic of research in our lab and in many other labs.
The potential use of this technology in the legal system is problematic. Many psychology studies have demonstrated that eyewitness testimony is notoriously unreliable. Witnesses often have poor memory but often do not realize that their memory is poor; their recollections tend to be biased by intervening events, inadvertent coaching, and rehearsal (prior recall); and they often confabulate stories to make logical sense of events that they cannot recall well. These errors are thought to stem from several factors: poor initial storage of information in memory; changes to stored memories over time; and faulty recall. Any brain reading device that aims to decode stored memories will inevitably be limited not only by the technology itself, but also by the quality of the stored information (an accurate read-out of a faulty memory only provides misleading information). Therefore, any future application of this technology in the legal system will have to be considered with extreme caution.
Not in the foreseeable future. In this experiment brain activity was measured using fMRI, which is only indirectly coupled to neural activity. There is no known technology that can remotely send signals to the brain in a way that would be organized enough to elicit a meaningful mental state.
No, brain reading cannot be performed remotely, and it cannot be performed without the knowledge of the subject. Brain reading requires that brain activity be measured at fairly high spatial and temporal resolution. At this time there are no brain reading technologies that can record brain activity remotely at a resolution sufficient to decode mental activity. All successful brain reading experiments use devices such as fMRI machines, MEG machines or EEG recording arrays that must be placed in close proximity to the brain. These devices are quite large (for example, an fMRI machine weighs several tons), and they would certainly be noticed if they were being used. Furthermore, any small movements such as head motion, blinking or coughing will dramatically reduce the quality of the brain activity measurements. Therefore, all these technologies require the cooperation of the individual whose brain activity is being measured.
Current methods for decoding brain activity are relatively primitive. Current computational models are immature and in order to construct a model of someone’s visual system they must spend many hours in a large, stationary magnetic resonance scanner. For this reason it is unlikely that brain reading technology could be used in practical applications any time soon. That said, both the technology for measuring brain activity and the computational models are improving continuously. It is possible that decoding brain activity could have serious ethical and privacy implications downstream in, say, the 30- to 50-year time frame. As an analogy, consider the current debates regarding availability of genetic information. Genetic sequencing is becoming cheaper by the year, and it will soon be possible for everyone to have their own genome sequenced. This raises many issues regarding privacy and the accessibility of individual genetic information. We believe strongly that no one should ever be subjected to any form of brain-reading process involuntarily, covertly, or without complete informed consent.