Humans make decisions all the time. In some cases—like ordering “the usual” at the local diner over trying the new chef special—we have two choices: to stick with what is familiar or to explore another option.
A team of researchers from the University College London (UCL) and the California Institute of Technology has just discovered what parts of the brain we use in weighing these options. Their findings could be significant in the study of disorders like compulsive gambling or drug addiction.
“We know relatively a lot about how people use their experience with the consequences of decisions—say making a lot of money or losing a lot of money—to improve their behavior in the future,” said Nathaniel Daw, a neuroscience postdoc at UCL and one of the study’s lead authors. “What we knew less about was how they gather up that experience, so we wanted to explore that aspect of decision making.”
Using a modified version of video game slot machines, subjects participating in the study gambled while researchers monitored their brain activity. While lying down in an fMRI scanner, subjects used one hand to press one of four buttons, each corresponding to a different slot machine, while watching a screen reflected in a mirror above them. As if at a casino, they could choose between the different slot machines. But instead of paying off only once in a while, the machines were programmed to pay off every time. The amount they paid oscillated around a different average value for each machine. The subjects’ goal was to figure out which one had the highest average pay off.
“Exploitation may be the basic drive to choose the ‘good option,’ while exploration can override this normal tendency.”
Traditionally, psychologists divide decision making into two strategies: exploitation and exploration. People exploit when they consistently choose the same option, believing it yields the highest rewards; they explore if they try new options, looking for something better.
In the gambling scenario, subjects exploited if they stuck with the slot machine they thought had the highest pay off; they explored if they hunted for a better one.
According to the scientists, explorers had high activity in the frontopolar cortex, a region behind the forehead thought to be involved in higher-level cognitive function. Exploiters, they found, use the prefrontal cortex, a region associated with reward and pleasure that’s known to be very old evolutionarily, said John O’Doherty, the study’s co-author and a professor of psychology at Caltech.
“It’s the same kind of pattern seen in a lot of other experiments, where logic and emotion come into conflict,” said Daw. “When people have to make hard moral decisions or have to choose between having one cookie now or 10 cookies in 10 minutes.”
The exploitative choice—having a cookie immediately—is often characterized as emotional, while the explorative choice of waiting is logical. According to O’Doherty, exploitation may be the basic drive to choose the “good option,” while exploration can override this normal tendency.
“One of the key aspects of decision making is that you always have to continue to explore your environment because you don’t know what’s the best option available,” said Daeyeol Lee, a neurobiology professor at Yale. “It’s almost like blind exploration in the beginning—if you go to a new town, or if you go to a new city, or if you go to a new store, you have to do some exploration before you actually start choosing the optimal option.”
The results of this study may be helpful in understanding the mechanisms behind several neurological problems, the researchers say. Compulsive gamblers, for example, could be prone to exploitative behavior and less likely to explore, O’Doherty surmised. Research into exploitative behavior may also help to characterize disorders like drug addiction and Parkinson’s disease, which are both associated with damage to the prefrontal cortex.
Originally published July 13, 2006