Correction appended.
You are standing in line at Mama Kim's — your mind is racing and your mouth is salivating. Two weeks ago, you had the beef bulgogi rice set, and it was delicious. But tonight you are torn. Do you go with the bulgogi again because you know how much you like it, or do you try the beef galbi and potentially find an even better culinary delight? Should you exploit known rewards or explore, hoping to reap an even better prize?
David Badre and Michael Frank, professors of cognitive, linguistic and psychological sciences, cannot help you with your Mama Kim's dilemma. But thanks to their study published in the Feb. 9 issue of the journal Neuron, they can tell you what area of the brain is active when you make decisions based on relative uncertainty — the front part of your frontal lobe, known as the rostrolateral prefrontal cortex.
"This exploration-exploitation trade-off is a very classic problem, both for decisions everybody makes on a day-to-day basis and from an artificial intelligence standpoint," Frank said. "How do you know how often to go with what you think is best versus to explore?"
To try to answer this question, Badre and Frank used an experiment that Frank helped design and has used in previous studies. Participants had to stop a clock at a time between one and five seconds, earning a certain number of points depending on when they chose to stop it. The challenge in the game was that participants did not know the point values that corresponded to each time, and those point values were not constant.
Instead, points were assigned based on a probability distribution, meaning, for example, that when participants stopped the clock early they would have a greater chance of receiving points, but the point values would be lower. If they stopped the clock later on, they would have a smaller chance of receiving points, but when they did receive a reward, that reward would be much greater.
Participants who decided to "exploit" a 15-point reward — which they usually received when stopping the clock after one second — would fail to discover that a 350-point reward was occasionally available if they stopped the clock later.
This task involves a number of complex mental processes. To perform well, participants had to keep track of the number of times they received good and bad outcomes for different times and make decisions based on their level of uncertainty about each time, Frank said. Some participants did just that — they "explored," while others repeatedly stopped the clock at a time they knew from experience was likely to give them points, never venturing to discover if bigger rewards were available.
Though other studies have employed the clock game, this was the first time participants' brains were scanned using fMRI machines while playing.
"We already knew that there were explorers and non-explorers," Frank said. The purpose of their study was to discover what was going on in the brains of the different participants.
Based on his work and the work of others, Badre said, "there's some suggestion that those type of abstract signals are computed in the more rostral part of the frontal cortex."
The study confirmed their hypothesis that players who explored would show more activity in their rostrolateral prefrontal cortex than those who did not. This activity appeared milliseconds before subjects made their decisions.
Badre said questions remain about why some people explored and others didn't.
"We saw that this region in the frontal cortex only tracked relative uncertainty in the people who used it," he said. "Why is that?"
He and Frank both said it is unclear if some participants did not explore because they could not keep track of relative uncertainty, or if they simply chose to play the game using a different strategy and thus did not waste energy by computing what Frank called "operations on top of operations."
Frank said their research could be used in the future to more clearly understand diseases like obsessive-compulsive disorder and schizophrenia. He said many scientists believe that people with OCD are averse to uncertainty. Frank and Badre have run the experiment in patients with OCD, though they have not yet written up the results.
Running the clock test on people with OCD could reveal more about the nature of the disorder, Frank said. The fact that there is a specific brain region associated with making decisions using relative uncertainty could one day help "shed light on potential treatments," Frank said.
Badre warned that though the menu example is a classic way to explain making decisions that involve uncertainty, "it's hard to even speculate what this (research) would translate to in everyday behavior," he said.
Silvia Bunge, assistant professor of psychology at the University of California at Berkeley, called Badre and Frank's study "a sophisticated approach to the study of cognition" in an email to The Herald.
"As Kenny Rogers said, ‘You got to know when to hold 'em, know when to fold 'em, know when to walk away and know when to run,'" she wrote. "Rostrolateral prefrontal cortex, the subject of this study, appears to help us do just that."
An earlier version of this article incorrectly identified the rostrolateral prefrontal cortex as the back part of the frontal lobe. In fact, it is the front part of the frontal lobe. The article also stated the clock test had not been run on patients with obsessive compulsive disorder. In fact, the test has been run on OCD patients but results have not yet been written up. The Herald regrets the errors.
