Systems Neurobiology Group

Goal

Humans and animals can learn and make decision to obtain more rewards. Goal of our research is to explore the neuronal mechanism of reward-based learning and decision. We aim to understand the mechanism from the viewpoint of computational theories, such as, the reinforcement learning. We have been testing the following two hypotheses by neurobiological experiments.

Hypotheses

Neromodulators control parameters in reward-based learning

The reinforcement learning theory provides the method of learning based on reward maximization. However, the some parameters in the algorithms should be tuned by hands. On the other hand, the brain seems to have a certain mechanism to tune these parameters, because humans and animals can learn novel behaviors under a wide variety of environments. We hypothesized that neuromodulatory systems control these parameters. More specifically, we propose the following set of hypothesis (Doya, 2002):

  1. Dopamine signals reward prediction error.
  2. Serotonin controls the time scale of prediction of future rewards.
  3. Noradrenaline controls the width of exploration.
  4. Acetylcholine controls the rate of memory updates.

A reinforcement learning algorithm is implemented in the basal-cortico circuit

Dopamine neurons in the midbrain fire when the amount of actual reward is larger than animal's expectation of reward. That is, dopamine neurons can be considered to code reward prediction error. The striatum, which is the input site of the basal ganglia, receives the dopaminergic input and as well as input from the cortex. In the reinforcement learning algorithm, the reward prediction error has a essential role to learn the optimal behavior. Therefore, we hypothesized that a reinforcement learning algorithm is implemented in the basal-cortico circuit (Doya, 2000, 2002).

References

  • Doya K., (2000),
    Complementary roles of basal ganglia and cerebellum in learning and motor control.
    Current Opinion in Neurobiology, 10(6), 732-739.
  • Doya K., (2002),
    Metalearning and neuromodulation.
    Neural Networks, 15, 495-506

Research topics

Currently, we have been performing following experiments:

  1. Recording activity of serotonergic neuron and measurement of concentration of serotonin during goal-directed behaviors of rats.
  2. Recording neuronal activity in the basal-ganglia of rats during free choice tasks.
  3. Functional magnetic resonance imaging (fMRI) on decision making of humans.
  4. Sequential learning of rats and humans.