Seminars on neural basis for emotion and the basal ganglia circuit

Dear All,

Neural Computation Unit (Doya Unit) would like to invite you to two seminars on neural basis for emotion and the basal ganglia circuit as follows.

-------------------
Date: Monday, April 14, 2014
Time: 14:45-16:15
Venue: OIST Center Bldg. Seminar room B250
--------------------
Speaker: Prof. Kenji Doya   14:45-15:30
Title: How to cope with delayed rewards

Abstract: In our life, reward or punishment often comes after an uncertain delay.After seeing and trying many things, how can we tell what caused a good or bad result? This is a classic case of “temporal credit assignment” or delayed reward problem.The theory of reinforcement learning prescribes a number of strategies to cope with this problem:
1) look back the past and attribute the cause to the more recent events.
2) set up a decaying “eligibility trace” for each event and associate them with the reward acquired.
3) learn to predict the expected future rewards and regard its increase as an effective reward.
4) learn a dynamic model of the change of the world and predict what actions will lead to the reward.

While we engineers have been excited with developments of new reinforcement learning algorithms during the last several decades, it might be that our brain had discovered such methods millions of years ago during the course of evolution. More specifically,
1) the hippocampus may provide episodic memory for looking back past events for attribution.
2) molecular signals in each synapse may provide eligibility trace for event-reward association.
3) reward predictive response of dopamine neurons may work as reward in advance.
4) internal models learned in the cerebellum and the frontal cortex may realize “mental simulation” to search for right actions.

In addition to the choice and combination of these strategies, an interesting issue is how far into the future or back into the past we try to predict or attribute to. We hypothesize that serotonin is involved in such regulation of time scale and introduce our recent work on recording and manipulation of serotonin neurons in rodents.

Speaker: Prof. Jeff Wickens 15:30-16:15

Title: Modulation of corticostriatal synaptic plasticity by the dopaminergic system.
Authors:Tomomi Shindou, Mayumi Ochi-Shindou, Jeffery Wickens

Abstract: We are investigating dopamine and activity-dependent synaptic plasticity in the striatum as a cellular mechanism for reinforcement learning. Phasic dopamine release is associated with positive reinforcement. However, neural activity related to decision-making and motor responses may precede delivery of reinforcement by several seconds. How does reinforcement act retrospectively to modify the appropriate synapses? To bridge the delay between action and outcome, computational models of reinforcement learning assume a synaptic eligibility trace: synapses involved in a pre-and-post synaptic conjunction of activity are tagged by the eligibility trace; and a later global reinforcement signal acts restropectively to strengthen tagged synapses. To test the eligibility trace hypothesis in corticostriatal synapses we combined spike-time dependent plasticity induction protocols with phasic application of dopamine by photolytic uncaging. In dopamine D1-receptor expressing spiny projection neurons, pre-before-post spike timing normally caused long-term depression. Dopamine pulses occurring 2s after pre-post spike timing reversed this long-term depression. Under certain conditions long-term potentiation was observed. These findings support the eligibility trace hypothesis, and show how phasic release of dopamine can selectively strengthen recently active synapses. Further work is needed to identify the cellular and molecular mechanisms underlying the eligibility trace.
--------------------

We look forward to seeing many of you there.