Past Projects

1. Development of functional hierarchy for sensory-motor systems by MTRNN (Yamashita, Nishimoto, Tani)

This study examines how functional hierarchy can develop through sensory-motor interactions of agents. In this study, we propose a dynamic neural network model so-called the Multiple Timescales Recurrent Neural Network (MTRNN) which is characterized by coexistences of multiple timescale dynamics within a single network. A humanoid robot implemented with MTRNN was iteratively tutored for multiple goal-directed actions. The experimental results showed that compositional structures emerge through stages of developments where behavior primitives are generated in earlier stages and their sequences of achieving goals appear later stages. It was also observed that mental imagery is generated in earlier stages compared to actual behaviors. These results are corresponded to theories in developmental psychology with Piaget’s constructivist views.

Robot video page  Related paper page 

2. Model reconstruction of monkey whole-brain electrocorticography (ECoG) recording

(Komatsu and Tani) collaboration with Dr. Fujii @Adaptive Intelligence Lab in Riken BSI

This study under collaboration with Fujii Lab in RIKEN BSI aims to achieve model reconstruction of 64 channel ECoG data recorded from various cortical regions of monkeys which are accompanied with behavioral data. The results of the model reconstruction will provide direct feedback to large-scale brain network models which will be employed in our robotics experiments, particularly with regard to functional connectivity among cortical regions. Also, this study will provide an opportunity to testifying model predictions which are developed in our neuro-robotics studies.

3. Modeling of inferior parietal lobe as a sensory-forward model (Arie, Nishimoto, Tani)

In this modeling, we assume that inferior parietal lobe (IPL) might have function of so-called the sensory-forward model which predicts coming visuo-proprioceptive flow associated with target goals. With receiving the goal inputs from ventral premotor (PVm) or prefrontal cortex IPL may feedback the predicted visual state and the proprioceptive state to visual cortex and somatosensory cortex, respectively. Furthermore it is assumed that the predicted body posture state in terms of the proprioception might be sent to motor cortex as the next step target where necessary motor torques to achieve this target posture is obtained. IPL may generate mental simulations as well by setting feedback of own prediction outputs to the inputs. During actual behavior generation, the current goal might be modulated if there are gaps between the reality and the expected one.

4. Self-organization of sensory-motor scheme by using RNNPB (Tani, Ito at Sony, Nishimoto, Igari)

We propose a connetionist model, the recurrent neural network with parametric biases (RNNPB), in which multiple behavior schemata can be learned by the network in a distributed manner. The parametric biases in the network play an essential role in both generating and recognizing behavior patterns. We examine how cyclic and discrete movement patterns can be simultaneoulsly embedded in a single network with using an arm robot. The hallmark of this study is explaining how self-organizing internal structures can contribute to generalization in learning, and diversity in behavior generation, in the proposed distributed representationscheme.

Introduction of RNNPB (pdf) Experiments with an arm robot (pdf) 

5. The embodied language project (Sugita and Tani)

The associative learning of language and behavior is conducted while applying the RNNPB. The robotics experiments show that certain compositional structures are self-organized in the RNNPB, which achieves certain generalization capability for recognizing unseen sentences. This study proposes novel ideas of how symbolic structures can be grounded in the behavior experiences of cognitive agents.

More details (pdf)

A mobile robot approach and hit a red object after recognizing a sentence "HIT RED" even against severe perturbations.

6. Imitative interactions with SONY humanoid robot [collaborative studies with SONY] (Ito at Sony and Tani)

This paper studies a synthetic model for joint attentions and turn taking by conducting experiments of robot-user imitative interactions. A SONY humanoid robot learns multiple cyclic movement patterns as embedded in a neural network model proposed by us and each of memorized movement patterns can be retrieved and generated by means of entrainment by external inputs in terms of users' movement patterns perceived. Our experiments on a simple imitation game showed that multiple movement patterns are generated as synchronized between the robot and users while the shared patterns shift spontaneously from one to another. Based on the experimental results, we show possible dynamical systems accounts for the underlying mechanisms for joint attentions and turn taking.

More details pdf) 

7. Behavioral and brain imaging studies for human sequence learning

(Horie, Hirata, Tani) collaboration with Tanaka

Behavior studies and brain imaging using fMRI are conducted in human sequence learning tasks. Horie is investigating the neural correlates of the state transitions in a task of learning to predict symbolic sequences generated by finite state machines. Hirata is investigating the localizations of brain activities in a task of learning to predict movement pattern sequences generated from nonlinear map functions.

8. Neural mechanism of complex bird song in the Bengalese Finch

(Okumura and Tani) collaboration with Okanoya

Bengalese Finch learns to sing songs that have generation complexities of finite state machines. The neuronal correlations of the songs are investigated. Currently, the auditory responses to own songs are recorded in HVC using 4 Tetrodes in order to understand how such regularities are encoded.