Publication List of the Adaptive Systems Group

2017

• Wang, J., Uchibe, E., & Doya, K. (2017). Adaptive Baseline Enhances EM-based Policy Search: Validation in a View-based Positioning Task of a Smartphone Balancer. Frontiers in Neurorobotics, 11:1.
• Elfwing, S., Uchibe, E., & Doya, K. (2017). Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning. arXiv:1702.03118.

2016

• Elfwing, S., Uchibe, E., & Doya, K. (2016). From free energy to expected energy: Improving energy-based value function approximation in reinforcement learning. Neural Networks, 84: 17-27.
• Wang, J., Uchibe, E., & Doya, K. (2016).EM-based Policy Hyper Parameter Exploration: Application to Standing and Balancing of a Two-wheeled Smartphone Robot. Journal of Artificial Life and Robotics. vol. 21, issue 1, pp. 125-131.
• Uchibe, E. (2016). Deep inverse reinforcement learning by logistic regression. In Proc. of the 23rd International Conference on Neural Information Processing (ICONIP), pp. 23-31.
• Huang, Q., Uchibe, E., & Doya, K. (2016). Emergence of communication among reinforcement learning agents under coordination environment. In Proc. of the 6th Joint IEEE International Conference on Developmental Learning and on Epigenetic Robotics, accepted.
• Reinke, C., Uchibe, E., & Doya, K. (2016). From Neuroscience to Artificial Intelligence: Maximizing Average Reward in Episodic Reinforcement Learning Tasks with an Ensemble of Q-Learners. In the Third CiNet Conference, Neural mechanisms of decision making: Achievements and new directions, Osaka, poster.
• Reinke, C., Uchibe, E., & Doya, K. (2016). Learning of Stress Adaptive Habits with an Ensemble of Q-Learners. In The 2nd International Workshop on Cognitive Neuroscience Robotics, Osaka, poster.

2015

• Elfwing, S., Uchibe, E., & Doya, K. (2015).Expected energy-based restricted Boltzmann machine for classification. Neural Networks. vol. 64, 29-38.
• Reinke, C., Uchibe, E., & Doya, K. (2015). Maximizing the average reward in episodic reinforcement learning tasks. In Proc. of IEEE International Conference on Intelligent Informatics and BioMedical Sciences, Okinawa, pp. 420-421, 2015.
• Wang, J., Uchibe, E., & Doya, K. (2015). Two-wheeled smartphone robot learns to stand up and balance by EM-based policy hyper parameter exploration. In Proc. of the 20th International Symposium on Artificial Life and Robotics.
• Uchibe, E., & Doya, K. (2015). Inverse Reinforcement Learning with Density Ratio Estimation. The 2nd Multidisciplinary Conference on Reinforcement Learning and Decision Making, University of Alberta, Canada, poster.
• Reinke, C., Uchibe, E., & Doya, K. (2015). Gamma-QCL: Learning multiple goals with a gamma submodular reinforcement learning framework. In Winter Workshop on Mechanism of Brain and Mind. (poster presentation).

2014

• Elfwing, S., & Doya, K. (2014). Emergence of polymorphic mating strategies in robot colonies. PLoS ONE, 9(4), e93622.
• Uchibe, E., & Doya, K. (2014). Inverse Reinforcement Learning Using Dynamic Policy Programming. In Proc. of the 4th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, pp. 222-228.
• Uchibe, E., & Doya, K. (2014). Combining learned controllers to achieve new goals based on linearly solvable MDPs. In Proc. of IEEE International Conference on Robotics and Automation, pp. 5252-5259.
• Eren Sezener, C., Uchibe, E., & Doya, K. (2014). Ters Peki，stirmeli Ogrenme ile Farelerin Odul Fonksiyonunun Elde Edilmesi. In Proc. of Turkiye Otonom Robotlar Konferans? (TORK). [published in Turkey, but see also an English version].
• Kinjo, K., Uchibe, E., & Doya, K. (2014). Robustness of Linearly Solvable Markov Games with inaccurate dynamics model. In Proc. of the 19th International Symposium on Artificial Life and Robotics.
• Wang, J. Uchibe, E., & Doya, K. (2014). Control of Two-Wheeled Balancing and Standing-up Behaviors by an Android Phone Robot. Proc. of the 32nd Annual Conference of Robotics Society of Japan, Kyushu Sangyo University, 2014.

2013

• Kinjo, K., Uchibe, E., & Doya, K. (2013). Evaluation of linearly solvable Markov decision process with dynamic model learning in a mobile robot navigation task. Frontiers in Neurorobotics, 7(7).
• Elfwing, S., Uchibe, E., & Doya, K. (2013). Scaled free-energy based reinforcement learning for robust and efficient learning in high-dimensional state spaces. Frontiers in Neurorobotics, 7(February), 3.
• Sakuma, T., Shimizu, T., Miki, Y., Doya, K., & Uchibe, E. (2013). Computation of Driving Pleasure based on Driver's Learning Process Simulation by Reinforcement Learning. In Proc. of Asia Pacific Automotive Engineering Conference.
• Yoshida, N., Uchibe, E., & Doya, K. (2013). Reinforcement learning with state-dependent discount factor. In Proc. of the 3rd Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (pp. 1-6). IEEE.
• Wang, J., Uchibe, E., & Doya, K. (2013). Standing-up and Balancing Behaviors of Android Phone Robot. In Proc. of IEICE-NLP2013-122, 49-54.
• Uchibe, E., Ota, S., & Doya, K. (2013). Inverse Reinforcement Learning for Analysis of Human Behaviors. The 1st Multidisciplinary Conference on Reinforcement Learning and Decision Making, Princeton, New Jersey, USA, poster.
• Ota, S., Uchibe, E., & Doya, K. (2013). Analysis of human behaviors by inverse reinforcement learning in a pole balancing task. The 3rd International Symposium on Biology of Decision Making, Paris, France, poster.

2011

• Uchibe, E., & Doya, K. (2011). Evolution of rewards and learning mechanisms in Cyber Rodents. J. K. Krichmar and H. Wagatsuma (eds.), Neuromorphic and Brain-Based Robotics, chapter 6, 109-128.
• Elfwing, S., Uchibe, E., Doya, K., & Christensen, H. I. (2011). Darwinian embodied evolution of the learning ability for survival. Adaptive Behavior, 19(2), 101-120.

2010

• Morimura, T., Uchibe, E., Yoshimoto, J., Peters, J., & Doya, K. (2010). Derivatives of logarithmic stationary distributions for policy gradient reinforcement learning. Neural Computation, 22(2), 342-76.
• Elfwing, S., Otsuka, M., Uchibe, E., & Doya, K. (2010). Free-Energy Based Reinforcement Learning for Vision-Based Navigation with High-Dimensional Sensory Inputs. In Proc. of the 17th International Conference on Neural Information Processing (pp. 215-222).

2009

• Uchibe, E., & Doya, K. (2009). Constrained Reinforcement Learning from Intrinsic and Extrinsic Rewards. In M. J. Er & Y. Zhou (Eds.), Theory and Novel Applications of Machine Learning. IN-TECH.
• Elfwing, S., Uchibe, E., Doya, K., & Christensen, H. I. (2009). Co-evolution of Rewards and Meta-parameters in Embodied Evolution. In B. Sendhoff, E. Korner, O. Sporns, H. Ritter, & K. Doya (Eds.), Creating Brain-Like Intelligence (pp. 278-302). Springer.
• Morimura, T., Uchibe, E., Yoshimoto, J., & Doya, K. (2009). A Generalized Natural Actor-Critic Algorithm. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, & A. Culotta (Eds.), Advances in Neural Information Processing Systems 22 (pp. 1312-1320). MIT Press.
• Elfwing, S., Uchibe, E., & Doya, K. (2009). Emergence of Different Mating Strategies in Artificial Embodied Evolution. In Proc. of the 16th International Conference on Neural Information Processing (pp. 638-647).

2008

• Uchibe, E., & Doya, K. (2008). Finding intrinsic rewards by embodied evolution and constrained reinforcement learning. Neural Networks, 21(10), 1447-55.
• Elfwing, S., Uchibe, E., Doya, K., & Christensen, H. I. (2008). Co-evolution of Shaping Rewards and Meta-Parameters in Reinforcement Learning. Adaptive Behavior, 16(6), 400-412.
• Sato, T., Uchibe, E., & Doya, K. (2008). Learning how, what, and whether to communicate: emergence of protocommunication in reinforcement learning agents. Journal of Artificial Life and Robotics, 12, 70-74.
• Morimura, T., Uchibe, E., Yoshimoto, J., & Doya, K. (2008). A New Natural Policy Gradient by Stationary Distribution Metric. In Proc. of European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (pp. 82-97). Springer Berlin / Heidelberg.
• Morimura, T., Uchibe, E., & Doya, K. (2008). Natural policy gradient with baseline adjustment function for variance reduction. Artificial Life and Robotics, 2008.
• Kamioka, T., Uchibe, E., & Doya, K. (2008). Neuroevolution Based on Reusable and Hierarchical Modular Representation. Proc. of INNS-NNN Symposia.

2007

• Elfwing, S., Uchibe, E., Doya, K., & Christensen, H. I. (2007). Evolutionary Development of Hierarchical Learning Structures. IEEE Transactions on Evolutionary Computation, 11(2), 249-264.
• Uchibe, E., & Doya, K. (2007). Constrained reinforcement learning from intrinsic and extrinsic rewards. In Proc. of International Conference on Developmental Learning. London, UK: IEEE.
• Uchibe, E., & Doya, K. (2007). Finding Exploratory Rewards by Embodied Evolution and Constrained Reinforcement Learning in the Cyber Rodents. In Proc. of the 14th International Conference on Neural Information Processing, 167-176. Kitakushu, Japan: Springer Berlin.

2006

• Uchibe, E., & Asada, M. (2006). Incremental Coevolution With Competitive and Cooperative Tasks in a Multirobot Environment. Proceedings of the IEEE, 94(7), 1412-1424.
• Brunskill, E., Uchibe, E., & Doya, K. (2006). Adaptive state space construction with reinforcement learning for robots. poster presentation in Proc. of the International Conference on Robotics and Automation.

2005

• Doya, K., & Uchibe, E. (2005). The Cyber Rodent Project: Exploration of Adaptive Mechanisms for Self-Preservation and Self-Reproduction. Adaptive Behavior, 13, 149-160.
• Morimura, T., Uchibe, E., & Doya, K. (2005). Utilizing the natural gradient in temporal difference reinforcement learning with eligibility traces. In Proc. of the 2nd International Symposium on Information Geometry and its Application (pp. 256-263).
• Uchibe, E., & Doya, K. (2005). Reinforcement Learning with Multiple Modules: A Framework for Developmental Robot Learning. In Proc. of the 4th International Conference on Developmental Learning, 87-92.
• Elfwing, S., Uchibe, E., Doya, K., & Christensen, H. I. (2005). Biologically inspired embodied evolution of survival. In Proc. of the IEEE Congress on Evolutionary Computation, 2210-2216.

2004

• Elfwing, S., Uchibe, E., Doya, K., & Christensen, H. I. (2004). Multi-agent reinforcement learning: using macro actions to learn a mating task. In Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (Vol. 4, pp. 3164-3169).
• Uchibe, E., & Doya, K. (2004). Competitive-Cooperative-Concurrent Reinforcement Learning with Importance Sampling. In S. Schaal, A. Ijspeert, A. Billard, S. Vijayakumar, J. Hallam, & J.-A. Meyer (Eds.), Proc. of the Eighth International Conference on Simulation of Adaptive Behavior: From Animals to Animats 8 (pp. 287?296). MIT Press, Cambridge, MA.