[Seminar] "Synergizing habitual and goal-directed behaviors for advancing decision-making AI" by Dr. Dongqi Han, Microsoft Research Asia

Dr. Dongqi Han, Senior Researcher, Microsoft Research Asia

Title: Synergizing habitual and goal-directed behaviors for advancing decision-making AI

Abstract:

Decision-making is fundamental in embodied environments for both biological and artificial agents. In cognitive science and psychology, two primary categories of behaviors have been identified: habitual and goal-directed. Habitual behavior is automatic and fast, without consideration of outcomes. Goal-directed behavior, on the other hand, involves planning of the outcomes of actions to achieve a desired objective, making it more reliable but computationally less efficient.

Numerous findings highlight the distinction and interaction between these behavioral modes. For example, with repeated practice on the same task, behavior gradually shifts from goal-directed to habitual. However, the underlying computational mechanisms remain unclear: Why does the brain adopt both modes? How are they orchestrated? And what can these insights contribute to improving decision-making in AI?

In this talk, I will begin by introducing a novel theoretical framework, referred to as Bayesian Behavior, that applies variational Bayesian principles to model the interaction and synergy between habits and goals in the brain, thereby explaining key experimental findings. Subsequently, I will discuss how we identified an underlying connection to diffusion model-based planning and action generation in decision-making AI (known as diffusion planning, e.g., Diffuser). Building on the insights from Bayesian Behavior, our recent studies have made diffusion planning (1) more flexible — capable of handling open-ended unseen tasks goals like in goal-directed behavior; and (2) more efficient — speeding computations by orders of magnitudes while preserving SOTA performance, akin to habitual behavior. The results highlight how insights from Bayesian Behavior can pave the way for more flexible, efficient, and effective embodied intelligence.

Bio:

Dongqi Han is currently a senior researcher in Microsoft Research Asia. He got his bachelor’s degree at the physics department of University of Science and Technology of China. Then, he obtained a Ph.D. degree at Okinawa Institute of Science and Technology, Japan. Dongqi are interested in neural networks, both biological and artificial ones; as well as the cognitive mechanisms of intelligent decision making. His work has been published at conferences such as ICLR, NeurIPS, ICML and journals like Nature Communications and npj Parkinson’s Disease. His recent research focuses on brain-inspired AI, embodied AI, as well as understanding the brain with advanced AI techniques.