[Seminar] MLDS Unit Seminar 2025-3 by Dr. Ziyin Liu, MIT and NTT Research

Speaker: Dr. Ziyin Liu,  MIT and NTT Research 

Title: Universal Phenomena, Irreversibility, and Thermodynamics in Deep

Abstract: With the rapid discovery of emergent phenomena in deep learning and large language models, explaining and understanding their cause has become an urgent need. In this semianr, I present a rigorous entropic-force theory for understanding the learning dynamics of neural networks trained with stochastic gradient descent (SGD) and its variants. Building on the theory of parameter symmetries and an entropic loss landscape, I show that representation learning is crucially governed by emergent entropic forces arising from stochasticity and discrete-time updates. These forces systematically break continuous parameter symmetries and preserve discrete ones, leading to a series of gradient balance phenomena that resemble the equipartition property of thermal systems. These phenomena, in turn, (a) explain the universal alignment of neural representations between AI models and lead to a proof of the Platonic Representation Hypothesis, and (b) reconcile the seemingly contradictory observations of sharpness- and flatness-seeking behavior of deep learning optimization. I present theory and experiments to demonstrate that a combination of entropic forces and symmetry breaking is key to understanding emergent phenomena in deep learning.

Bio: I am a Postdoctoral Fellow at MIT and NTT Research. At MIT, I work with Prof. Isaac Chuang. I also collaborate with Prof. Tomaso Poggio in the BCS department. My research focus is on the theoretical foundation of deep learning and, recently, computational neuroscience. Prior to coming to MIT, I received my PhD in physics at the University of Tokyo under the supervision of Prof. Masahito Ueda. I received a Bachelor's degree in physics and mathematics at Carnegie Mellon University.