[Online Machine Learning Insights and Innovations (MLII) Seminar Series] Self-Supervised Learning from Images and Videos using Optimal Transport, By Yuki M. Asano, University of Amsterdam

Zoom: https://oist.zoom.us/j/92257619030?pwd=d2IvVktjQUVPME8rdFhqWFlmNERRQT09
Meeting ID: 922 5761 9030
Passcode: 595720

Speaker: Dr. Yuki M. Asano, Assistant Professor, QUVA Lab, University of Amsterdam

Title: Self-Supervised Learning from Images and Videos using Optimal Transport

Abstract:
In this talk we will learn more about self-supervised learning -- the principles, the methods and how properly utilizing video data will unlock unprecendented visual performances.
I will first provide a brief overview of self-supervised learning and show how clustering can be combined with representation learning using optimal transport ([1] @ ICLR'20 spotlight). Next, I will show how this method can be generalised to multiple modalities ([2] @NeurIPS'20) and for unsupervised segmentation in images ([3] @CVPR'22) and in videos ([4] @ICCV'23). Finally, I show how optimal transport can be utilized to learn models from scratch from just a single Walking Tour video that outperform those trained on ImageNet, demonstrating high potential for future video-based embodied learning ([5] @ICLR'24). 
[1] Self-labelling via simultaneous clustering and representation learning
Asano, Rupprecht, Vedaldi.  ICLR, 2020
[2] Labelling unlabelled videos from scratch with multi-modal self-supervision. Asano, Patrick, Rupprecht, Vedaldi. NeurIPS 2020
[3] Self-supervised learning of object parts for semantic segmentation. Ziegler and Asano. CVPR 2022
[4] Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representations. Salehi, Gavves, Snoek, Asano.
[5] Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled video. Venkataramanan, Rizve, Carreira, Avrithis, Asano. ICLR 2024.