Yuncong Yang

I am currently a research assistant at the MIT-IBM Watson AI Lab, advised by Prof. Chuang Gan. Previously, I graduated from Columbia's Fu Foundation School of Engineering with both an M.S. and a B.S. in Computer Science (Summa Cum Laude). I was fortunate to work under the supervision of Prof. Shih-Fu Chang at Columbia University and with Dr. Jim Fan, Prof. Yuke Zhu, and Prof. Anima Anandkumar at NVIDIA Research.

My research interests lie in the area of multimodal learning, foundation models, and embodied AI.

CV / Google Scholar / Twitter / Github

News

[2023.01] One paper was accepted by ICLR 2023.
[2022.11] MineDojo has won Outstanding Paper Award at NeurIPS 2022!
[2022.07] One paper was accepted by ECCV 2022.
[2022.06] Officially released MineDojo!
[2022.01] Join the Nvidia AI research group, supervised by Dr. Jim Fan, Prof. Yuke Zhu, and Prof. Anima Anandkumar.
[2021.09] Join the Digital Video and Multimedia (DVMM) Lab at Columbia University, supervised by Prof. Shih-Fu Chang

Research

(* indicates equal contribution)

TempCLR: Temporal Alignment Representation with Contrastive Learning
Yuncong Yang*, Jiawei Ma*, Shiyuan Huang, Long Chen, Xudong Lin, Guangxing Han, Shih-Fu Chang
International Conference on Learning Representations (ICLR), 2023
paper / code

We proposed TempCLR, a new contrastive learning framework that considers sequence-level temporal order consistency. The sequence-level framework improves multimodal representation learning in video understanding.

MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
Linxi Fan, Guanzhi Wang*, Yunfan Jiang*, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, Animashree Anandkumar
Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2022 (Outstanding Paper Award, Featured Paper Presentation)
project page / paper / code

We introduce MineDojo, a new framework based on the popular Minecraft game for building generally capable, open-ended embodied agents.

Few-Shot End-to-End Object Detection via Constantly Concentrated Encoding across Heads
Jiawei Ma, Guangxing Han, Shiyuan Huang, Yuncong Yang, Shih-Fu Chang
European Conference on Computer Vision (ECCV), 2022
paper / code comming soon

Misc Projects

	Adversarial Training with Support Dataset for Prototypical Network on Few Shot Learning COMS 6998 Security Robustness ML Systems, Fall 2021 paper
	Dynamic Grasping with Moving Obstacles COMS 6998 Topics in Robot Learning, Fall 2021 report

Teaching

Teaching Assistant: COMS 4732 Computer Vision (Spring 2022)

Thanks for the template from Jon Barron!