Yuncong Yang

I am currently a first-year PhD student at UMass Amherst, where I am supervised by Prof. Chuang Gan and collaborate with Prof. Yilun Du. Previously, I graduated from Columbia's Fu Foundation School of Engineering with both an M.S. and a B.S. in Computer Science (Summa Cum Laude). I was fortunate to work under the supervision of Prof. Shih-Fu Chang at Columbia University and with Dr. Jim Fan, Prof. Yuke Zhu, and Prof. Anima Anandkumar at NVIDIA Research.

My research interests lie in the area of Spatial Intelligence, Embodied AI, and Multi-modal Foundation Models.

CV / Google Scholar / Twitter / Github

News

[2025.03] 3D-Mem is accepted by CVPR 2025
[2025.01] I will be interning at Microsoft Research this summer.
[2024.09] Starting my PhD Journey at UMass Amherst!
[2023.01] One paper was accepted by ICLR 2023.
[2022.11] MineDojo has won Outstanding Paper Award at NeurIPS 2022!
[2022.07] One paper was accepted by ECCV 2022.
[2022.06] Officially released MineDojo!
[2022.01] Join the Nvidia AI research group, supervised by Dr. Jim Fan, Prof. Yuke Zhu, and Prof. Anima Anandkumar.
[2021.09] Join the Digital Video and Multimedia (DVMM) Lab at Columbia University, supervised by Prof. Shih-Fu Chang

Research

(* indicates equal contribution)

	3D-Mem: 3D Scene Memory for Embodied Exploration and Reasoning Yuncong Yang, Han Yang, Jiachen Zhou, Peihao Chen, Hongxin Zhang, Yilun Du, Chuang Gan CVPR 2025 project page / paper / code / twitter We proposed 3D-Mem, a framework that serves as 3D scene memory to empower embodied agents with lifelong exploration and reasoning abilities in 3D environments.
	TempCLR: Temporal Alignment Representation with Contrastive Learning Yuncong Yang, Jiawei Ma, Shiyuan Huang, Long Chen, Xudong Lin, Guangxing Han, Shih-Fu Chang ICLR 2023 paper / code We proposed TempCLR, a new contrastive learning framework that considers sequence-level temporal order consistency in Long-Video Understanding.
	MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, Animashree Anandkumar NeurIPS 2022 Datasets and Benchmarks Track (Outstanding Paper Award, Featured Paper Presentation) project page / paper / code We introduce MineDojo, a new framework based on the popular Minecraft game for building generally capable, open-ended embodied agents.
	Few-Shot End-to-End Object Detection via Constantly Concentrated Encoding across Heads Jiawei Ma, Guangxing Han, Shiyuan Huang, Yuncong Yang, Shih-Fu Chang ECCV 2022 paper

Misc Projects

	Dynamic Grasping with Moving Obstacles COMS 6998 Topics in Robot Learning, Fall 2021 report
	Adversarial Training for Few-Shot Image Classifications COMS 6998 Security Robustness ML Systems, Fall 2021 paper

Teaching

Teaching Assistant: COMS 4732 Computer Vision II (Spring 2022)

Thanks for the template from Jon Barron!