Du, Yilun

102 publications

CVPR 2025 3D-Mem: 3D Scene Memory for Embodied Exploration and Reasoning Yuncong Yang, Han Yang, Jiachen Zhou, Peihao Chen, Hongxin Zhang, Yilun Du, Chuang Gan

ICML 2025 AdaWorld: Learning Adaptable World Models with Latent Actions Shenyuan Gao, Siyuan Zhou, Yilun Du, Jun Zhang, Chuang Gan

ICLR 2025 AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai, Enxin Song, Yilun Du, Chenlin Meng, Vashisht Madhavan, Omer Bar-Tal, Jenq-Neng Hwang, Saining Xie, Christopher D Manning

ICLR 2025 COMBO: Compositional World Models for Embodied Multi-Agent Cooperation Hongxin Zhang, Zeyuan Wang, Qiushi Lyu, Zheyuan Zhang, Sunli Chen, Tianmin Shu, Behzad Dariush, Kwonjoon Lee, Yilun Du, Chuang Gan

ICML 2025 Compositional Scene Understanding Through Inverse Generative Modeling Yanbo Wang, Justin Dauwels, Yilun Du

NeurIPS 2025 EvoLM: In Search of Lost Training Dynamics for Language Model Reasoning Zhenting Qi, Fan Nie, Alexandre Alahi, James Zou, Himabindu Lakkaraju, Yilun Du, Eric P. Xing, Sham M. Kakade, Hanlin Zhang

NeurIPS 2025 Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Models Louis Béthune, David Vigouroux, Yilun Du, Rufin VanRullen, Thomas Serre, Victor Boutin

NeurIPS 2025 Generalizable Reasoning Through Compositional Energy Minimization Alexandru Oarga, Yilun Du

NeurIPS 2025 Generative Trajectory Stitching Through Diffusion Composition Yunhao Luo, Utkarsh Aashu Mishra, Yilun Du, Danfei Xu

ICLR 2025 Grounding Video Models to Actions Through Goal Conditioned Exploration Yunhao Luo, Yilun Du

ICML 2025 History-Guided Video Diffusion Kiwhan Song, Boyuan Chen, Max Simchowitz, Yilun Du, Russ Tedrake, Vincent Sitzmann

NeurIPS 2025 Learning 3D Persistent Embodied World Models Siyuan Zhou, Yilun Du, Yuncong Yang, Lei Han, Peihao Chen, Dit-Yan Yeung, Chuang Gan

ICCV 2025 Learning 4D Embodied World Models Haoyu Zhen, Qiao Sun, Hongxin Zhang, Junyan Li, Siyuan Zhou, Yilun Du, Chuang Gan

ICLR 2025 Looped Transformers for Length Generalization Ying Fan, Yilun Du, Kannan Ramchandran, Kangwook Lee

NeurIPS 2025 MindJourney: Test-Time Scaling with World Models for Spatial Reasoning Yuncong Yang, Jiageng Liu, Zheyuan Zhang, Siyuan Zhou, Reuben Tan, Jianwei Yang, Yilun Du, Chuang Gan

ICLRW 2025 Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers (Abridged) Shalev Lifshitz, Sheila A. McIlraith, Yilun Du

ICLR 2025 Multiagent Finetuning: Self Improvement with Diverse Reasoning Chains Vighnesh Subramaniam, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba, Shuang Li, Igor Mordatch

ICLR 2025 Solving New Tasks by Adapting Internet Video Knowledge Calvin Luo, Zilai Zeng, Yilun Du, Chen Sun

ICLRW 2025 Solving Robotic Tasks via Self-Adapting Improvement Loops with Internet Video Knowledge Calvin Luo, Zilai Zeng, Yilun Du, Chen Sun

JAIR 2025 Towards Generalist Robot Learning from Internet Video: A Survey Robert McCarthy, Daniel Chee Hian Tan, Dominik Schmidt, Fernando Acero, Nathan Herr, Yilun Du, Thomas George Thuruthel, Zhibin Li

NeurIPS 2025 Towards Understanding Camera Motions in Any Video Zhiqiu Lin, Siyuan Cen, Daniel Jiang, Jay Karhade, Hewei Wang, Chancharik Mitra, Yu Tong Tiffany Ling, Yuhan Huang, Rushikesh Zawar, Xue Bai, Yilun Du, Chuang Gan, Deva Ramanan

NeurIPS 2025 Win Fast or Lose Slow: Balancing Speed and Accuracy in Latency-Sensitive Decisions of LLMs Hao Kang, Qingru Zhang, Han Cai, Weiyuan Xu, Tushar Krishna, Yilun Du, Tsachy Weissman

ICML 2024 3D-VLA: A 3D Vision-Language-Action Generative World Model Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du, Yining Hong, Chuang Gan

ICLR 2024 Building Cooperative Embodied Agents Modularly with Large Language Models Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B. Tenenbaum, Tianmin Shu, Chuang Gan

ICLR 2024 Compositional Generative Inverse Design Tailin Wu, Takashi Maruyama, Long Wei, Tao Zhang, Yilun Du, Gianluca Iaccarino, Jure Leskovec

ICML 2024 Compositional Image Decomposition with Diffusion Models Jocelin Su, Nan Liu, Yanbo Wang, Joshua B. Tenenbaum, Yilun Du

NeurIPS 2024 Diffusion Forcing: Next-Token Prediction Meets Full-Sequence Diffusion Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, Vincent Sitzmann

NeurIPS 2024 Few-Shot Task Learning Through Inverse Generative Modeling Aviv Netanyahu, Yilun Du, Antonia Bronars, Jyothish Pari, Joshua Tenenbaum, Tianmin Shu, Pulkit Agrawal

ICLR 2024 HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environments Qinhong Zhou, Sunli Chen, Yisong Wang, Haozhe Xu, Weihua Du, Hongxin Zhang, Yilun Du, Joshua B. Tenenbaum, Chuang Gan

ICML 2024 Improving Factuality and Reasoning in Language Models Through Multiagent Debate Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, Igor Mordatch

ECCV 2024 Large-Scale Reinforcement Learning for Diffusion Models Yinan Zhang, Eric Tzeng, Yilun Du, Dmitry Kislyuk

ICLR 2024 Learning Interactive Real-World Simulators Sherry Yang, Yilun Du, Seyed Kamyar Seyed Ghasemipour, Jonathan Tompson, Leslie Pack Kaelbling, Dale Schuurmans, Pieter Abbeel

ICML 2024 Learning Iterative Reasoning Through Energy Diffusion Yilun Du, Jiayuan Mao, Joshua B. Tenenbaum

ICLR 2024 Learning to Act from Actionless Videos Through Dense Correspondences Po-Chen Ko, Jiayuan Mao, Yilun Du, Shao-Hua Sun, Joshua B. Tenenbaum

ICLR 2024 Learning to Jointly Understand Visual and Tactile Signals Yichen Li, Yilun Du, Chao Liu, Chao Liu, Francis Williams, Michael Foshey, Benjamin Eckart, Jan Kautz, Joshua B. Tenenbaum, Antonio Torralba, Wojciech Matusik

NeurIPSW 2024 Looped Transformers for Length Generalization Ying Fan, Yilun Du, Kannan Ramchandran, Kangwook Lee

ICML 2024 Position: Compositional Generative Modeling: A Single Model Is Not All You Need Yilun Du, Leslie Pack Kaelbling

ICML 2024 Position: Video as the New Language for Real-World Decision Making Sherry Yang, Jacob C Walker, Jack Parker-Holder, Yilun Du, Jake Bruce, Andre Barreto, Pieter Abbeel, Dale Schuurmans

ICML 2024 Potential Based Diffusion Motion Planning Yunhao Luo, Chen Sun, Joshua B. Tenenbaum, Yilun Du

ICLR 2024 Probabilistic Adaptation of Black-Box Text-to-Video Models Sherry Yang, Yilun Du, Bo Dai, Dale Schuurmans, Joshua B. Tenenbaum, Pieter Abbeel

ICML 2024 RoboDreamer: Learning Compositional World Models for Robot Imagination Siyuan Zhou, Yilun Du, Jiaben Chen, Yandong Li, Dit-Yan Yeung, Chuang Gan

ICLR 2024 Training Diffusion Models with Reinforcement Learning Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, Sergey Levine

ICLR 2024 Video Language Planning Yilun Du, Sherry Yang, Pete Florence, Fei Xia, Ayzaan Wahid, Brian Ichter, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B. Tenenbaum, Leslie Pack Kaelbling, Andy Zeng, Jonathan Tompson

CVPRW 2024 Wake-Sleep Energy Based Models for Continual Learning Vaibhav Singh, Anna Choromanska, Shuang Li, Yilun Du

CVPR 2023 3D Concept Learning and Reasoning from Multi-View Images Yining Hong, Chunru Lin, Yilun Du, Zhenfang Chen, Joshua B. Tenenbaum, Chuang Gan

NeurIPS 2023 3D-LLM: Injecting the 3D World into Large Language Models Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, Chuang Gan

NeurIPS 2023 Adaptive Online Replanning with Diffusion Models Siyuan Zhou, Yilun Du, Shun Zhang, Mengdi Xu, Yikang Shen, Wei Xiao, Dit-Yan Yeung, Chuang Gan

NeurIPSW 2023 Building Cooperative Embodied Agents Modularly with Large Language Models Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua Tenenbaum, Tianmin Shu, Chuang Gan

ICLR 2023 Composing Ensembles of Pre-Trained Models via Iterative Consensus Shuang Li, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba, Igor Mordatch

CoRL 2023 Compositional Diffusion-Based Continuous Constraint Solvers Zhutian Yang, Jiayuan Mao, Yilun Du, Jiajun Wu, Joshua B. Tenenbaum, Tomás Lozano-Pérez, Leslie Pack Kaelbling

NeurIPS 2023 Compositional Foundation Models for Hierarchical Planning Anurag Ajay, Seungwook Han, Yilun Du, Shuang Li, Abhi Gupta, Tommi Jaakkola, Josh Tenenbaum, Leslie P. Kaelbling, Akash Srivastava, Pulkit Agrawal

NeurIPSW 2023 Compositional Foundation Models for Hierarchical Planning Anurag Ajay, Seungwook Han, Yilun Du, Shuang Li, Abhi Gupta, Tommi Jaakkola, Joshua Tenenbaum, Leslie Kaelbling, Akash Srivastava, Pulkit Agrawal

NeurIPSW 2023 Compositional Generative Inverse Design Tailin Wu, Takashi Maruyama, Long Wei, Tao Zhang, Yilun Du, Gianluca Iaccarino, Jure Leskovec

NeurIPS 2023 DiffuseBot: Breeding Soft Robots with Physics-Augmented Generative Diffusion Models Tsun-Hsuan Johnson Wang, Juntian Zheng, Pingchuan Ma, Yilun Du, Byungchul Kim, Andrew Spielberg, Josh Tenenbaum, Chuang Gan, Daniela Rus

NeurIPS 2023 FlowCam: Training Generalizable 3D Radiance Fields Without Camera Poses via Pixel-Aligned Scene Flow Cameron Smith, Yilun Du, Ayush Tewari, Vincent Sitzmann

ICML 2023 Inferring Relational Potentials in Interacting Systems Armand Comas, Yilun Du, Christian Fernandez Lopez, Sandesh Ghimire, Mario Sznaier, Joshua B. Tenenbaum, Octavia Camps

ICLR 2023 Is Conditional Generative Modeling All You Need for Decision Making? Anurag Ajay, Yilun Du, Abhi Gupta, Joshua B. Tenenbaum, Tommi S. Jaakkola, Pulkit Agrawal

NeurIPSW 2023 Learning Interactive Real-World Simulators Sherry Yang, Yilun Du, Seyed Kamyar Seyed Ghasemipour, Jonathan Tompson, Dale Schuurmans, Pieter Abbeel

NeurIPS 2023 Learning Universal Policies via Text-Guided Video Generation Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, Pieter Abbeel

CVPR 2023 Learning to Render Novel Views from Wide-Baseline Stereo Pairs Yilun Du, Cameron Smith, Ayush Tewari, Vincent Sitzmann

ICLR 2023 Neural Groundplans: Persistent Neural Scene Representations from a Single Image Prafull Sharma, Ayush Tewari, Yilun Du, Sergey Zakharov, Rares Andrei Ambrus, Adrien Gaidon, William T. Freeman, Fredo Durand, Joshua B. Tenenbaum, Vincent Sitzmann

ICLR 2023 Planning with Sequence Models Through Iterative Energy Minimization Hongyi Chen, Yilun Du, Yiye Chen, Joshua B. Tenenbaum, Patricio A. Vela

ICML 2023 Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC Yilun Du, Conor Durkan, Robin Strudel, Joshua B. Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, Will Sussman Grathwohl

NeurIPS 2023 Secure Out-of-Distribution Task Generalization with Energy-Based Models Shengzhuang Chen, Long-Kai Huang, Jonathan Richard Schwarz, Yilun Du, Ying Wei

ICMLW 2023 Training Diffusion Models with Reinforcement Learning Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, Sergey Levine

ICCV 2023 Unsupervised Compositional Concepts Discovery with Text-to-Image Generative Models Nan Liu, Yilun Du, Shuang Li, Joshua B. Tenenbaum, Antonio Torralba

NeurIPS 2022 3D Concept Grounding on Neural Fields Yining Hong, Yilun Du, Chunru Lin, Josh Tenenbaum, Chuang Gan

ECCV 2022 Compositional Visual Generation with Composable Diffusion Models Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, Joshua B. Tenenbaum

CoLLAs 2022 Energy-Based Models for Continual Learning Shuang Li, Yilun Du, Gido Ven, Igor Mordatch

NeurIPSW 2022 Is Conditional Generative Modeling All You Need for Decision-Making? Anurag Ajay, Yilun Du, Abhi Gupta, Joshua B. Tenenbaum, Tommi S. Jaakkola, Pulkit Agrawal

CVPR 2022 Kubric: A Scalable Dataset Generator Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J. Fleet, Dan Gnanapragasam, Florian Golemo, Charles Herrmann, Thomas Kipf, Abhijit Kundu, Dmitry Lagun, Issam Laradji, Hsueh-Ti Liu, Henning Meyer, Yishu Miao, Derek Nowrouzezahrai, Cengiz Oztireli, Etienne Pot, Noha Radwan, Daniel Rebain, Sara Sabour, Mehdi S. M. Sajjadi, Matan Sela, Vincent Sitzmann, Austin Stone, Deqing Sun, Suhani Vora, Ziyu Wang, Tianhao Wu, Kwang Moo Yi, Fangcheng Zhong, Andrea Tagliasacchi

ICML 2022 Learning Iterative Reasoning Through Energy Minimization Yilun Du, Shuang Li, Joshua Tenenbaum, Igor Mordatch

NeurIPS 2022 Learning Neural Acoustic Fields Andrew Luo, Yilun Du, Michael Tarr, Josh Tenenbaum, Antonio Torralba, Chuang Gan

CoRL 2022 MIRA: Mental Imagery for Robotic Affordances Yen-Chen Lin, Pete Florence, Andy Zeng, Jonathan T. Barron, Yilun Du, Wei-Chiu Ma, Anthony Simeonov, Alberto Rodriguez Garcia, Phillip Isola

ICML 2022 Planning with Diffusion for Flexible Behavior Synthesis Michael Janner, Yilun Du, Joshua Tenenbaum, Sergey Levine

NeurIPS 2022 Pre-Trained Language Models for Interactive Decision-Making Shuang Li, Xavier Puig, Chris Paxton, Yilun Du, Clinton Wang, Linxi Fan, Tao Chen, De-An Huang, Ekin Akyürek, Anima Anandkumar, Jacob Andreas, Igor Mordatch, Antonio Torralba, Yuke Zhu

CoRL 2022 SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields Anthony Simeonov, Yilun Du, Yen-Chen Lin, Alberto Rodriguez Garcia, Leslie Pack Kaelbling, Tomás Lozano-Pérez, Pulkit Agrawal

ICML 2022 Streaming Inference for Infinite Feature Models Rylan Schaeffer, Yilun Du, Gabrielle K Liu, Ila Fiete

CoLLAs 2022 Streaming Inference for Infinite Non-Stationary Clustering Rylan Schaeffer, Gabrielle Kaili-may Liu, Yilun Du, Scott Linderman, Ila R. Fiete

ICLRW 2022 Streaming Inference for Infinite Non-Stationary Clustering Rylan Schaeffer, Gabrielle Kaili-May Liu, Yilun Du, Scott Linderman, Ila R Fiete

ICCV 2021 3D Shape Generation and Completion Through Point-Voxel Diffusion Linqi Zhou, Yilun Du, Jiajun Wu

ICCV 2021 Curious Representation Learning for Embodied Intelligence Yilun Du, Chuang Gan, Phillip Isola

ICLRW 2021 Energy-Based Models for Continual Learning Shuang Li, Yilun Du, Gido Martijn van de Ven, Igor Mordatch

ICLRW 2021 Improved Contrastive Divergence Training of Energy Based Models Yilun Du, Shuang Li, Joshua B. Tenenbaum, Igor Mordatch

ICML 2021 Improved Contrastive Divergence Training of Energy-Based Models Yilun Du, Shuang Li, Joshua Tenenbaum, Igor Mordatch

NeurIPS 2021 Learning Signal-Agnostic Manifolds of Neural Fields Yilun Du, Katie Collins, Josh Tenenbaum, Vincent Sitzmann

NeurIPS 2021 Learning to Compose Visual Relations Nan Liu, Shuang Li, Yilun Du, Josh Tenenbaum, Antonio Torralba

ICCV 2021 Neural Radiance Flow for 4D View Synthesis and Video Processing Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2021 Unsupervised Discovery of 3D Physical Objects from Video Yilun Du, Kevin A. Smith, Tomer Ullman, Joshua B. Tenenbaum, Jiajun Wu

NeurIPS 2021 Unsupervised Learning of Compositional Energy Concepts Yilun Du, Shuang Li, Yash Sharma, Josh Tenenbaum, Igor Mordatch

ICCV 2021 Weakly Supervised Human-Object Interaction Detection in Video via Contrastive Spatiotemporal Regions Shuang Li, Yilun Du, Antonio Torralba, Josef Sivic, Bryan Russell

CoRL 2020 A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects Anthony Simeonov, Yilun Du, Beomjoon Kim, Francois Hogan, Joshua Tenenbaum, Pulkit Agrawal, Alberto Rodriguez

NeurIPS 2020 Compositional Visual Generation with Energy Based Models Yilun Du, Shuang Li, Igor Mordatch

ICLR 2020 Energy-Based Models for Atomic-Resolution Protein Conformations Yilun Du, Joshua Meier, Jerry Ma, Rob Fergus, Alexander Rives

ICLR 2020 Observational Overfitting in Reinforcement Learning Xingyou Song, Yiding Jiang, Stephen Tu, Yilun Du, Behnam Neyshabur

NeurIPS 2019 Implicit Generation and Modeling with Energy Based Models Yilun Du, Igor Mordatch

CoRL 2019 Model-Based Planning with Energy-Based Models Yilun Du, Toru Lin, Igor Mordatch

ICML 2019 Task-Agnostic Dynamics Priors for Deep Reinforcement Learning Yilun Du, Karthic Narasimhan

NeurIPS 2018 Learning to Exploit Stability for 3D Scene Parsing Yilun Du, Zhijian Liu, Hector Basevi, Ales Leonardis, Bill Freeman, Josh Tenenbaum, Jiajun Wu