Zhu, Yuke

77 publications

CoRL 2025 CASPER: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Models Huihan Liu, Rutav Shah, Shuijing Liu, Jack Pittenger, Mingyo Seo, Yuchen Cui, Yonatan Bisk, Roberto Martín-Martín, Yuke Zhu

CoRL 2025 Constraint-Preserving Data Generation for One-Shot Visuomotor Policy Generalization Kevin Lin, Varun Ragunath, Andrew McAlinden, Aaditya Prasad, Jimmy Wu, Yuke Zhu, Jeannette Bohg

CoRL 2025 DreamGen: Unlocking Generalization in Robot Learning Through Video World Models Joel Jang, Seonghyeon Ye, Zongyu Lin, Jiannan Xiang, Johan Bjorck, Yu Fang, Fengyuan Hu, Spencer Huang, Kaushil Kundalia, Yen-Chen Lin, Loïc Magne, Ajay Mandlekar, Avnish Narayan, You Liang Tan, Guanzhi Wang, Jing Wang, Qi Wang, Yinzhen Xu, Xiaohui Zeng, Kaiyuan Zheng, Ruijie Zheng, Ming-Yu Liu, Luke Zettlemoyer, Dieter Fox, Jan Kautz, Scott Reed, Yuke Zhu, Linxi Fan

ICLR 2025 Enhancing Document Understanding with Group Position Embedding: A Novel Approach to Incorporate Layout Information Yuke Zhu, Yue Zhang, Dongdong Liu, Chi Xie, Zihua Xiong, Bo Zheng, Sheng Guo

CoRL 2025 FLARE: Robot Learning with Implicit World Modeling Ruijie Zheng, Jing Wang, Scott Reed, Johan Bjorck, Yu Fang, Fengyuan Hu, Joel Jang, Kaushil Kundalia, Zongyu Lin, Loïc Magne, Avnish Narayan, You Liang Tan, Guanzhi Wang, Qi Wang, Jiannan Xiang, Yinzhen Xu, Seonghyeon Ye, Jan Kautz, Furong Huang, Yuke Zhu, Linxi Fan

CoRL 2025 GraspMolmo: Generalizable Task-Oriented Grasping via Large-Scale Synthetic Data Generation Abhay Deshpande, Yuquan Deng, Jordi Salvador, Arijit Ray, Winson Han, Jiafei Duan, Rose Hendrix, Yuke Zhu, Ranjay Krishna

ICCV 2025 LightCity: An Urban Dataset for Outdoor Inverse Rendering and Reconstruction Under Multi-Illumination Conditions Jingjing Wang, Qirui Hu, Chong Bao, Yuke Zhu, Hujun Bao, Zhaopeng Cui, Guofeng Zhang

ICLR 2025 LongVILA: Scaling Long-Context Visual Language Models for Long Videos Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, Yihui He, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu, Song Han

ICLR 2025 OmniKV: Dynamic Context Selection for Efficient Long-Context LLMs Jitai Hao, Yuke Zhu, Tian Wang, Jun Yu, Xin Xin, Bo Zheng, Zhaochun Ren, Sheng Guo

ICML 2025 One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation Zhendong Wang, Max Li, Ajay Mandlekar, Zhenjia Xu, Jiaojiao Fan, Yashraj Narang, Linxi Fan, Yuke Zhu, Yogesh Balaji, Mingyuan Zhou, Ming-Yu Liu, Yu Zeng

CoRL 2025 Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids Toru Lin, Kartik Sachdev, Linxi Fan, Jitendra Malik, Yuke Zhu

ICLRW 2025 VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making Jake Grigsby, Yuke Zhu, Michael S Ryoo, Juan Carlos Niebles

TMLR 2025 Wolf: Dense Video Captioning with a World Summarization Framework Boyi Li, Ligeng Zhu, Ran Tian, Shuhan Tan, Yuxiao Chen, Yao Lu, Yin Cui, Sushant Veer, Max Ehrlich, Jonah Philion, Xinshuo Weng, Fuzhao Xue, Linxi Fan, Yuke Zhu, Jan Kautz, Andrew Tao, Ming-Yu Liu, Sanja Fidler, Boris Ivanovic, Trevor Darrell, Jitendra Malik, Song Han, Marco Pavone

NeurIPS 2024 AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers Jake Grigsby, Justin Sasek, Samyak Parajuli, Daniel Adebi, Amy Zhang, Yuke Zhu

ICLR 2024 AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agents Jake Grigsby, Linxi Fan, Yuke Zhu

AAAI 2024 Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning Zizhao Wang, Caroline Wang, Xuesu Xiao, Yuke Zhu, Peter Stone

ICLR 2024 Eureka: Human-Level Reward Design via Coding Large Language Models Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, Anima Anandkumar

TMLR 2024 Granger Causal Interaction Skill Chains Caleb Chuck, Kevin Black, Aditya Arjun, Yuke Zhu, Scott Niekum

CoRL 2024 Harmon: Whole-Body Motion Generation of Humanoid Robots from Language Descriptions Zhenyu Jiang, Yuqi Xie, Jinhan Li, Ye Yuan, Yifeng Zhu, Yuke Zhu

CoRL 2024 Multi-Task Interactive Robot Fleet Learning with Visual World Models Huihan Liu, Yu Zhang, Vaarij Betala, Evan Zhang, James Liu, Crystal Ding, Yuke Zhu

CoRL 2024 OKAMI: Teaching Humanoid Robots Manipulation Skills Through Single Video Imitation Jinhan Li, Yifeng Zhu, Yuqi Xie, Zhenyu Jiang, Mingyo Seo, Georgios Pavlakos, Yuke Zhu

ICML 2024 PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs Soroush Nasiriany, Fei Xia, Wenhao Yu, Ted Xiao, Jacky Liang, Ishita Dasgupta, Annie Xie, Danny Driess, Ayzaan Wahid, Zhuo Xu, Quan Vuong, Tingnan Zhang, Tsang-Wei Edward Lee, Kuang-Huei Lee, Peng Xu, Sean Kirmani, Yuke Zhu, Andy Zeng, Karol Hausman, Nicolas Heess, Chelsea Finn, Sergey Levine, Brian Ichter

TMLR 2024 Voyager: An Open-Ended Embodied Agent with Large Language Models Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar

AAAI 2023 Building Compositional Robot Autonomy with Modularity and Abstraction Yuke Zhu

NeurIPS 2023 Cross-Episodic Curriculum for Transformer Agents Lucy Xiaoyang Shi, Yunfan Jiang, Jake Grigsby, Linxi Fan, Yuke Zhu

NeurIPSW 2023 Eureka: Human-Level Reward Design via Coding Large Language Models Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, Anima Anandkumar

NeurIPSW 2023 Eureka: Human-Level Reward Design via Coding Large Language Models Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, Anima Anandkumar

CVPR 2023 Fast Monocular Scene Reconstruction with Global-Sparse Local-Dense Grids Wei Dong, Christopher Choy, Charles Loop, Or Litany, Yuke Zhu, Anima Anandkumar

NeurIPS 2023 LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, Peter Stone

CoRL 2023 Learning Generalizable Manipulation Policies with Object-Centric 3D Representations Yifeng Zhu, Zhenyu Jiang, Peter Stone, Yuke Zhu

CoRL 2023 MUTEX: Learning Unified Policies from Multimodal Task Specifications Rutav Shah, Roberto Martín-Martín, Yuke Zhu

CoRL 2023 MimicGen: A Data Generation System for Scalable Robot Learning Using Human Demonstrations Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, Dieter Fox

CoRL 2023 MimicPlay: Long-Horizon Imitation Learning by Watching Human Play Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, Anima Anandkumar

NeurIPSW 2023 Target Rate Optimization: Avoiding Iterative Error Exploitation Braham Snyder, Amy Zhang, Yuke Zhu

ICML 2023 VIMA: Robot Manipulation with Multimodal Prompts Yunfan Jiang, Agrim Gupta, Zichen Zhang, Guanzhi Wang, Yongqiang Dou, Yanjun Chen, Li Fei-Fei, Anima Anandkumar, Yuke Zhu, Linxi Fan

NeurIPSW 2023 Voyager: An Open-Ended Embodied Agent with Large Language Models Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar

NeurIPSW 2023 Voyager: An Open-Ended Embodied Agent with Large Language Models Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar

NeurIPSW 2023 Voyager: An Open-Ended Embodied Agent with Large Language Models Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar

CVPR 2022 Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions Huaizu Jiang, Xiaojian Ma, Weili Nie, Zhiding Yu, Yuke Zhu, Anima Anandkumar

ICML 2022 Causal Dynamics Learning for Task-Independent State Abstraction Zizhao Wang, Xuesu Xiao, Zifan Xu, Yuke Zhu, Peter Stone

CVPR 2022 Coopernaut: End-to-End Driving with Cooperative Perception for Networked Vehicles Jiaxun Cui, Hang Qiu, Dian Chen, Peter Stone, Yuke Zhu

CVPR 2022 Ditto: Building Digital Twins of Articulated Objects from Interaction Zhenyu Jiang, Cheng-Chun Hsu, Yuke Zhu

CoRL 2022 Learning and Retrieval from Prior Data for Skill-Based Imitation Learning Soroush Nasiriany, Tian Gao, Ajay Mandlekar, Yuke Zhu

NeurIPS 2022 MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, Anima Anandkumar

NeurIPS 2022 Pre-Trained Language Models for Interactive Decision-Making Shuang Li, Xavier Puig, Chris Paxton, Yilun Du, Clinton Wang, Linxi Fan, Tao Chen, De-An Huang, Ekin Akyürek, Anima Anandkumar, Jacob Andreas, Igor Mordatch, Antonio Torralba, Yuke Zhu

NeurIPSW 2022 Raisin: Residual Algorithms for Versatile Offline Reinforcement Learning Braham Snyder, Yuke Zhu

ICLR 2022 RelViT: Concept-Guided Vision Transformer for Visual Relational Reasoning Xiaojian Ma, Weili Nie, Zhiding Yu, Huaizu Jiang, Chaowei Xiao, Yuke Zhu, Song-Chun Zhu, Anima Anandkumar

NeurIPSW 2022 VIMA: General Robot Manipulation with Multimodal Prompts Yunfan Jiang, Agrim Gupta, Zichen Zhang, Guanzhi Wang, Yongqiang Dou, Yanjun Chen, Li Fei-Fei, Anima Anandkumar, Yuke Zhu, Linxi Fan

CoRL 2022 VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors Yifeng Zhu, Abhishek Joshi, Peter Stone, Yuke Zhu

ICLR 2021 Adaptive Procedural Task Generation for Hard-Exploration Problems Kuan Fang, Yuke Zhu, Silvio Savarese, L. Fei-Fei

CoRL 2021 Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularization Youngwoon Lee, Joseph J Lim, Anima Anandkumar, Yuke Zhu

NeurIPSW 2021 Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks Soroush Nasiriany, Huihan Liu, Yuke Zhu

ICML 2021 Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition Bo Liu, Qiang Liu, Peter Stone, Animesh Garg, Yuke Zhu, Anima Anandkumar

ICCV 2021 DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision Shiyi Lan, Zhiding Yu, Christopher Choy, Subhashree Radhakrishnan, Guilin Liu, Yuke Zhu, Larry S. Davis, Anima Anandkumar

CVPR 2021 Dynamic Metric Learning: Towards a Scalable Metric Space to Accommodate Multiple Semantic Scales Yifan Sun, Yuke Zhu, Yuhan Zhang, Pengkun Zheng, Xi Qiu, Chi Zhang, Yichen Wei

ICML 2021 SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies Linxi Fan, Guanzhi Wang, De-An Huang, Zhiding Yu, Li Fei-Fei, Yuke Zhu, Animashree Anandkumar

ICML 2021 Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learning Anuj Mahajan, Mikayel Samvelyan, Lei Mao, Viktor Makoviychuk, Animesh Garg, Jean Kossaifi, Shimon Whiteson, Yuke Zhu, Animashree Anandkumar

CoRL 2021 What Matters in Learning from Offline Human Demonstrations for Robot Manipulation Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, Roberto Martín-Martín

NeurIPS 2020 Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning Weili Nie, Zhiding Yu, Lei Mao, Ankit B Patel, Yuke Zhu, Anima Anandkumar

IJCAI 2020 DualSMC: Tunneling Differentiable Filtering and Planning Under Continuous POMDPs Yunbo Wang, Bo Liu, Jiajun Wu, Yuke Zhu, Simon S. Du, Li Fei-Fei, Joshua B. Tenenbaum

CoRL 2020 Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion Xingye Da, Zhaoming Xie, David Hoeller, Byron Boots, Anima Anandkumar, Yuke Zhu, Buck Babich, Animesh Garg

UAI 2020 OCEAN: Online Task Inference for Compositional Tasks with Context Adaptation Hongyu Ren, Yuke Zhu, Jure Leskovec, Animashree Anandkumar, Animesh Garg

ECCV 2020 RubiksNet: Learnable 3D-Shift for Efficient Video Action Recognition Linxi Fan, Shyamal Buch, Guanzhi Wang, Ryan Cao, Yuke Zhu, Juan Carlos Niebles, Li Fei-Fei

ECCV 2020 Spherical Feature Transform for Deep Metric Learning Yuke Zhu, Yan Bai, Yichen Wei

CoRL 2019 Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation Kuan Fang, Yuke Zhu, Animesh Garg, Silvio Savarese, Li Fei-Fei

NeurIPS 2019 Regression Planning Networks Danfei Xu, Roberto Martín-Martín, De-An Huang, Yuke Zhu, Silvio Savarese, Li F Fei-Fei

CoRL 2018 ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning Through Imitation Ajay Mandlekar, Yuke Zhu, Animesh Garg, Jonathan Booher, Max Spero, Albert Tung, Julian Gao, John Emmons, Anchit Gupta, Emre Orbay, Silvio Savarese, Li Fei-Fei

CoRL 2018 SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark Linxi Fan, Yuke Zhu, Jiren Zhu, Zihua Liu, Orien Zeng, Anchit Gupta, Joan Creus-Costa, Silvio Savarese, Li Fei-Fei

CVPR 2017 Knowledge Acquisition for Visual Question Answering via Iterative Querying Yuke Zhu, Joseph J. Lim, Li Fei-Fei

CVPR 2017 Scene Graph Generation by Iterative Message Passing Danfei Xu, Yuke Zhu, Christopher B. Choy, Li Fei-Fei

ICCV 2017 Visual Semantic Planning Using Deep Successor Representations Yuke Zhu, Daniel Gordon, Eric Kolve, Dieter Fox, Li Fei-Fei, Abhinav Gupta, Roozbeh Mottaghi, Ali Farhadi

CVPR 2016 Visual7W: Grounded Question Answering in Images Yuke Zhu, Oliver Groth, Michael Bernstein, Li Fei-Fei

ICCV 2015 Action Recognition by Hierarchical Mid-Level Action Elements Tian Lan, Yuke Zhu, Amir Roshan Zamir, Silvio Savarese

MLJ 2014 Modelling Relational Statistics with Bayes Nets Oliver Schulte, Hassan Khosravi, Arthur E. Kirkpatrick, Tianxiang Gao, Yuke Zhu

ECCV 2014 Reasoning About Object Affordances in a Knowledge Base Representation Yuke Zhu, Alireza Fathi, Li Fei-Fei

AAAI 2014 StrokeBank: Automating Personalized Chinese Handwriting Generation Alfred Zong, Yuke Zhu

AAAI 2013 Graphical Model-Based Learning in High Dimensional Feature Spaces Zhao Song, Yuke Zhu