Ma, Xiaojian

43 publications

ICCV 2025 Embodied VideoAgent: Persistent Memory from Egocentric Videos and Embodied Sensors Enables Dynamic Scene Understanding Yue Fan, Xiaojian Ma, Rongpeng Su, Jun Guo, Rujie Wu, Xi Chen, Qing Li

ICML 2025 Falcon: Fast Visuomotor Policies via Partial Denoising Haojun Chen, Minghao Liu, Chengdong Ma, Xiaojian Ma, Zailin Ma, Huimin Wu, Yuanpei Chen, Yifan Zhong, Mingzhi Wang, Qing Li, Yaodong Yang

NeurIPS 2025 From Objects to Anywhere: A Holistic Benchmark for Multi-Level Visual Grounding in 3D Scenes Tianxu Wang, Zhuofan Zhang, Ziyu Zhu, Yue Fan, Jing Xiong, Pengxiang Li, Xiaojian Ma, Qing Li

ICLR 2025 GROOT-2: Weakly Supervised Multimodal Instruction Following Agents Shaofei Cai, Bowei Zhang, Zihao Wang, Haowei Lin, Xiaojian Ma, Anji Liu, Yitao Liang

NeurIPS 2025 Iterative Tool Usage Exploration for Multimodal Agents via Step-Wise Preference Tuning Pengxiang Li, Zhi Gao, Bofei Zhang, Yapeng Mi, Xiaojian Ma, Chenrui Shi, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, Qing Li

ICCV 2025 Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigation Ziyu Zhu, Xilin Wang, Yixuan Li, Zhuofan Zhang, Xiaojian Ma, Yixin Chen, Baoxiong Jia, Wei Liang, Qian Yu, Zhidong Deng, Siyuan Huang, Qing Li

ICLR 2025 Multi-Modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usage Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaojian Ma, Tao Yuan, Yue Fan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, Qing Li

ICLRW 2025 Multi-Modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usage Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaojian Ma, Tao Yuan, Yue Fan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, Qing Li

NeurIPS 2025 NEP: Autoregressive Image Editing via Next Editing Token Prediction Huimin Wu, Xiaojian Ma, Haozhe Zhao, Yanpeng Zhao, Qing Li

CVPR 2025 ROCKET-1: Mastering Open-World Interaction with Visual-Temporal Context Prompting Shaofei Cai, Zihao Wang, Kewei Lian, Zhancun Mu, Xiaojian Ma, Anji Liu, Yitao Liang

ICML 2024 An Embodied Generalist Agent in 3D World Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, Siyuan Huang

ICLRW 2024 An Embodied Generalist Agent in 3D World Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, Siyuan Huang

ICMLW 2024 An Embodied Generalist Agent in 3D World Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, Siyuan Huang

ICLR 2024 Bongard-OpenWorld: Few-Shot Reasoning for Free-Form Visual Concepts in the Real World Rujie Wu, Xiaojian Ma, Zhenliang Zhang, Wei Wang, Qing Li, Song-Chun Zhu, Yizhou Wang

CVPR 2024 CLOVA: A Closed-LOop Visual Assistant with Tool Usage and Update Zhi Gao, Yuntao Du, Xintong Zhang, Xiaojian Ma, Wenjuan Han, Song-Chun Zhu, Qing Li

ICMLW 2024 GROOT-1.5: Learning to Follow Multi-Modal Instructions from Weak Supervision Shaofei Cai, Bowei Zhang, Zihao Wang, Xiaojian Ma, Anji Liu, Yitao Liang

ICLR 2024 GROOT: Learning to Follow Instructions by Watching Gameplay Videos Shaofei Cai, Bowei Zhang, Zihao Wang, Xiaojian Ma, Anji Liu, Yitao Liang

ICLR 2024 MMICL: Empowering Vision-Language Model with Multi-Modal In-Context Learning Haozhe Zhao, Zefan Cai, Shuzheng Si, Xiaojian Ma, Kaikai An, Liang Chen, Zixuan Liu, Sheng Wang, Wenjuan Han, Baobao Chang

NeurIPS 2024 Multi-Modal Situated Reasoning in 3D Scenes Xiongkun Linghu, Jiangyong Huang, Xuesong Niu, Xiaojian Ma, Baoxiong Jia, Siyuan Huang

NeurIPS 2024 OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents Zihao Wang, Shaofei Cai, Zhancun Mu, Haowei Lin, Ceyao Zhang, Xuejie Liu, Qing Li, Anji Liu, Xiaojian Ma, Yitao Liang

ICMLW 2024 OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents Zihao Wang, Shaofei Cai, Zhancun Mu, Haowei Lin, Ceyao Zhang, Xuejie Liu, Qing Li, Anji Liu, Xiaojian Ma, Yitao Liang

NeurIPSW 2024 RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning and Verification in Long-Horizon Generation Zihao Wang, Anji Liu, Haowei Lin, Jiaqi Li, Xiaojian Ma, Yitao Liang

NeurIPSW 2024 ROCKET-1: Master Open-World Interaction with Visual-Temporal Context Prompting Shaofei Cai, Zihao Wang, Kewei Lian, Zhancun Mu, Xiaojian Ma, Anji Liu, Yitao Liang

NeurIPS 2024 UltraEdit: Instruction-Based Fine-Grained Image Editing at Scale Haozhe Zhao, Xiaojian Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, Baobao Chang

ECCV 2024 Unifying 3D Vision-Language Understanding via Promptable Queries Ziyu Zhu, Zhuofan Zhang, Xiaojian Ma, Xuesong Niu, Yixin Chen, Baoxiong Jia, Zhidong Deng, Siyuan Huang, Qing Li

ECCV 2024 VideoAgent: A Memory-Augmented Multimodal Agent for Video Understanding Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, Qing Li

ICCV 2023 3D-VisTA: Pre-Trained Transformer for 3D Vision and Text Alignment Ziyu Zhu, Xiaojian Ma, Yixin Chen, Zhidong Deng, Siyuan Huang, Qing Li

NeurIPS 2023 Describe, Explain, Plan and Select: Interactive Planning with LLMs Enables Open-World Multi-Task Agents Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma, Yitao Liang

NeurIPSW 2023 GROOT: Learning to Follow Instructions by Watching Gameplay Videos Shaofei Cai, Bowei Zhang, Zihao Wang, Xiaojian Ma, Anji Liu, Yitao Liang

NeurIPSW 2023 GROOT: Learning to Follow Instructions by Watching Gameplay Videos Shaofei Cai, Bowei Zhang, Zihao Wang, Xiaojian Ma, Anji Liu, Yitao Liang

NeurIPSW 2023 JARVIS-1: Open-World Multi-Task Agents with Memory-Augmented Multimodal Language Models Zihao Wang, Shaofei Cai, Anji Liu, Xiaojian Ma, Yitao Liang

NeurIPS 2023 Learning Energy-Based Prior Model with Diffusion-Amortized MCMC Peiyu Yu, Yaxuan Zhu, Sirui Xie, Xiaojian Ma, Ruiqi Gao, Song-Chun Zhu, Ying Nian Wu

CVPR 2023 Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction Shaofei Cai, Zihao Wang, Xiaojian Ma, Anji Liu, Yitao Liang

ICLR 2023 SQA3D: Situated Question Answering in 3D Scenes Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, Siyuan Huang

CVPR 2022 Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions Huaizu Jiang, Xiaojian Ma, Weili Nie, Zhiding Yu, Yuke Zhu, Anima Anandkumar

ICML 2022 Latent Diffusion Energy-Based Model for Interpretable Text Modelling Peiyu Yu, Sirui Xie, Xiaojian Ma, Baoxiong Jia, Bo Pang, Ruiqi Gao, Yixin Zhu, Song-Chun Zhu, Ying Nian Wu

ICLR 2022 RelViT: Concept-Guided Vision Transformer for Visual Relational Reasoning Xiaojian Ma, Weili Nie, Zhiding Yu, Huaizu Jiang, Chaowei Xiao, Yuke Zhu, Song-Chun Zhu, Anima Anandkumar

ICML 2021 Adversarial Option-Aware Hierarchical Imitation Learning Mingxuan Jing, Wenbing Huang, Fuchun Sun, Xiaojian Ma, Tao Kong, Chuang Gan, Lei Li

NeurIPS 2021 Unsupervised Foreground Extraction via Deep Region Competition Peiyu Yu, Sirui Xie, Xiaojian Ma, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu

AAAI 2020 Reinforcement Learning from Imperfect Demonstrations Under Soft Expert Guidance Mingxuan Jing, Xiaojian Ma, Wenbing Huang, Fuchun Sun, Chao Yang, Bin Fang, Huaping Liu

AAAI 2020 Theory-Based Causal Transfer: Integrating Instance-Level Induction and Abstract-Level Structure Learning Mark Edmonds, Xiaojian Ma, Siyuan Qi, Yixin Zhu, Hongjing Lu, Song-Chun Zhu

NeurIPS 2019 Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement Chao Yang, Xiaojian Ma, Wenbing Huang, Fuchun Sun, Huaping Liu, Junzhou Huang, Chuang Gan

AAAI 2019 Task Transfer by Preference-Based Cost Learning Mingxuan Jing, Xiaojian Ma, Wen-bing Huang, Fuchun Sun, Huaping Liu