Huang, Siyuan

80 publications

ICLR 2025 Building Interactable Replicas of Complex Articulated Objects via Gaussian Splatting Yu Liu, Baoxiong Jia, Ruijie Lu, Junfeng Ni, Song-Chun Zhu, Siyuan Huang

CoRL 2025 CLONE: Closed-Loop Whole-Body Humanoid Teleoperation for Long-Horizon Tasks Yixuan Li, Yutang Lin, Jieming Cui, Tengyu Liu, Wei Liang, Yixin Zhu, Siyuan Huang

ICLRW 2025 ControlManip: Few-Shot Manipulation Fine-Tuning via Object-Centric Conditional Control Puhao Li, Yingying Wu, Wanlin Li, Yuzhe Huang, Zhiyuan Zhang, Yinghan Chen, Song-Chun Zhu, Tengyu Liu, Siyuan Huang

CoRL 2025 ControlVLA: Few-Shot Object-Centric Adaptation for Pre-Trained Vision-Language-Action Models Puhao Li, Yingying Wu, Ziheng Xi, Wanlin Li, Yuzhe Huang, Zhiyuan Zhang, Yinghan Chen, Jianan Wang, Song-Chun Zhu, Tengyu Liu, Siyuan Huang

CVPRW 2025 Cross-Spectral Body Recognition with Side Information Embedding: Benchmarks on LLCM and Analyzing Range-Induced Occlusions on IJB-MDF Anirudh Nanduri, Siyuan Huang, Rama Chellappa

CVPR 2025 Decompositional Neural Scene Reconstruction with Generative Diffusion Prior Junfeng Ni, Yu Liu, Ruijie Lu, Zirui Zhou, Song-Chun Zhu, Yixin Chen, Siyuan Huang

ICLR 2025 Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want Weifeng Lin, Xinyu Wei, Ruichuan An, Peng Gao, Bocheng Zou, Yulin Luo, Siyuan Huang, Shanghang Zhang, Hongsheng Li

CVPR 2025 Dynamic Motion Blending for Versatile Motion Editing Nan Jiang, Hongjie Li, Ziye Yuan, Zimo He, Yixin Chen, Tengyu Liu, Yixin Zhu, Siyuan Huang

NeurIPS 2025 EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation Siyuan Huang, Liliang Chen, Pengfei Zhou, Shengcong Chen, Yue Liao, Zhengkai Jiang, Yue Hu, Peng Gao, Hongsheng Li, Maoqing Yao, Guanghui Ren

CVPR 2025 GROVE: A Generalized Reward for Learning Open-Vocabulary Physical Skill Jieming Cui, Tengyu Liu, Ziyu Meng, Jiale Yu, Ran Song, Wei Zhang, Yixin Zhu, Siyuan Huang

ICCV 2025 GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devices Quanfeng Lu, Wenqi Shao, Zitao Liu, Lingxiao Du, Fanqing Meng, Boxuan Li, Botong Chen, Siyuan Huang, Kaipeng Zhang, Ping Luo

ICCV 2025 GWM: Towards Scalable Gaussian World Models for Robotic Manipulation Guanxing Lu, Baoxiong Jia, Puhao Li, Yixin Chen, Ziwei Wang, Yansong Tang, Siyuan Huang

CVPR 2025 InteractAnything: Zero-Shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing Jinlu Zhang, Yixin Chen, Zan Wang, Jie Yang, Yizhou Wang, Siyuan Huang

CoRL 2025 Learning a Unified Policy for Position and Force Control in Legged Loco-Manipulation Peiyuan Zhi, Peiyang Li, Jianqin Yin, Baoxiong Jia, Siyuan Huang

CVPR 2025 METASCENES: Towards Automated Replica Creation for Real-World 3D Scans Huangyue Yu, Baoxiong Jia, Yixin Chen, Yandan Yang, Puhao Li, Rongpeng Su, Jiaxin Li, Qing Li, Wei Liang, Song-Chun Zhu, Tengyu Liu, Siyuan Huang

CVPR 2025 MOVIS: Enhancing Multi-Object Novel View Synthesis for Indoor Scenes Ruijie Lu, Yixin Chen, Junfeng Ni, Baoxiong Jia, Yu Liu, Diwen Wan, Gang Zeng, Siyuan Huang

CVPR 2025 ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning Kailin Li, Puhao Li, Tengyu Liu, Yuyang Li, Siyuan Huang

CVPR 2025 Masked Point-Entity Contrast for Open-Vocabulary 3D Scene Understanding Yan Wang, Baoxiong Jia, Ziyu Zhu, Siyuan Huang

ICCV 2025 Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigation Ziyu Zhu, Xilin Wang, Yixuan Li, Zhuofan Zhang, Xiaojian Ma, Yixin Chen, Baoxiong Jia, Wei Liang, Qian Yu, Zhidong Deng, Siyuan Huang, Qing Li

CVPR 2025 Object-Centric Prompt-Driven Vision-Language-Action Model for Robotic Manipulation Xiaoqi Li, Jingyun Xu, Mingxu Zhang, Jiaming Liu, Yan Shen, Iaroslav Ponomarenko, Jiahui Xu, Liang Heng, Siyuan Huang, Shanghang Zhang, Hao Dong

ICLR 2025 PixWizard: Versatile Image-to-Image Visual Assistant with Open-Language Instructions Weifeng Lin, Xinyu Wei, Renrui Zhang, Le Zhuo, Shitian Zhao, Siyuan Huang, Junlin Xie, Peng Gao, Hongsheng Li

ICCV 2025 PrimHOI: Compositional Human-Object Interaction via Reusable Primitives Kai Jia, Tengyu Liu, Mingtao Pei, Yixin Zhu, Siyuan Huang

NeurIPS 2025 RoboCerebra: A Large-Scale Benchmark for Long-Horizon Robotic Manipulation Evaluation Songhao Han, Boxiang Qiu, Yue Liao, Siyuan Huang, Chen Gao, Shuicheng Yan, Si Liu

NeurIPS 2025 Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decoding Yiming Wang, Pei Zhang, Siyuan Huang, Baosong Yang, Zhuosheng Zhang, Fei Huang, Rui Wang

NeurIPS 2025 SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agent Yandan Yang, Baoxiong Jia, Shujie Zhang, Siyuan Huang

ICCV 2025 TACO: Taming Diffusion for In-the-Wild Video Amodal Completion Ruijie Lu, Yixin Chen, Yu Liu, Jiaxiang Tang, Junfeng Ni, Diwen Wan, Gang Zeng, Siyuan Huang

NeurIPS 2025 Taccel: Scaling up Vision-Based Tactile Robotics via High-Performance GPU Simulation Yuyang Li, Wenxin Du, Chang Yu, Puhao Li, Zihang Zhao, Tengyu Liu, Chenfanfu Jiang, Yixin Zhu, Siyuan Huang

ICCV 2025 Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracing Hongyu Shen, Junfeng Ni, Yixin Chen, Weishuo Li, Mingtao Pei, Siyuan Huang

CVPR 2025 Unveiling the Mist over 3D Vision-Language Understanding: Object-Centric Evaluation with Chain-of-Analysis Jiangyong Huang, Baoxiong Jia, Yan Wang, Ziyu Zhu, Xiongkun Linghu, Qing Li, Song-Chun Zhu, Siyuan Huang

WACV 2025 VILLS : Video-Image Learning to Learn Semantics for Person Re-Identification Siyuan Huang, Ram Prabhakar Kathirvel, Yuxiang Guo, Rama Chellappa, Cheng Peng

IJCAI 2024 3D Vision and Language Pretraining with Large-Scale Synthetic Data Dejie Yang, Zhu Xu, Wentao Mo, Qingchao Chen, Siyuan Huang, Yang Liu

CoRL 2024 A3VLM: Actionable Articulation-Aware Vision Language Model Siyuan Huang, Haonan Chang, Yuhan Liu, Yimeng Zhu, Hao Dong, Abdeslam Boularias, Peng Gao, Hongsheng Li

ICML 2024 An Embodied Generalist Agent in 3D World Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, Siyuan Huang

ICLRW 2024 An Embodied Generalist Agent in 3D World Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, Siyuan Huang

ICMLW 2024 An Embodied Generalist Agent in 3D World Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, Siyuan Huang

CVPR 2024 AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agents Jieming Cui, Tengyu Liu, Nian Liu, Yaodong Yang, Yixin Zhu, Siyuan Huang

NeurIPS 2024 Cluster-Wise Graph Transformer with Dual-Granularity Kernelized Attention Siyuan Huang, Yunchong Song, Jiayue Zhou, Zhouhan Lin

ECCV 2024 F-HOI: Toward Fine-Grained Semantic-Aligned 3D Human-Object Interactions Jie Yang, Xuesong Niu, Nan Jiang, Ruimao Zhang, Siyuan Huang

ICLR 2024 Graph Parsing Networks Yunchong Song, Siyuan Huang, Xinbing Wang, Chenghu Zhou, Zhouhan Lin

CVPR 2024 Move as You Say Interact as You Can: Language-Guided Human Motion Generation with Scene Affordance Zan Wang, Yixin Chen, Baoxiong Jia, Puhao Li, Jinlu Zhang, Jingze Zhang, Tengyu Liu, Yixin Zhu, Wei Liang, Siyuan Huang

NeurIPS 2024 Multi-Modal Situated Reasoning in 3D Scenes Xiongkun Linghu, Jiangyong Huang, Xuesong Niu, Xiaojian Ma, Baoxiong Jia, Siyuan Huang

ICLR 2024 Neural-Symbolic Recursive Machine for Systematic Generalization Qing Li, Yixin Zhu, Yitao Liang, Ying Nian Wu, Song-Chun Zhu, Siyuan Huang

NeurIPS 2024 PhyRecon: Physically Plausible Neural Scene Reconstruction Junfeng Ni, Yixin Chen, Bohan Jing, Nan Jiang, Bin Wang, Bo Dai, Puhao Li, Yixin Zhu, Song-Chun Zhu, Siyuan Huang

CVPR 2024 PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI Yandan Yang, Baoxiong Jia, Peiyuan Zhi, Siyuan Huang

ICML 2024 SPHINX-X: Scaling Data and Parameters for a Family of Multi-Modal Large Language Models Dongyang Liu, Renrui Zhang, Longtian Qiu, Siyuan Huang, Weifeng Lin, Shitian Zhao, Shijie Geng, Ziyi Lin, Peng Jin, Kaipeng Zhang, Wenqi Shao, Chao Xu, Conghui He, Junjun He, Hao Shao, Pan Lu, Yu Qiao, Hongsheng Li, Peng Gao

ECCV 2024 SPHINX: A Mixer of Weights, Visual Embeddings and Image Scales for Multi-Modal Large Language Models Ziyi Lin, Dongyang Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Wenqi Shao, Keqin Chen, Jiaming Han, Siyuan Huang, Yichi Zhang, Xuming He, Yu Qiao, Hongsheng Li

CVPR 2024 Scaling up Dynamic Human-Scene Interaction Modeling Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma, Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, Siyuan Huang

ECCV 2024 SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding Baoxiong Jia, Yixin Chen, Huangyue Yu, Yan Wang, Xuesong Niu, Tengyu Liu, Qing Li, Siyuan Huang

ECCV 2024 SlotLifter: Slot-Guided Feature Lifting for Learning Object-Centric Radiance Fields Yu Liu, Baoxiong Jia, Yixin Chen, Siyuan Huang

ECCV 2024 Unifying 3D Vision-Language Understanding via Promptable Queries Ziyu Zhu, Zhuofan Zhang, Xiaojian Ma, Xuesong Niu, Yixin Chen, Baoxiong Jia, Zhidong Deng, Siyuan Huang, Qing Li

ICCV 2023 3D-VisTA: Pre-Trained Transformer for 3D Vision and Text Alignment Ziyu Zhu, Xiaojian Ma, Yixin Chen, Zhidong Deng, Siyuan Huang, Qing Li

ICLR 2023 A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics Qing Li, Siyuan Huang, Yining Hong, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu

ICCV 2023 ARNOLD: A Benchmark for Language-Grounded Task Learning with Continuous States in Realistic 3D Scenes Ran Gong, Jiangyong Huang, Yizhou Zhao, Haoran Geng, Xiaofeng Gao, Qingyang Wu, Wensi Ai, Ziheng Zhou, Demetri Terzopoulos, Song-Chun Zhu, Baoxiong Jia, Siyuan Huang

CVPR 2023 Diffusion-Based Generation, Optimization, and Planning in 3D Scenes Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, Song-Chun Zhu

ICCV 2023 Full-Body Articulated Human-Object Interaction Nan Jiang, Tengyu Liu, Zhexuan Cao, Jieming Cui, Zhiyuan Zhang, Yixin Chen, He Wang, Yixin Zhu, Siyuan Huang

CVPR 2023 GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Parts Haoran Geng, Helin Xu, Chengyang Zhao, Chao Xu, Li Yi, Siyuan Huang, He Wang

ICLR 2023 Improving Object-Centric Learning with Query Optimization Baoxiong Jia, Yu Liu, Siyuan Huang

NeurIPS 2023 ProBio: A Protocol-Guided Multimodal Dataset for Molecular Biology Lab Jieming Cui, Ziren Gong, Baoxiong Jia, Siyuan Huang, Zilong Zheng, Jianzhu Ma, Yixin Zhu

CVPR 2023 Prompt, Generate, Then Cache: Cascade of Foundation Models Makes Strong Few-Shot Learners Renrui Zhang, Xiangfei Hu, Bohao Li, Siyuan Huang, Hanqiu Deng, Yu Qiao, Peng Gao, Hongsheng Li

ICLR 2023 SQA3D: Situated Question Answering in 3D Scenes Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, Siyuan Huang

NeurIPS 2023 Tailoring Self-Attention for Graph via Rooted Subtrees Siyuan Huang, Yunchong Song, Jiayue Zhou, Zhouhan Lin

CVPR 2022 Adversarial Texture for Fooling Person Detectors in the Physical World Zhanhao Hu, Siyuan Huang, Xiaopei Zhu, Fuchun Sun, Bo Zhang, Xiaolin Hu

NeurIPS 2022 EgoTaskQA: Understanding Human Tasks in Egocentric Videos Baoxiong Jia, Ting Lei, Song-Chun Zhu, Siyuan Huang

NeurIPS 2022 HUMANISE: Language-Conditioned Human Motion Generation in 3D Scenes Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, Wei Liang, Siyuan Huang

CVPR 2022 Infrared Invisible Clothing: Hiding from Infrared Detectors at Multiple Angles in Real World Xiaopei Zhu, Zhanhao Hu, Siyuan Huang, Jianmin Li, Xiaolin Hu

AAAI 2022 Learning V1 Simple Cells with Vector Representation of Local Content and Matrix Representation of Local Motion Ruiqi Gao, Jianwen Xie, Siyuan Huang, Yufan Ren, Song-Chun Zhu, Ying Nian Wu

CVPR 2021 Learning Neural Representation of Camera Pose with Matrix Representation of Pose Shift via View Synthesis Yaxuan Zhu, Ruiqi Gao, Siyuan Huang, Song-Chun Zhu, Ying Nian Wu

AAAI 2021 Learning by Fixing: Solving Math Word Problems with Weak Supervision Yining Hong, Qing Li, Daniel Ciao, Siyuan Huang, Song-Chun Zhu

AAAI 2021 SMART: A Situation Model for Algebra Story Problems via Attributed Grammar Yining Hong, Qing Li, Ran Gong, Daniel Ciao, Siyuan Huang, Song-Chun Zhu

ICCV 2021 Spatio-Temporal Self-Supervised Representation Learning for 3D Point Clouds Siyuan Huang, Yichen Xie, Song-Chun Zhu, Yixin Zhu

ICCV 2021 VLGrammar: Grounded Grammar Induction of Vision and Language Yining Hong, Qing Li, Song-Chun Zhu, Siyuan Huang

ICCV 2021 YouRefIt: Embodied Reference Understanding with Language and Gesture Yixin Chen, Qing Li, Deqian Kong, Yik Lun Kei, Song-Chun Zhu, Tao Gao, Yixin Zhu, Siyuan Huang

ECCV 2020 A Competence-Aware Curriculum for Visual Concepts Learning via Question Answering Qing Li, Siyuan Huang, Yining Hong, Song-Chun Zhu

ICML 2020 Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning Qing Li, Siyuan Huang, Yining Hong, Yixin Chen, Ying Nian Wu, Song-Chun Zhu

ECCV 2020 LEMMA: A Multi-View Dataset for LEarning Multi-Agent Multi-Task Activities Baoxiong Jia, Yixin Chen, Siyuan Huang, Yixin Zhu, Song-Chun Zhu

AAAI 2020 Streaming Batch Gradient Tracking for Neural Network Training (Student Abstract) Siyuan Huang, Brian D. Hoskins, Matthew W. Daniels, Mark D. Stiles, Gina C. Adam

NeurIPS 2019 PerspectiveNet: 3D Object Detection from a Single RGB Image via Perspective Points Siyuan Huang, Yixin Chen, Tao Yuan, Siyuan Qi, Yixin Zhu, Song-Chun Zhu

NeurIPS 2018 Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimation Siyuan Huang, Siyuan Qi, Yinxue Xiao, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu

ECCV 2018 Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image Siyuan Huang, Siyuan Qi, Yixin Zhu, Yinxue Xiao, Yuanlu Xu, Song-Chun Zhu

ICCV 2017 Predicting Human Activities Using Stochastic Grammar Siyuan Qi, Siyuan Huang, Ping Wei, Song-Chun Zhu