Ji, Heng

62 publications

TMLR 2026 A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence Huan-ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Qihan Ren, Yiran Wu, Hongru Wang, Han Xiao, Yuhang Zhou, Shaokun Zhang, Jiayi Zhang, Jinyu Xiang, Yixiong Fang, Qiwen Zhao, Dongrui Liu, Cheng Qian, Zhenhailong Wang, Minda Hu, Huazheng Wang, Qingyun Wu, Heng Ji, Mengdi Wang

TMLR 2026 The Landscape of Agentic Reinforcement Learning for LLMs: A Survey Guibin Zhang, Hejia Geng, Xiaohang Yu, Zhenfei Yin, Zaibin Zhang, Zelin Tan, Heng Zhou, Zhong-Zhi Li, Xiangyuan Xue, Yijiang Li, Yifan Zhou, Yang Chen, Chen Zhang, Yutao Fan, Zihu Wang, Songtao Huang, Francisco Piedrahita Velez, Yue Liao, Hongru Wang, Mengyue Yang, Heng Ji, Jun Wang, Shuicheng Yan, Philip Torr, Lei Bai

FnTML 2025 Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems Xuan Zhang, Limei Wang, Jacob Helwig, Youzhi Luo, Cong Fu, Yaochen Xie, Meng Liu, Yuchao Lin, Zhao Xu, Keqiang Yan, Keir Adams, Maurice Weiler, Xiner Li, Tianfan Fu, Yucheng Wang, Alex Strasser, Haiyang Yu, Yuqing Xie, Xiang Fu, Shenglong Xu, Yi Liu, Yuanqi Du, Alexandra Saxton, Hongyi Ling, Hannah Lawrence, Hannes Stärk, Shurui Gui, Carl Edwards, Nicholas Gao, Adriana Ladera, Tailin Wu, Elyssa F. Hofgard, Aria Mansouri Tehrani, Rui Wang, Ameya Daigavane, Montgomery Bohde, Jerry Kurtin, Qian Huang, Tuong Phung, Minkai Xu, Chaitanya K. Joshi, Simon V. Mathis, Kamyar Azizzadenesheli, Ada Fang, Alán Aspuru-Guzik, Erik J. Bekkers, Michael M. Bronstein, Marinka Zitnik, Anima Anandkumar, Stefano Ermon, Pietro Liò, Rose Yu, Stephan Günnemann, Jure Leskovec, Heng Ji, Jimeng Sun, Regina Barzilay, Tommi S. Jaakkola, Connor W. Coley, Xiaoning Qian, Xiaofeng Qian, Tess E. Smidt, Shuiwang Ji

IJCAI 2025 Automating Intervention Discovery from Scientific Literature: A Progressive Ontology Prompting and Dual-LLM Framework Yuting Hu, Dancheng Liu, Qingyun Wang, Charles Yu, Chenhui Xu, Qingxiao Zheng, Heng Ji, Jinjun Xiong

ICLR 2025 CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking Tarun Suresh, Revanth Gangi Reddy, Yifei Xu, Zach Nussbaum, Andriy Mulyar, Brandon Duderstadt, Heng Ji

ICML 2025 Contrastive Visual Data Augmentation Yu Zhou, Bingxuan Li, Tang Mohan, Xiaomeng Jin, Te-Lin Wu, Kuan-Hao Huang, Heng Ji, Kai-Wei Chang, Nanyun Peng

NeurIPS 2025 DyMU: Dynamic Merging and Virtual Unmerging for Efficient Variable-Length VLMs Zhenhailong Wang, Senthil Purushwalkam, Caiming Xiong, Silvio Savarese, Heng Ji, Ran Xu

ICLR 2025 Eliminating Position Bias of Language Models: A Mechanistic Approach Ziqi Wang, Hanlin Zhang, Xiner Li, Kuan-Hao Huang, Chi Han, Shuiwang Ji, Sham M. Kakade, Hao Peng, Heng Ji

ICML 2025 EmbodiedBench: Comprehensive Benchmarking Multi-Modal Large Language Models for Vision-Driven Embodied Agents Rui Yang, Hanyang Chen, Junyu Zhang, Mark Zhao, Cheng Qian, Kangrui Wang, Qineng Wang, Teja Venkat Koripella, Marziyeh Movahedi, Manling Li, Heng Ji, Huan Zhang, Tong Zhang

NeurIPS 2025 Fire360: A Benchmark for Robust Perception and Episodic Memory in Degraded 360° Firefighting Video Aditi Tiwari, Farzaneh Masoud, Dac Trong Nguyen, Jill Kraft, Heng Ji, Klara Nahrstedt

ICLR 2025 Fragment and Geometry Aware Tokenization of Molecules for Structure-Based Drug Design Using Language Models Cong Fu, Xiner Li, Blake Olson, Heng Ji, Shuiwang Ji

ICML 2025 Geometry Informed Tokenization of Molecules for Language Model Generation Xiner Li, Limei Wang, Youzhi Luo, Carl Edwards, Shurui Gui, Yuchao Lin, Heng Ji, Shuiwang Ji

ICLRW 2025 MolCap-Arena: A Comprehensive Captioning Benchmark on Language-Enhanced Molecular Property Prediction Carl Edwards, Ziqing Lu, Ehsan Hajiramezanali, Tommaso Biancalani, Heng Ji, Gabriele Scalia

ICLR 2025 OpenHands: An Open Platform for AI Software Developers as Generalist Agents Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, Graham Neubig

NeurIPS 2025 PARTONOMY: Large Multimodal Models with Part-Level Visual Understanding Ansel Blume, Jeonghwan Kim, Hyeonjeong Ha, Elen Chatikyan, Xiaomeng Jin, Khanh Duy Nguyen, Nanyun Peng, Kai-Wei Chang, Derek Hoiem, Heng Ji

TMLR 2025 Scaling Laws for Predicting Downstream Performance in LLMs Yangyi Chen, Binxuan Huang, Yifan Gao, Zhengyang Wang, Jingfeng Yang, Heng Ji

CVPR 2025 Search and Detect: Training-Free Long Tail Object Detection via Web-Image Retrieval Mankeerat Sidhu, Hetarth Chopra, Ansel Blume, Jeonghwan Kim, Revanth Gangi Reddy, Heng Ji

ICML 2025 SyncMind: Measuring Agent Out-of-Sync Recovery in Collaborative Software Engineering Xuehang Guo, Xingyao Wang, Yangyi Chen, Sha Li, Chi Han, Manling Li, Heng Ji

NeurIPS 2025 ToolRL: Reward Is All Tool Learning Needs Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-Tür, Gokhan Tur, Heng Ji

TMLR 2025 Towards LifeSpan Cognitive Systems Yu Wang, Chi Han, Tongtong Wu, Xiaoxin He, Wangchunshu Zhou, Nafis Sadeq, Xiusi Chen, Zexue He, Wei Wang, Gholamreza Haffari, Heng Ji, Julian McAuley

ICML 2025 Training Software Engineering Agents and Verifiers with SWE-Gym Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, Yizhe Zhang

ICLRW 2025 Training Software Engineering Agents and Verifiers with SWE-Gym Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, Yizhe Zhang

TMLR 2025 Understanding Emergent In-Context Learning from a Kernel Regression Perspective Chi Han, Ziqi Wang, Han Zhao, Heng Ji

NeurIPS 2025 Variational Supervised Contrastive Learning Ziwen Wang, Jiajun Fan, Thao Nguyen, Heng Ji, Ge Liu

ICCV 2025 Verbalized Representation Learning for Interpretable Few-Shot Generalization Cheng-Fu Yang, Da Yin, Wenbo Hu, Heng Ji, Nanyun Peng, Bolei Zhou, Kai-Wei Chang

TMLR 2025 Visually Descriptive Language Model for Vector Graphics Reasoning Zhenhailong Wang, Joy Hsu, Xingyao Wang, Kuan-Hao Huang, Manling Li, Jiajun Wu, Heng Ji

TMLR 2024 A Single Transformer for Scalable Vision-Language Modeling Yangyi Chen, Xingyao Wang, Hao Peng, Heng Ji

ICML 2024 CHEMREASONER: Heuristic Search over a Large Language Model’s Knowledge Space Using Quantum-Chemical Feedback Henry W. Sprueill, Carl Edwards, Khushbu Agarwal, Mariefel V Olarte, Udishnu Sanyal, Conrad Johnston, Hongbin Liu, Heng Ji, Sutanay Choudhury

ICLR 2024 CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets Lifan Yuan, Yangyi Chen, Xingyao Wang, Yi Fung, Hao Peng, Heng Ji

CVPR 2024 DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, Ajay Divakaran

NeurIPSW 2024 Eliminating Position Bias of Language Models: A Mechanistic Approach Ziqi Wang, Hanlin Zhang, Xiner Li, Kuan-Hao Huang, Chi Han, Shuiwang Ji, Sham M. Kakade, Hao Peng, Heng Ji

ICLR 2024 Enabling Lanuguage Models to Implicitly Learn Self-Improvement Ziqi Wang, Le Hou, Tianjian Lu, Yuexin Wu, Yunxuan Li, Hongkun Yu, Heng Ji

ICML 2024 Executable Code Actions Elicit Better LLM Agents Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, Heng Ji

ICLRW 2024 Executable Code Actions Elicit Better LLM Agents Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, Heng Ji

ICLRW 2024 If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents Ke Yang, Jiateng Liu, John Wu, Chaoqi Yang, Yi Fung, Sha Li, Zixuan Huang, Xu Cao, Xingyao Wang, Heng Ji, ChengXiang Zhai

NeurIPSW 2024 Infogent: An Agent-Based Framework for Web Information Aggregation Revanth Gangi Reddy, Sagnik Mukherjee, Jeonghwan Kim, Zhenhailong Wang, Dilek Hakkani Tur, Heng Ji

NeurIPS 2024 Invariant Tokenization of Crystalline Materials for Language Model Enabled Generation Keqiang Yan, Xiner Li, Hongyi Ling, Kenna Ashen, Carl Edwards, Raymundo Arróyave, Marinka Zitnik, Heng Ji, Xiaofeng Qian, Xiaoning Qian, Shuiwang Ji

ICML 2024 Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF Under KL-Constraint Wei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, Tong Zhang

ICLR 2024 MINT: Evaluating LLMs in Multi-Turn Interaction with Tools and Language Feedback Xingyao Wang, Zihan Wang, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, Heng Ji

ICML 2024 Position: TrustLLM: Trustworthiness in Large Language Models Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Hanchi Sun, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bertie Vidgen, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric P. Xing, Furong Huang, Hao Liu, Heng Ji, Hongyi Wang, Huan Zhang, Huaxiu Yao, Manolis Kellis, Marinka Zitnik, Meng Jiang, Mohit Bansal, James Zou, Jian Pei, Jian Liu, Jianfeng Gao, Jiawei Han, Jieyu Zhao, Jiliang Tang, Jindong Wang, Joaquin Vanschoren, John Mitchell, Kai Shu, Kaidi Xu, Kai-Wei Chang, Lifang He, Lifu Huang, Michael Backes, Neil Zhenqiang Gong, Philip S. Yu, Pin-Yu Chen, Quanquan Gu, Ran Xu, Rex Ying, Shuiwang Ji, Suman Jana, Tianlong Chen, Tianming Liu, Tianyi Zhou, William Yang Wang, Xiang Li, Xiangliang Zhang, Xiao Wang, Xing Xie, Xun Chen, Xuyu Wang, Yan Liu, Yanfang Ye, Yinzhi Cao, Yong Chen, Yue Zhao

ICMLW 2024 Weak-to-Strong Extrapolation Expedites Alignment Chujie Zheng, Ziqi Wang, Heng Ji, Minlie Huang, Nanyun Peng

AAAI 2023 ADEPT: A DEbiasing PrompT Framework Ke Yang, Charles Yu, Yi Ren Fung, Manling Li, Heng Ji

ICLR 2023 Augmentation with Projection: Towards an Effective and Efficient Data Augmentation Paradigm for Distillation Ziqi Wang, Yuexin Wu, Frederick Liu, Daogao Liu, Le Hou, Hongkun Yu, Jing Li, Heng Ji

ICLR 2023 Learning to Decompose Visual Features with Latent Textual Prompts Feng Wang, Manling Li, Xudong Lin, Hairong Lv, Alex Schwing, Heng Ji

ICLR 2023 Logical Entity Representation in Knowledge-Graphs for Differentiable Rule Learning Chi Han, Qizheng He, Charles Yu, Xinya Du, Hanghang Tong, Heng Ji

NeurIPS 2023 Paxion: Patching Action Knowledge in Video-Language Foundation Models Zhenhailong Wang, Ansel Blume, Sha Li, Genglin Liu, Jaemin Cho, Zineng Tang, Mohit Bansal, Heng Ji

NeurIPS 2023 Revisiting Out-of-Distribution Robustness in NLP: Benchmarks, Analysis, and LLMs Evaluations Lifan Yuan, Yangyi Chen, Ganqu Cui, Hongcheng Gao, FangYuan Zou, Xingyi Cheng, Heng Ji, Zhiyuan Liu, Maosong Sun

AAAI 2023 SumREN: Summarizing Reported Speech About Events in News Revanth Gangi Reddy, Heba Elfardy, Hou Pong Chan, Kevin Small, Heng Ji

CVPR 2023 Towards Fast Adaptation of Pretrained Contrastive Models for Multi-Channel Video-Language Retrieval Xudong Lin, Simran Tiwari, Shiyuan Huang, Manling Li, Mike Zheng Shou, Heng Ji, Shih-Fu Chang

AAAI 2023 Video Event Extraction via Tracking Visual States of Arguments Guang Yang, Manling Li, Jiajie Zhang, Xudong Lin, Heng Ji, Shih-Fu Chang

NeurIPSW 2023 What a Scientific Language Model Knows and Doesn't Know About Chemistry Lawrence Zhao, Carl Edwards, Heng Ji

CVPR 2022 CLIP-Event: Connecting Text and Images with Event Structures Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang

ICLR 2022 Chemical-Reaction-Aware Molecule Representation Learning Hongwei Wang, Weijiang Li, Xiaomeng Jin, Kyunghyun Cho, Heng Ji, Jiawei Han, Martin D. Burke

ICLR 2022 Cross-Lingual Transfer with Class-Weighted Language-Invariant Representations Ruicheng Xian, Heng Ji, Han Zhao

NeurIPS 2022 Language Models with Image Descriptors Are Strong Few-Shot Video-Language Learners Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji

AAAI 2022 MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Grounding Revanth Gangi Reddy, Xilin Rui, Manling Li, Xudong Lin, Haoyang Wen, Jaemin Cho, Lifu Huang, Mohit Bansal, Avirup Sil, Shih-Fu Chang, Alexander G. Schwing, Heng Ji

AAAI 2022 Open Vocabulary Electroencephalography-to-Text Decoding and Zero-Shot Sentiment Classification Zhenhailong Wang, Heng Ji

CVPR 2021 Separating Skills and Concepts for Novel Visual Question Answering Spencer Whitehead, Hui Wu, Heng Ji, Rogerio Feris, Kate Saenko

NeurIPS 2021 Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method Yifan Chen, Qi Zeng, Heng Ji, Yun Yang

ECCV 2020 Learning to Learn Words from Visual Scenes Dídac Surís, Dave Epstein, Heng Ji, Shih-Fu Chang, Carl Vondrick

AAAI 2015 A Novel Neural Topic Model and Its Supervised Extension Ziqiang Cao, Sujian Li, Yang Liu, Wenjie Li, Heng Ji

IJCAI 2015 Constrained Information-Theoretic Tripartite Graph Clustering to Identify Semantically Similar Relations Chenguang Wang, Yangqiu Song, Dan Roth, Chi Wang, Jiawei Han, Heng Ji, Ming Zhang