Gao, Jianfeng

132 publications

CVPRW 2025 An Interactive Agent Foundation Model Zane Durante, Ran Gong, Bidipta Sarkar, Naoki Wake, Rohan Taori, Paul Tang, Shrinidhi Kowshika Lakshmikanth, Kevin A. Schulman, Arnold Milstein, Hoi Vo, Ehsan Adeli, Demetri Terzopoulos, Li Fei-Fei, Jianfeng Gao

ICML 2025 CollabLLM: From Passive Responders to Active Collaborators Shirley Wu, Michel Galley, Baolin Peng, Hao Cheng, Gavin Li, Yao Dou, Weixin Cai, James Zou, Jure Leskovec, Jianfeng Gao

ICLR 2025 DataGen: Unified Synthetic Dataset Generation via Large Language Models Yue Huang, Siyuan Wu, Chujie Gao, Dongping Chen, Qihui Zhang, Yao Wan, Tianyi Zhou, Chaowei Xiao, Jianfeng Gao, Lichao Sun, Xiangliang Zhang

NeurIPS 2025 Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation Liliang Ren, Congcong Chen, Haoran Xu, Young Jin Kim, Adam Atkinson, Zheng Zhan, Jiankai Sun, Baolin Peng, Liyuan Liu, Shuohang Wang, Hao Cheng, Jianfeng Gao, Weizhu Chen, Yelong Shen

NeurIPS 2025 Elevating Visual Perception in Multimodal LLMs with Visual Embedding Distillation Jitesh Jain, Zhengyuan Yang, Humphrey Shi, Jianfeng Gao, Jianwei Yang

ICLR 2025 ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learning Xiao Yu, Baolin Peng, Vineeth Vajipey, Hao Cheng, Michel Galley, Jianfeng Gao, Zhou Yu

CVPR 2025 Florence-VL: Enhancing Vision-Language Models with Generative Vision Encoder and Depth-Breadth Fusion Jiuhai Chen, Jianwei Yang, Haiping Wu, Dianqi Li, Jianfeng Gao, Tianyi Zhou, Bin Xiao

NeurIPS 2025 GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents Qianhui Wu, Kanzhi Cheng, Rui Yang, Chaoyun Zhang, Jianwei Yang, Huiqiang Jiang, Jian Mu, Baolin Peng, Bo Qiao, Reuben Tan, Si Qin, Lars Liden, Qingwei Lin, Huan Zhang, Tong Zhang, Jianbing Zhang, Dongmei Zhang, Jianfeng Gao

ICLR 2025 GUI-World: A Video Benchmark and Dataset for Multimodal GUI-Oriented Understanding Dongping Chen, Yue Huang, Siyuan Wu, Jingyu Tang, Huichi Zhou, Qihui Zhang, Zhigang He, Yilin Bai, Chujie Gao, Liuyi Chen, Yiqiang Li, Chenlong Wang, Yue Yu, Tianshuo Zhou, Zhen Li, Yi Gui, Yao Wan, Pan Zhou, Jianfeng Gao, Lichao Sun

ICLR 2025 Generative Adapter: Contextualizing Language Models in Parameters with a Single Forward Pass Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Benjamin Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng

NeurIPS 2025 Interpretable Next-Token Prediction via the Generalized Induction Head Eunji Kim, Sriya Mantena, Weiwei Yang, Chandan Singh, Sungroh Yoon, Jianfeng Gao

ICLR 2025 Latent Action Pretraining from Videos Seonghyeon Ye, Joel Jang, Byeongguk Jeon, Se June Joo, Jianwei Yang, Baolin Peng, Ajay Mandlekar, Reuben Tan, Yu-Wei Chao, Bill Yuchen Lin, Lars Liden, Kimin Lee, Jianfeng Gao, Luke Zettlemoyer, Dieter Fox, Minjoon Seo

ICML 2025 MMInference: Accelerating Pre-Filling for Long-Context Visual Language Models via Modality-Aware Permutation Sparse Attention Yucheng Li, Huiqiang Jiang, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu

ICLRW 2025 MMInference: Accelerating Pre-Filling for Long-Context Visual Language Models via Modality-Aware Permutation Sparse Attention Yucheng Li, Huiqiang Jiang, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu

CVPR 2025 Magma: A Foundation Model for Multimodal AI Agents Jianwei Yang, Reuben Tan, Qianhui Wu, Ruijie Zheng, Baolin Peng, Yongyuan Liang, Yu Gu, Mu Cai, Seonghyeon Ye, Joel Jang, Yuquan Deng, Jianfeng Gao

ICLR 2025 Matryoshka Multimodal Models Mu Cai, Jianwei Yang, Jianfeng Gao, Yong Jae Lee

JAIR 2025 Mechanisms of Symbol Processing for In-Context Learning in Transformer Networks Paul Smolensky, Roland Fernandez, Zhenghao Herbert Zhou, Mattia Opper, Adam Davies, Jianfeng Gao

NeurIPS 2025 Mixture of Inputs: Text Generation Beyond Discrete Token Sampling Yufan Zhuang, Liyuan Liu, Chandan Singh, Jingbo Shang, Jianfeng Gao

NeurIPS 2025 Reinforcement Learning for Reasoning in Large Language Models with One Training Example Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Liyuan Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, Yelong Shen

NeurIPS 2025 SAS: Simulated Attention Score Chuanyang Zheng, Jiankai Sun, Yihang Gao, Yuehao Wang, Peihao Wang, Jing Xiong, Liliang Ren, Hao Cheng, Janardhan Kulkarni, Yelong Shen, Zhangyang Wang, Mac Schwager, Anderson Schneider, Xiaodong Liu, Jianfeng Gao

ICLR 2025 SCBench: A KV Cache-Centric Analysis of Long-Context Methods Yucheng Li, Huiqiang Jiang, Qianhui Wu, Xufang Luo, Surin Ahn, Chengruidong Zhang, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu

ICCV 2025 SITE: Towards Spatial Intelligence Thorough Evaluation Wenqi Wang, Reuben Tan, Pengyue Zhu, Jianwei Yang, Zhengyuan Yang, Lijuan Wang, Andrey Kolobov, Jianfeng Gao, Boqing Gong

ICLR 2025 SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Xufang Luo, Hao Cheng, Dongsheng Li, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Jianfeng Gao

ICML 2025 Simplifying DINO via Coding Rate Regularization Ziyang Wu, Jingyuan Zhang, Druv Pai, Xudong Wang, Chandan Singh, Jianwei Yang, Jianfeng Gao, Yi Ma

TMLR 2025 TRA: Better Length Generalisation with Threshold Relative Attention Mattia Opper, Roland Fernandez, Paul Smolensky, Jianfeng Gao

ICLR 2025 TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policies Ruijie Zheng, Yongyuan Liang, Shuaiyi Huang, Jianfeng Gao, Hal Daumé Iii, Andrey Kolobov, Furong Huang, Jianwei Yang

NeurIPS 2025 Training Language Models to Generate Quality Code with Program Analysis Feedback Feng Yao, Zilong Wang, Liyuan Liu, Junxia Cui, Li Zhong, Xiaohan Fu, Haohui Mai, Vish Krishnan, Jianfeng Gao, Jingbo Shang

ICLR 2025 Vector-ICL: In-Context Learning with Continuous Vector Representations Yufan Zhuang, Chandan Singh, Liyuan Liu, Jingbo Shang, Jianfeng Gao

NeurIPS 2024 Compositional Generalization Across Distributional Shifts with Sparse Tree Operations Paul Soulos, Henry Conklin, Mattia Opper, Paul Smolensky, Jianfeng Gao, Roland Fernandez

NeurIPSW 2024 Compositional Generalization Across Distributional Shifts with Sparse Tree Operations Paul Soulos, Henry Conklin, Mattia Opper, Paul Smolensky, Jianfeng Gao, Roland Fernandez

NeurIPS 2024 Crafting Interpretable Embeddings for Language Neuroscience by Asking LLMs Questions Vinamra Benara, Chandan Singh, John X. Morris, Richard J. Antonello, Ion Stoica, Alexander G. Huth, Jianfeng Gao

NeurIPS 2024 DeepStack: Deeply Stacking Visual Tokens Is Surprisingly Simple and Effective for LMMs Lingchen Meng, Jianwei Yang, Rui Tian, Xiyang Dai, Zuxuan Wu, Jianfeng Gao, Yu-Gang Jiang

ICLR 2024 Fast-ELECTRA for Efficient Pre-Training Chengyu Dong, Liyuan Liu, Hao Cheng, Jingbo Shang, Jianfeng Gao, Xiaodong Liu

NeurIPSW 2024 GUI-WORLD: A GUI-Oriented Video Dataset for Multimodal LLM-Based Agents Dongping Chen, Yue Huang, Siyuan Wu, Jingyu Tang, Huichi Zhou, Qihui Zhang, Zhigang He, Yilin Bai, Chujie Gao, Liuyi Chen, Yiqiang Li, Chenlong Wang, Yue Yu, Tianshuo Zhou, Zhen Li, Yi Gui, Yao Wan, Pan Zhou, Jianfeng Gao, Lichao Sun

NeurIPSW 2024 Generative Adapter: Contextualizing Language Models in Parameters with a Single Forward Pass Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Benjamin Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng

TMLR 2024 IMProv: Inpainting-Based Multimodal Prompting for Computer Vision Tasks Jiarui Xu, Yossi Gandelsman, Amir Bar, Jianwei Yang, Jianfeng Gao, Trevor Darrell, Xiaolong Wang

ICLR 2024 Is Self-Repair a Silver Bullet for Code Generation? Theo X. Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, Armando Solar-Lezama

ECCV 2024 LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models Hao Zhang, Hongyang Li, Feng Li, Tianhe Ren, Xueyan Zou, Shilong Liu, Shijia Huang, Jianfeng Gao, Lei Zhang, Chunyuan Li, Jianwei Yang

ECCV 2024 LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu, Lei Zhang, Jianfeng Gao, Chunyuan Li

TMLR 2024 Learning a Decision Tree Algorithm with Transformers Yufan Zhuang, Liyuan Liu, Chandan Singh, Jingbo Shang, Jianfeng Gao

ICLR 2024 MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, Jianfeng Gao

NeurIPSW 2024 Matryoshka Multimodal Models Mu Cai, Jianwei Yang, Jianfeng Gao, Yong Jae Lee

ICLR 2024 Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, Jianfeng Gao

ECCV 2024 Pix2Gif: Motion-Guided Diffusion for GIF Generation Hitesh Kandala, Jianfeng Gao, Jianwei Yang

ICML 2024 Position: TrustLLM: Trustworthiness in Large Language Models Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Hanchi Sun, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bertie Vidgen, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric P. Xing, Furong Huang, Hao Liu, Heng Ji, Hongyi Wang, Huan Zhang, Huaxiu Yao, Manolis Kellis, Marinka Zitnik, Meng Jiang, Mohit Bansal, James Zou, Jian Pei, Jian Liu, Jianfeng Gao, Jiawei Han, Jieyu Zhao, Jiliang Tang, Jindong Wang, Joaquin Vanschoren, John Mitchell, Kai Shu, Kaidi Xu, Kai-Wei Chang, Lifang He, Lifu Huang, Michael Backes, Neil Zhenqiang Gong, Philip S. Yu, Pin-Yu Chen, Quanquan Gu, Ran Xu, Rex Ying, Shuiwang Ji, Suman Jana, Tianlong Chen, Tianming Liu, Tianyi Zhou, William Yang Wang, Xiang Li, Xiangliang Zhang, Xiao Wang, Xing Xie, Xun Chen, Xuyu Wang, Yan Liu, Yanfang Ye, Yinzhi Cao, Yong Chen, Yue Zhao

NeurIPSW 2024 SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Xufang Luo, Hao Cheng, Dongsheng Li, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Jianfeng Gao

ECCV 2024 Segment and Recognize Anything at Any Granularity Feng Li, Hao Zhang, Peize Sun, Xueyan Zou, Shilong Liu, Chunyuan Li, Jianwei Yang, Lei Zhang, Jianfeng Gao

ICLR 2024 Tell Your Model Where to Attend: Post-Hoc Attention Steering for LLMs Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, Tuo Zhao

NeurIPSW 2024 TemporalBench: Benchmarking Fine-Grained Temporal Understanding for Multimodal Video Models Mu Cai, Reuben Tan, Jianrui Zhang, Bocheng Zou, Kai Zhang, Yao Feng, Fangrui Zhu, Jing Gu, Yiwu Zhong, Yuzhang Shang, Yao Dou, Jaden Park, Jianfeng Gao, Yong Jae Lee, Jianwei Yang

CVPR 2024 Visual In-Context Prompting Feng Li, Qing Jiang, Hao Zhang, Tianhe Ren, Shilong Liu, Xueyan Zou, Huaizhe Xu, Hongyang Li, Jianwei Yang, Chunyuan Li, Lei Zhang, Jianfeng Gao

NeurIPSW 2023 An Empirical Study of Scaling Instruct-Tuned Large Multimodal Models Yadong Lu, Chunyuan Li, Haotian Liu, Jianwei Yang, Jianfeng Gao, Yelong Shen

NeurIPS 2023 Augmenting Language Models with Long-Term Memory Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, Furu Wei

NeurIPSW 2023 Automatic Hallucination Assessment for Aligned Large Language Models via Transferable Adversarial Attacks Xiaodong Yu, Hao Cheng, Xiaodong Liu, Dan Roth, Jianfeng Gao

NeurIPS 2023 Bridging Discrete and Backpropagation: Straight-Through and Beyond Liyuan Liu, Chengyu Dong, Xiaodong Liu, Bin Yu, Jianfeng Gao

NeurIPS 2023 Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Jianfeng Gao

NeurIPSW 2023 Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Jianfeng Gao

ICLR 2023 DeBERTaV3: Improving DeBERTa Using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing Pengcheng He, Jianfeng Gao, Weizhu Chen

ICML 2023 Differentiable Tree Operations Promote Compositional Generalization Paul Soulos, Edward J Hu, Kate Mccurdy, Yunmo Chen, Roland Fernandez, Paul Smolensky, Jianfeng Gao

ICMLW 2023 Differentiable Tree Operations Promote Compositional Generalization Paul Soulos, Edward J Hu, Kate McCurdy, Yunmo Chen, Roland Fernandez, Paul Smolensky, Jianfeng Gao

NeurIPSW 2023 Explaining Black Box Text Modules in Natural Language with Language Models Chandan Singh, Aliyah Hsu, Richard Antonello, Shailee Jain, Alexander Huth, Bin Yu, Jianfeng Gao

CVPR 2023 GLIGEN: Open-Set Grounded Text-to-Image Generation Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, Yong Jae Lee

CVPR 2023 Generalized Decoding for Pixel, Image, and Language Xueyan Zou, Zi-Yi Dou, Jianwei Yang, Zhe Gan, Linjie Li, Chunyuan Li, Xiyang Dai, Harkirat Behl, Jianfeng Wang, Lu Yuan, Nanyun Peng, Lijuan Wang, Yong Jae Lee, Jianfeng Gao

NeurIPS 2023 Guiding Large Language Models via Directional Stimulus Prompting Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, Xifeng Yan

NeurIPS 2023 LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, Jianfeng Gao

CVPR 2023 Learning Customized Visual Models with Retrieval-Augmented Knowledge Haotian Liu, Kilho Son, Jianwei Yang, Ce Liu, Jianfeng Gao, Yong Jae Lee, Chunyuan Li

ICLR 2023 Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions Ansong Ni, Jeevana Priya Inala, Chenglong Wang, Alex Polozov, Christopher Meek, Dragomir Radev, Jianfeng Gao

NeurIPS 2023 Localized Symbolic Knowledge Distillation for Visual Commonsense Models Jae Sung Park, Jack Hessel, Khyathi Chandu, Paul Pu Liang, Ximing Lu, Peter West, Youngjae Yu, Qiuyuan Huang, Jianfeng Gao, Ali Farhadi, Yejin Choi

NeurIPSW 2023 MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, Jianfeng Gao

NeurIPSW 2023 Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, Jianfeng Gao

ICLR 2023 Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learning Sheng Zhang, Hao Cheng, Jianfeng Gao, Hoifung Poon

NeurIPS 2023 Segment Everything Everywhere All at Once Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, Yong Jae Lee

ICMLW 2023 Self-Verification Improves Few-Shot Clinical Information Extraction Zelalem Gero, Chandan Singh, Hao Cheng, Tristan Naumann, Michel Galley, Jianfeng Gao, Hoifung Poon

NeurIPSW 2023 Sparse Backpropagation for MoE Training Liyuan Liu, Jianfeng Gao, Weizhu Chen

NeurIPSW 2023 Tell Your Model Where to Attend: Post-Hoc Attention Steering for LLMs Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, Tuo Zhao

TMLR 2023 Training Vision-Language Transformers from Captions Liangke Gui, Yingshan Chang, Qiuyuan Huang, Subhojit Som, Alexander G Hauptmann, Jianfeng Gao, Yonatan Bisk

ICLR 2023 Visually-Augmented Language Modeling Weizhi Wang, Li Dong, Hao Cheng, Haoyu Song, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, Furu Wei

NeurIPS 2022 Coarse-to-Fine Vision-Language Pre-Training with Fusion in the Backbone Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, Lijuan Wang

NeurIPS 2022 ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models Chunyuan Li, Haotian Liu, Liunian Li, Pengchuan Zhang, Jyoti Aneja, Jianwei Yang, Ping Jin, Houdong Hu, Zicheng Liu, Yong Jae Lee, Jianfeng Gao

ICLR 2022 Efficient Self-Supervised Vision Transformers for Representation Learning Chunyuan Li, Jianwei Yang, Pengchuan Zhang, Mei Gao, Bin Xiao, Xiyang Dai, Lu Yuan, Jianfeng Gao

NeurIPS 2022 Fault-Aware Neural Code Rankers Jeevana Priya Inala, Chenglong Wang, Mei Yang, Andres Codas, Mark Encarnación, Shuvendu Lahiri, Madanlal Musuvathi, Jianfeng Gao

NeurIPS 2022 Few-Shot Task-Agnostic Neural Architecture Search for Distilling Large Language Models Dongkuan Xu, Subhabrata Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao

NeurIPS 2022 Focal Modulation Networks Jianwei Yang, Chunyuan Li, Xiyang Dai, Jianfeng Gao

NeurIPS 2022 GLIPv2: Unifying Localization and Vision-Language Understanding Haotian Zhang, Pengchuan Zhang, Xiaowei Hu, Yen-Chun Chen, Liunian Li, Xiyang Dai, Lijuan Wang, Lu Yuan, Jenq-Neng Hwang, Jianfeng Gao

CVPR 2022 Grounded Language-Image Pre-Training Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, Jianfeng Gao

IJCAI 2022 Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang

NeurIPS 2022 K-LITE: Learning Transferable Visual Models with External Knowledge Sheng Shen, Chunyuan Li, Xiaowei Hu, Yujia Xie, Jianwei Yang, Pengchuan Zhang, Zhe Gan, Lijuan Wang, Lu Yuan, Ce Liu, Kurt Keutzer, Trevor Darrell, Anna Rohrbach, Jianfeng Gao

ICLR 2022 No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

CVPR 2022 RegionCLIP: Region-Based Language-Image Pretraining Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, Jianfeng Gao

AAAI 2022 RetGen: A Joint Framework for Retrieval and Grounded Text Generation Modeling Yizhe Zhang, Siqi Sun, Xiang Gao, Yuwei Fang, Chris Brockett, Michel Galley, Jianfeng Gao, Bill Dolan

ICLR 2022 Taming Sparsely Activated Transformer with Stochastic Experts Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan, Ruofei Zhang, Jianfeng Gao, Tuo Zhao

CVPR 2022 Unified Contrastive Learning in Image-Text-Label Space Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Bin Xiao, Ce Liu, Lu Yuan, Jianfeng Gao

AAAI 2022 ValueNet: A New Dataset for Human Value Driven Dialogue System Liang Qiu, Yizhou Zhao, Jinchao Li, Pan Lu, Baolin Peng, Jianfeng Gao, Song-Chun Zhu

CVPR 2022 WebQA: Multihop and Multimodal QA Yingshan Chang, Mridu Narang, Hisami Suzuki, Guihong Cao, Jianfeng Gao, Yonatan Bisk

AAAI 2021 A Controllable Model of Grounded Response Generation Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang, Xiang Gao, Chris Quirk, Rik Koncel-Kedziorski, Jianfeng Gao, Hannaneh Hajishirzi, Mari Ostendorf, Bill Dolan

AAAI 2021 Data Augmentation for Abstractive Query-Focused Multi-Document Summarization Ramakanth Pasunuru, Asli Celikyilmaz, Michel Galley, Chenyan Xiong, Yizhe Zhang, Mohit Bansal, Jianfeng Gao

ICLR 2021 DeBERTa: Decoding-Enhanced BERT with Disentangled Attention Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen

NeurIPS 2021 Focal Attention for Long-Range Interactions in Vision Transformers Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Xiyang Dai, Bin Xiao, Lu Yuan, Jianfeng Gao

ICCV 2021 Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding Pengchuan Zhang, Xiyang Dai, Jianwei Yang, Bin Xiao, Lu Yuan, Lei Zhang, Jianfeng Gao

ICCV 2021 TACo: Token-Aware Cascade Contrastive Learning for Video-Text Alignment Jianwei Yang, Yonatan Bisk, Jianfeng Gao

NeurIPS 2021 Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer Ge Yang, Edward Hu, Igor Babuschkin, Szymon Sidor, Xiaodong Liu, David Farhi, Nick Ryder, Jakub Pachocki, Weizhu Chen, Jianfeng Gao

AAAI 2021 VIVO: Visual Vocabulary Pre-Training for Novel Object Captioning Xiaowei Hu, Xi Yin, Kevin Lin, Lei Zhang, Jianfeng Gao, Lijuan Wang, Zicheng Liu

CVPR 2021 VinVL: Revisiting Visual Representations in Vision-Language Models Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, Jianfeng Gao

AAAI 2020 Complementary Auxiliary Classifiers for Label-Conditional Text Generation Yuan Li, Chunyuan Li, Yizhe Zhang, Xiujun Li, Guoqing Zheng, Lawrence Carin, Jianfeng Gao

ICML 2020 Feature Quantization Improves GAN Training Yang Zhao, Chunyuan Li, Ping Yu, Jianfeng Gao, Changyou Chen

ICML 2020 Mapping Natural-Language Problems to Formal-Language Solutions Using Structured Neural Representations Kezhen Chen, Qiuyuan Huang, Hamid Palangi, Paul Smolensky, Ken Forbus, Jianfeng Gao

ICLR 2020 On the Variance of the Adaptive Learning Rate and Beyond Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han

ECCV 2020 Oscar: Object-Semantics Aligned Pre-Training for Vision-Language Tasks Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, Jianfeng Gao

AAAI 2020 PIQA: Reasoning About Physical Commonsense in Natural Language Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, Yejin Choi

ICLR 2020 RaCT: Toward Amortized Ranking-Critical Training for Collaborative Filtering Sam Lobel, Chunyuan Li, Jianfeng Gao, Lawrence Carin

ICML 2020 UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Jianfeng Gao, Songhao Piao, Ming Zhou, Hsiao-Wuen Hon

AAAI 2020 Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou, Hamid Palangi, Lei Zhang, Houdong Hu, Jason J. Corso, Jianfeng Gao

AAAI 2020 What Makes a Good Story? Designing Composite Rewards for Visual Storytelling Junjie Hu, Yu Cheng, Zhe Gan, Jingjing Liu, Jianfeng Gao, Graham Neubig

AAAI 2019 Interactive Semantic Parsing for If-Then Recipes via Hierarchical Reinforcement Learning Ziyu Yao, Xiujun Li, Jianfeng Gao, Brian M. Sadler, Huan Sun

AAAI 2019 Switch-Based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learning Yuexin Wu, Xiujun Li, Jingjing Liu, Jianfeng Gao, Yiming Yang

NeurIPS 2019 Unified Language Model Pre-Training for Natural Language Understanding and Generation Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

AAAI 2018 A Knowledge-Grounded Neural Conversation Model Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, Michel Galley

AAAI 2018 BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems Zachary C. Lipton, Xiujun Li, Jianfeng Gao, Lihong Li, Faisal Ahmed, Li Deng

NeurIPS 2018 Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, Bill Dolan

NeurIPS 2018 M-Walk: Learning to Walk over Graphs Using Monte Carlo Tree Search Yelong Shen, Jianshu Chen, Po-Sen Huang, Yuqing Guo, Jianfeng Gao

NeurIPS 2018 Navigating with Graph Representations for Fast and Scalable Decoding of Neural Language Models Minjia Zhang, Wenhan Wang, Xiaodong Liu, Jianfeng Gao, Yuxiong He

CVPR 2017 Semantic Compositional Networks for Visual Captioning Zhe Gan, Chuang Gan, Xiaodong He, Yunchen Pu, Kenneth Tran, Jianfeng Gao, Lawrence Carin, Li Deng

CVPR 2017 StyleNet: Generating Attractive Visual Captions with Styles Chuang Gan, Zhe Gan, Xiaodong He, Jianfeng Gao, Li Deng

ICLR 2017 TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency Adji B. Dieng, Chong Wang, Jianfeng Gao, John W. Paisley

ECCV 2016 MS-Celeb-1m: A Dataset and Benchmark for Large-Scale Face Recognition Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, Jianfeng Gao

ICLR 2016 Reasoning in Vector Space: An Exploratory Study of Question Answering Moontae Lee, Xiaodong He, Wen-tau Yih, Jianfeng Gao, Li Deng, Paul Smolensky

CVPR 2016 Stacked Attention Networks for Image Question Answering Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Smola

ICLR 2015 Embedding Entities and Relations for Learning and Inference in Knowledge Bases Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, Li Deng

NeurIPS 2015 End-to-End Learning of LDA by Mirror-Descent Back Propagation over a Deep Architecture Jianshu Chen, Ji He, Yelong Shen, Lin Xiao, Xiaodong He, Jianfeng Gao, Xinying Song, Li Deng

CVPR 2015 From Captions to Visual Concepts and Back Hao Fang, Saurabh Gupta, Forrest Iandola, Rupesh K. Srivastava, Li Deng, Piotr Dollar, Jianfeng Gao, Xiaodong He, Margaret Mitchell, John C. Platt, C. Lawrence Zitnick, Geoffrey Zweig

AAAI 2014 Online Classification Using a Voted RDA Method Tianbing Xu, Jianfeng Gao, Lin Xiao, Amelia C. Regan

ICML 2007 Scalable Training of L1-Regularized Log-Linear Models Galen Andrew, Jianfeng Gao