Neubig, Graham

94 publications

ICLR 2026 Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-Tuning of LLM Agents Yueqi Song, Ketan Ramaneti, Zaid Sheikh, Ziru Chen, Boyu Gou, Tianbao Xie, Yiheng Xu, Danyang Zhang, Apurva Gandhi, Fan Yang, Joseph Liu, Tianyue Ou, Zhihao Yuan, Frank F. Xu, Shuyan Zhou, Xingyao Wang, Xiang Yue, Tao Yu, Huan Sun, Yu Su, Graham Neubig

ICLR 2026 Ambig-SWE: Interactive Agents to Overcome Underspecificity in Software Engineering Sanidhya Vijayvargiya, Xuhui Zhou, Akhila Yerukola, Maarten Sap, Graham Neubig

ICLR 2026 EDIT-Bench: Evaluating LLM Abilities to Perform Real-World Instructed Code Edits Wayne Chi, Valerie Chen, Ryan Shar, Aditya Mittal, Jenny Liang, Wei-Lin Chiang, Anastasios Nikolas Angelopoulos, Ion Stoica, Graham Neubig, Ameet Talwalkar, Chris Donahue

ICLR 2026 Go-Browse: Training Web Agents with Structured Exploration Apurva Gandhi, Graham Neubig

ICLR 2026 OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety Sanidhya Vijayvargiya, Aditya Bharat Soni, Xuhui Zhou, Zora Zhiruo Wang, Nouha Dziri, Graham Neubig, Maarten Sap

ICLR 2026 Prompt-MII: Meta-Learning Instruction Induction for LLMs Emily Xiao, Yixiao Zeng, Ada Chen, Chin-Jou Li, Amanda Bertsch, Graham Neubig

ICLR 2026 RefineBench: Evaluating Refinement Capability of Language Models via Checklists Young-Jun Lee, Seungone Kim, Byung-Kwan Lee, Minkyeong Moon, Yechan Hwang, Jong Myoung Kim, Graham Neubig, Sean Welleck, Ho-Jin Choi

ICLR 2026 The CoT Encyclopedia: Analyzing, Predicting, and Controlling How a Reasoning Model Will Think Seongyun Lee, Seungone Kim, Minju Seo, Yongrae Jo, Dongyoung Go, Hyeonbin Hwang, Jinho Park, Xiang Yue, Sean Welleck, Graham Neubig, Moontae Lee, Minjoon Seo

ICLR 2026 The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution Junlong Li, Wenshuo Zhao, Jian Zhao, Weihao Zeng, Haoze Wu, Xiaochen Wang, Rui Ge, Yuxuan Cao, Yuzhen Huang, Wei Liu, Junteng Liu, Zhaochen Su, Yiyang Guo, Fan Zhou, Lueyang Zhang, Juan Michelini, Xingyao Wang, Xiang Yue, Shuyan Zhou, Graham Neubig, Junxian He

TMLR 2025 A Gold Standard Dataset for the Reviewer Assignment Problem Ivan Stelmakh, John Frederick Wieting, Yang Xi, Graham Neubig, Nihar B Shah

ICML 2025 Agent Workflow Memory Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, Graham Neubig

CVPR 2025 AutoPresent: Designing Structured Visuals from Scratch Jiaxin Ge, Zora Zhiruo Wang, Xuhui Zhou, Yi-Hao Peng, Sanjay Subramanian, Qinyue Tan, Maarten Sap, Alane Suhr, Daniel Fried, Graham Neubig, Trevor Darrell

ICLR 2025 Better Instruction-Following Through Minimum Bayes Risk Ian Wu, Patrick Fernandes, Amanda Bertsch, Seungone Kim, Sina Khoshfetrat Pakazad, Graham Neubig

NeurIPS 2025 Checklists Are Better than Reward Models for Aligning Language Models Vijay Viswanathan, Yanchao Sun, Xiang Kong, Meng Cao, Graham Neubig, Tongshuang Wu

ICML 2025 Demystifying Long Chain-of-Thought Reasoning Shiming Yang, Yuxuan Tong, Xinyao Niu, Graham Neubig, Xiang Yue

ICLRW 2025 Demystifying Long Chain-of-Thought Reasoning in LLMs Edward Yeo, Yuxuan Tong, Xinyao Niu, Graham Neubig, Xiang Yue

ICLRW 2025 Exploring the Pre-Conditions for Memory-Learning Agents Vishwa Shah, Vishruth Veerendranath, Graham Neubig, Daniel Fried, Zora Zhiruo Wang

ICLR 2025 Harnessing Webpage UIs for Text-Rich Visual Understanding Junpeng Liu, Tianyue Ou, Yifan Song, Yuxiao Qu, Wai Lam, Chenyan Xiong, Wenhu Chen, Graham Neubig, Xiang Yue

ICLR 2025 OpenHands: An Open Platform for AI Software Developers as Generalist Agents Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, Graham Neubig

ICML 2025 Overtrained Language Models Are Harder to Fine-Tune Jacob Mitchell Springer, Sachin Goyal, Kaiyue Wen, Tanishq Kumar, Xiang Yue, Sadhika Malladi, Graham Neubig, Aditi Raghunathan

ICLRW 2025 Overtrained Language Models Are Harder to Fine-Tune Jacob Mitchell Springer, Sachin Goyal, Kaiyue Wen, Tanishq Kumar, Xiang Yue, Sadhika Malladi, Graham Neubig, Aditi Raghunathan

ICLR 2025 Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages Xiang Yue, Yueqi Song, Akari Asai, Seungone Kim, Jean de Dieu Nyandwi, Simran Khanuja, Anjali Kantharuban, Lintang Sutawika, Sathyanarayanan Ramamoorthy, Graham Neubig

ICML 2025 RAGGED: Towards Informed Design of Scalable and Stable RAG Systems Jennifer Hsia, Afreen Shaikh, Zora Zhiruo Wang, Graham Neubig

ICLR 2025 Repetition Improves Language Model Embeddings Jacob Mitchell Springer, Suhas Kotha, Daniel Fried, Graham Neubig, Aditi Raghunathan

TMLR 2025 The BrowserGym Ecosystem for Web Agent Research Thibault Le Sellier de Chezelles, Maxime Gasse, Alexandre Lacoste, Massimo Caccia, Alexandre Drouin, Léo Boisvert, Megh Thakkar, Tom Marty, Rim Assouel, Sahar Omidi Shayegan, Lawrence Keunho Jang, Xing Han Lù, Ori Yoran, Dehan Kong, Frank F. Xu, Siva Reddy, Graham Neubig, Quentin Cappart, Russ Salakhutdinov, Nicolas Chapados

NeurIPS 2025 TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks Frank F. Xu, Yufan Song, Boxuan Li, Yuxuan Tang, Kritanjali Jain, Mengxue Bao, Zora Zhiruo Wang, Xuhui Zhou, Zhitong Guo, Murong Cao, Mingyang Yang, Hao Yang Lu, Amaad Martin, Zhe Su, Leander Melroy Maben, Raj Mehta, Wayne Chi, Lawrence Keunho Jang, Yiqing Xie, Shuyan Zhou, Graham Neubig

ICML 2025 Training Software Engineering Agents and Verifiers with SWE-Gym Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, Yizhe Zhang

ICLRW 2025 Training Software Engineering Agents and Verifiers with SWE-Gym Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, Yizhe Zhang

NeurIPS 2024 Alignment for Honesty Yuqing Yang, Ethan Chern, Xipeng Qiu, Graham Neubig, Pengfei Liu

NeurIPS 2024 Divergences Between Language Models and Human Brains Yuchen Zhou, Emmy Liu, Graham Neubig, Michael J. Tarr, Leila Wehbe

ECCV 2024 Evaluating Text-to-Visual Generation with Image-to-Text Generation Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, Deva Ramanan

CVPRW 2024 Evaluating and Improving Compositional Text-to-Visual Generation Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Xide Xia, Pengchuan Zhang, Graham Neubig, Deva Ramanan

TMLR 2024 From Decoding to Meta-Generation: Inference-Time Algorithms for Large Language Models Sean Welleck, Amanda Bertsch, Matthew Finlayson, Hailey Schoelkopf, Alex Xie, Graham Neubig, Ilia Kulikov, Zaid Harchaoui

ICMLW 2024 In-Context Learning with Long-Context Models: An In-Depth Exploration Amanda Bertsch, Maor Ivgi, Uri Alon, Jonathan Berant, Matthew R. Gormley, Graham Neubig

ICLR 2024 Learning Performance-Improving Code Edits Alexander G Shypula, Aman Madaan, Yimeng Zeng, Uri Alon, Jacob R. Gardner, Yiming Yang, Milad Hashemi, Graham Neubig, Parthasarathy Ranganathan, Osbert Bastani, Amir Yazdanbakhsh

NeurIPS 2024 MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures Jinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng, Mahir Shah, Kabir Jain, Graham Neubig, Yang You

TMLR 2024 Multitask Learning Can Improve Worst-Group Outcomes Atharva Kulkarni, Lucio M. Dery, Amrith Setlur, Aditi Raghunathan, Ameet Talwalkar, Graham Neubig

NeurIPS 2024 NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples Baiqi Li, Zhiqiu Lin, Wenxuan Peng, Jean de Dieu Nyandwi, Daniel Jiang, Zixian Ma, Simran Khanuja, Ranjay Krishna, Graham Neubig, Deva Ramanan

NeurIPSW 2024 RAGGED: Towards Informed Design of Retrieval Augmented Generation Systems Jennifer Hsia, Afreen Shaikh, Zhiruo Wang, Graham Neubig

ICLR 2024 SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, Maarten Sap

NeurIPS 2024 Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scale Tianyue Ou, Frank F. Xu, Aman Madaan, Jiarui Liu, Robert Lo, Abishek Sridhar, Sudipta Sengupta, Dan Roth, Graham Neubig, Shuyan Zhou

ICML 2024 TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Tasks Zhiruo Wang, Graham Neubig, Daniel Fried

ICLRW 2024 VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, Daniel Fried

ICLR 2024 WebArena: A Realistic Web Environment for Building Autonomous Agents Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, Graham Neubig

ICLR 2023 AANG : Automating Auxiliary Learning Lucio M. Dery, Paul Michel, Mikhail Khodak, Graham Neubig, Ameet Talwalkar

ICLR 2023 Computational Language Acquisition with Theory of Mind Andy Liu, Hao Zhu, Emmy Liu, Yonatan Bisk, Graham Neubig

ICML 2023 Cross-Modal Fine-Tuning: Align Then Refine Junhong Shen, Liam Li, Lucio M. Dery, Corey Staten, Mikhail Khodak, Graham Neubig, Ameet Talwalkar

ICLR 2023 DiffusER: Diffusion via Edit-Based Reconstruction Machel Reid, Vincent Josua Hellendoorn, Graham Neubig

ICLR 2023 DocPrompting: Generating Code by Retrieving the Docs Shuyan Zhou, Uri Alon, Frank F. Xu, Zhengbao Jiang, Graham Neubig

CVPR 2023 EXCALIBUR: Encouraging and Evaluating Embodied Exploration Hao Zhu, Raghav Kapoor, So Yeon Min, Winson Han, Jiatai Li, Kaiwen Geng, Graham Neubig, Yonatan Bisk, Aniruddha Kembhavi, Luca Weihs

ICLR 2023 Mega: Moving Average Equipped Gated Attention Xuezhe Ma, Chunting Zhou, Xiang Kong, Junxian He, Liangke Gui, Graham Neubig, Jonathan May, Luke Zettlemoyer

ICML 2023 PAL: Program-Aided Language Models Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, Graham Neubig

NeurIPSW 2023 SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, Maarten Sap

NeurIPS 2023 Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch, Uri Alon, Graham Neubig, Matthew Gormley

NeurIPSW 2023 WebArena: A Realistic Web Environment for Building Autonomous Agents Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, Graham Neubig

NeurIPSW 2023 WebArena: A Realistic Web Environment for Building Autonomous Agents Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, Graham Neubig

ICML 2023 Why Do Nearest Neighbor Language Models Work? Frank F. Xu, Uri Alon, Graham Neubig

ICLRW 2022 A Systematic Evaluation of Large Language Models of Code Frank F. Xu, Uri Alon, Graham Neubig, Vincent Josua Hellendoorn

JAIR 2022 Can We Automate Scientific Reviewing? Weizhe Yuan, Pengfei Liu, Graham Neubig

ICLR 2022 Capturing Structural Locality in Non-Parametric Language Models Frank F. Xu, Junxian He, Graham Neubig, Vincent Josua Hellendoorn

ICLR 2022 Distributionally Robust Models with Parametric Likelihood Ratios Paul Michel, Tatsunori Hashimoto, Graham Neubig

AAAI 2022 Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations Siddhant Arora, Danish Pruthi, Norman M. Sadeh, William W. Cohen, Zachary C. Lipton, Graham Neubig

NeurIPS 2022 Learning to Scaffold: Optimizing Model Explanations for Teaching Patrick Fernandes, Marcos Treviso, Danish Pruthi, André Martins, Graham Neubig

ICLRW 2022 Learning to Superoptimize Real-World Programs Alexander G Shypula, Pengcheng Yin, Jeremy Lacomis, Claire Le Goues, Edward Schwartz, Graham Neubig

ICML 2022 Neuro-Symbolic Language Modeling with Automaton-Augmented Retrieval Uri Alon, Frank Xu, Junxian He, Sudipta Sengupta, Dan Roth, Graham Neubig

ICMLW 2022 Neuro-Symbolic Language Modeling with Automaton-Augmented Retrieval Uri Alon, Frank F. Xu, Junxian He, Sudipta Sengupta, Dan Roth, Graham Neubig

ICLR 2022 Should We Be Pre-Training? an Argument for End-Task Aware Training as an Alternative Lucio M. Dery, Paul Michel, Ameet Talwalkar, Graham Neubig

ICML 2022 Symmetric Machine Theory of Mind Melanie Sclar, Graham Neubig, Yonatan Bisk

ICLR 2022 Towards a Unified View of Parameter-Efficient Transfer Learning Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, Graham Neubig

NeurIPS 2021 BARTScore: Evaluating Generated Text as Text Generation Weizhe Yuan, Graham Neubig, Pengfei Liu

ICML 2021 Examining and Combating Spurious Features Under Distribution Shift Chunting Zhou, Xuezhe Ma, Paul Michel, Graham Neubig

ICML 2021 Few-Shot Language Coordination by Modeling Theory of Mind Hao Zhu, Graham Neubig, Yonatan Bisk

ICLR 2021 Learning Structural Edits via Incremental Tree Transformations Ziyu Yao, Frank F. Xu, Pengcheng Yin, Huan Sun, Graham Neubig

ICLR 2021 Meta Back-Translation Hieu Pham, Xinyi Wang, Yiming Yang, Graham Neubig

ICLR 2021 Modeling the Second Player in Distributionally Robust Optimization Paul Michel, Tatsunori Hashimoto, Graham Neubig

ICLR 2020 A Probabilistic Formulation of Unsupervised Text Style Transfer Junxian He, Xinyi Wang, Graham Neubig, Taylor Berg-Kirkpatrick

JAIR 2020 A Set of Recommendations for Assessing Human-Machine Parity in Language Translation Samuel Läubli, Sheila Castilho, Graham Neubig, Rico Sennrich, Qinlan Shen, Antonio Toral

ICLR 2020 Cross-Lingual Alignment vs Joint Training: A Comparative Study and a Simple Unified Framework Zirui Wang, Jiateng Xie, Ruochen Xu, Yiming Yang, Graham Neubig, Jaime Carbonell

ICLR 2020 Differentiable Reasoning over a Virtual Knowledge Base Bhuwan Dhingra, Manzil Zaheer, Vidhisha Balachandran, Graham Neubig, Ruslan Salakhutdinov, William W. Cohen

AAAI 2020 Latent Relation Language Models Hiroaki Hayashi, Zecong Hu, Chenyan Xiong, Graham Neubig

NeurIPS 2020 Learning Sparse Prototypes for Text Generation Junxian He, Taylor Berg-Kirkpatrick, Graham Neubig

AAAI 2020 Merging Weak and Active Supervision for Semantic Parsing Ansong Ni, Pengcheng Yin, Graham Neubig

ICML 2020 Optimizing Data Usage via Differentiable Rewards Xinyi Wang, Hieu Pham, Paul Michel, Antonios Anastasopoulos, Jaime Carbonell, Graham Neubig

ICLR 2020 Understanding Knowledge Distillation in Non-Autoregressive Machine Translation Chunting Zhou, Graham Neubig, Jiatao Gu

AAAI 2020 What Makes a Good Story? Designing Composite Rewards for Visual Storytelling Junjie Hu, Yu Cheng, Zhe Gan, Jingjing Liu, Jianfeng Gao, Graham Neubig

ICML 2020 XTREME: A Massively Multilingual Multi-Task Benchmark for Evaluating Cross-Lingual Generalisation Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, Melvin Johnson

NeurIPS 2019 Are Sixteen Heads Really Better than One? Paul Michel, Omer Levy, Graham Neubig

ICLR 2019 Lagging Inference Networks and Posterior Collapse in Variational Autoencoders Junxian He, Daniel Spokoyny, Graham Neubig, Taylor Berg-Kirkpatrick

ICLR 2019 Learning to Represent Edits Pengcheng Yin, Graham Neubig, Miltiadis Allamanis, Marc Brockschmidt, Alexander L. Gaunt

ICLR 2019 Multilingual Neural Machine Translation with Soft Decoupled Encoding Xinyi Wang, Hieu Pham, Philip Arthur, Graham Neubig

AAAI 2019 Zero-Shot Neural Transfer for Cross-Lingual Entity Linking Shruti Rijhwani, Jiateng Xie, Graham Neubig, Jaime G. Carbonell

AAAI 2018 A Continuous Relaxation of Beam Search for End-to-End Training of Neural Sequence Models Kartik Goyal, Graham Neubig, Chris Dyer, Taylor Berg-Kirkpatrick

NeurIPS 2017 Controllable Invariance Through Adversarial Feature Learning Qizhe Xie, Zihang Dai, Yulun Du, Eduard Hovy, Graham Neubig

NeurIPS 2017 On-the-Fly Operation Batching in Dynamic Computation Graphs Graham Neubig, Yoav Goldberg, Chris Dyer