Cohan, Arman

20 publications

ICLR 2025 ChemAgent: Self-Updating Memories in Large Language Models Improves Chemical Reasoning Xiangru Tang, Tianyu Hu, Muyang Ye, Yanjun Shao, Xunjian Yin, Siru Ouyang, Wangchunshu Zhou, Pan Lu, Zhuosheng Zhang, Yilun Zhao, Arman Cohan, Mark Gerstein

NeurIPS 2025 DyFlow: Dynamic Workflow Framework for Agentic Reasoning Yanbo Wang, Zixiang Xu, Yue Huang, Xiangqi Wang, Zirui Song, Lang Gao, Chenxi Wang, Xiangru Tang, Yue Zhao, Arman Cohan, Xiangliang Zhang, Xiuying Chen

ICLRW 2025 ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code Xiangru Tang, Yuliang Liu, Zefan Cai, Daniel Shao, Junjie Lu, Yichi Zhang, Zexuan Deng, Helan Hu, Kaikai An, Ruijun Huang, Shuzheng Si, Chen Sheng, Haozhe Zhao, Liang Chen, Tianyu Liu, Yujia Qin, Wangchunshu Zhou, Yilun Zhao, Zhiwei Jiang, Baobao Chang, Arman Cohan, Mark Gerstein

CVPR 2025 MMVU: Measuring Expert-Level Multi-Discipline Video Understanding Yilun Zhao, Haowei Zhang, Lujing Xie, Tongyan Hu, Guo Gan, Yitao Long, Zhiyuan Hu, Weiyuan Chen, Chuhan Li, Zhijian Xu, Chengye Wang, Ziyao Shangguan, Zhenwen Liang, Yixin Liu, Chen Zhao, Arman Cohan

NeurIPS 2025 Measuring What Matters: Construct Validity in Large Language Model Benchmarks Andrew M. Bean, Ryan Othniel Kearns, Angelika Romanou, Franziska Sofia Hafner, Harry Mayne, Jan Batzner, Negar Foroutan, Chris Schmitz, Karolina Korgul, Hunar Batra, Oishi Deb, Emma Beharry, Cornelius Emde, Thomas Foster, Anna Gausen, María Grandury, Simeng Han, Valentin Hofmann, Lujain Ibrahim, Hazel Kim, Hannah Rose Kirk, Fangru Lin, Gabrielle Kaili-May Liu, Lennart Luettgau, Jabez Magomere, Jonathan Rystrøm, Anna Sotnikova, Yushi Yang, Yilun Zhao, Adel Bibi, Antoine Bosselut, Ronald Clark, Arman Cohan, Jakob Nicolaus Foerster, Yarin Gal, Scott A. Hale, Inioluwa Deborah Raji, Christopher Summerfield, Philip Torr, Cozmin Ududec, Luc Rocher, Adam Mahdi

NeurIPS 2025 On Evaluating LLM Alignment by Evaluating LLMs as Judges Yixin Liu, Pengfei Liu, Arman Cohan

ICLRW 2025 PHYSICS: Benchmarking Foundation Models for Problem Solving in Physics Kaiyue Feng, Yilun Zhao, Yixin Liu, Tianyu Yang, Chen Zhao, John Sous, Arman Cohan

AAAI 2025 RouterRetriever: Routing over a Mixture of Expert Embedding Models Hyunji Lee, Luca Soldaini, Arman Cohan, Minjoon Seo, Kyle Lo

NeurIPS 2025 SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks Yilun Zhao, Kaiyan Zhang, Tiansheng Hu, Sihong Wu, Ronan Le Bras, Yixin Liu, Xiangru Tang, Joseph Chee Chang, Jesse Dodge, Jonathan Bragg, Chen Zhao, Hannaneh Hajishirzi, Doug Downey, Arman Cohan

ICLR 2025 TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models Ziyao Shangguan, Chuhan Li, Yuxuan Ding, Yanan Zheng, Yilun Zhao, Tesca Fitzgerald, Arman Cohan

NeurIPSW 2024 COMAL: A Convergent Meta-Algorithm for Aligning LLMs with General Preferences Yixin Liu, Argyris Oikonomou, Weiqiang Zheng, Yang Cai, Arman Cohan

ICLRW 2024 MedAgents: Large Language Models as Collaborators for Zero-Shot Medical Reasoning Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, Mark Gerstein

ICML 2024 NExT: Teaching Large Language Models to Reason About Code Execution Ansong Ni, Miltiadis Allamanis, Arman Cohan, Yinlin Deng, Kensen Shi, Charles Sutton, Pengcheng Yin

ICML 2024 Observable Propagation: Uncovering Feature Vectors in Transformers Jacob Dunefsky, Arman Cohan

ICLRW 2024 Prioritizing Safeguarding over Autonomy: Risks of LLM Agents for Science Xiangru Tang, Qiao Jin, Kunlun Zhu, Tongxin Yuan, Yichi Zhang, Wangchunshu Zhou, Meng Qu, Yilun Zhao, Jian Tang, Zhuosheng Zhang, Arman Cohan, Zhiyong Lu, Mark Gerstein

NeurIPSW 2024 SCIURus: Shared Circuits for Interpretable Uncertainty Representations in Language Models Carter Teplica, Yixin Liu, Arman Cohan, Tim G. J. Rudner

NeurIPSW 2024 SCIURus: Shared Circuits for Interpretable Uncertainty Representations in Language Models Carter Teplica, Yixin Liu, Arman Cohan, Tim G. J. Rudner

NeurIPSW 2024 SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature David Wadden, Kejian Shi, Jacob Morrison, Aakanksha Naik, Shruti Singh, Nitzan Barzilay, Kyle Lo, Tom Hope, Luca Soldaini, Zejiang Shen, Doug Downey, Hannaneh Hajishirzi, Arman Cohan

NeurIPSW 2023 Benchmark Probing: Investigating Data Leakage in Large Language Models Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, Arman Cohan

NeurIPS 2021 FLEX: Unifying Evaluation for Few-Shot NLP Jonathan Bragg, Arman Cohan, Kyle Lo, Iz Beltagy