Bisk, Yonatan

46 publications

CoRL 2025 CASPER: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Models Huihan Liu, Rutav Shah, Shuijing Liu, Jack Pittenger, Mingyo Seo, Yuchen Cui, Yonatan Bisk, Roberto Martín-Martín, Yuke Zhu

TMLR 2025 Efficient Hardware Scaling and Diminishing Returns in Large-Scale Training of Language Models Jared Fernandez, Luca Wehrstedt, Leonid Shamis, Mostafa Elhoushi, Kalyan Saladi, Yonatan Bisk, Emma Strubell, Jacob Kahn

ICLR 2025 Explore Theory of Mind: Program-Guided Adversarial Data Generation for Theory of Mind Reasoning Melanie Sclar, Jane Dwivedi-Yu, Maryam Fazel-Zarandi, Yulia Tsvetkov, Yonatan Bisk, Yejin Choi, Asli Celikyilmaz

ICLR 2025 Language Models Need Inductive Biases to Count Inductively Yingshan Chang, Yonatan Bisk

ICML 2025 Position: You Can’t Manufacture a NeRF Ma Kimmel, Mueed Ur Rehman, Yonatan Bisk, Gary K. Fedder

CVPR 2025 Re-Thinking Temporal Search for Long-Form Video Understanding Jinhui Ye, Zihan Wang, Haosen Sun, Keshigeyan Chandrasegaran, Zane Durante, Cristobal Eyzaguirre, Yonatan Bisk, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei, Jiajun Wu, Manling Li

CoRL 2024 ANAVI: Audio Noise Awareness Using Visual of Indoor Environments for NAVIgation Vidhi Jain, Rishi Veerapaneni, Yonatan Bisk

NeurIPS 2024 Diffusion PID: Interpreting Diffusion via Partial Information Decomposition Shaurya Dewan, Rushikesh Zawar, Prakanshul Saxena, Yingshan Chang, Andrew Luo, Yonatan Bisk

NeurIPSW 2024 Embodied-RAG: General Non-Parametric Embodied Memory for Retrieval and Generation Quanting Xie, So Yeon Min, Tianyi Zhang, Kedi Xu, Aarav Bajaj, Russ Salakhutdinov, Matthew Johnson-Roberson, Yonatan Bisk

NeurIPSW 2024 Embodied-RAG: General Non-Parametric Embodied Memory for Retrieval and Generation Quanting Xie, So Yeon Min, Tianyi Zhang, Kedi Xu, Aarav Bajaj, Russ Salakhutdinov, Matthew Johnson-Roberson, Yonatan Bisk

CVPR 2024 OpenEQA: Embodied Question Answering in the Era of Foundation Models Arjun Majumdar, Anurag Ajay, Xiaohan Zhang, Pranav Putta, Sriram Yenamandra, Mikael Henaff, Sneha Silwal, Paul Mcvay, Oleksandr Maksymets, Sergio Arnaud, Karmesh Yadav, Qiyang Li, Ben Newman, Mohit Sharma, Vincent Berges, Shiqi Zhang, Pulkit Agrawal, Yonatan Bisk, Dhruv Batra, Mrinal Kalakrishnan, Franziska Meier, Chris Paxton, Alexander Sax, Aravind Rajeswaran

ICLR 2024 SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, Maarten Sap

ECCV 2024 Situated Instruction Following So Yeon Min, Xavier Puig, Devendra Singh Chaplot, Tsung-Yen Yang, Priyam Parashar, Akshara Rai, Ruslan Salakhutdinov, Yonatan Bisk, Roozbeh Mottaghi

NeurIPSW 2024 Situated Instruction Following Under Ambiguous Human Intent So Yeon Min, Xavier Puig, Devendra Singh Chaplot, Tsung-Yen Yang, Akshara Rai, Priyam Parashar, Russ Salakhutdinov, Yonatan Bisk, Roozbeh Mottaghi

NeurIPSW 2024 Situated Instruction Following Under Ambiguous Human Intent So Yeon Min, Xavier Puig, Devendra Singh Chaplot, Tsung-Yen Yang, Akshara Rai, Priyam Parashar, Russ Salakhutdinov, Yonatan Bisk, Roozbeh Mottaghi

ECCV 2024 Skews in the Phenomenon Space Hinder Generalization in Text-to-Image Generation Yingshan Chang, Yasi Zhang, Zhiyuan Fang, Ying Nian Wu, Yonatan Bisk, Feng Gao

ICLR 2024 WebArena: A Realistic Web Environment for Building Autonomous Agents Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, Graham Neubig

ICLR 2023 Computational Language Acquisition with Theory of Mind Andy Liu, Hao Zhu, Emmy Liu, Yonatan Bisk, Graham Neubig

CVPR 2023 EXCALIBUR: Encouraging and Evaluating Embodied Exploration Hao Zhu, Raghav Kapoor, So Yeon Min, Winson Han, Jiatai Li, Kaiwen Geng, Graham Neubig, Yonatan Bisk, Aniruddha Kembhavi, Luca Weihs

CoRL 2023 HomeRobot: Open-Vocabulary Mobile Manipulation Sriram Yenamandra, Arun Ramachandran, Karmesh Yadav, Austin S. Wang, Mukul Khanna, Theophile Gervet, Tsung-Yen Yang, Vidhi Jain, Alexander Clegg, John M. Turner, Zsolt Kira, Manolis Savva, Angel X. Chang, Devendra Singh Chaplot, Dhruv Batra, Roozbeh Mottaghi, Yonatan Bisk, Chris Paxton

NeurIPSW 2023 HomeRobot: Open-Vocabulary Mobile Manipulation Sriram Yenamandra, Arun Ramachandran, Karmesh Yadav, Austin S Wang, Mukul Khanna, Theophile Gervet, Tsung-Yen Yang, Vidhi Jain, Alexander Clegg, John M Turner, Zsolt Kira, Manolis Savva, Angel X Chang, Devendra Singh Chaplot, Dhruv Batra, Roozbeh Mottaghi, Yonatan Bisk, Chris Paxton

CoRL 2023 SLAP: Spatial-Language Attention Policies Priyam Parashar, Vidhi Jain, Xiaohan Zhang, Jay Vakil, Sam Powers, Yonatan Bisk, Chris Paxton

NeurIPSW 2023 SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, Maarten Sap

NeurIPS 2023 SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs Lijun Yu, Yong Cheng, Zhiruo Wang, Vivek Kumar, Wolfgang Macherey, Yanping Huang, David A. Ross, Irfan A. Essa, Yonatan Bisk, Ming-Hsuan Yang, Kevin P. Murphy, Alexander Hauptmann, Lu Jiang

NeurIPS 2023 SPRING: Studying Papers and Reasoning to Play Games Yue Wu, So Yeon Min, Shrimai Prabhumoye, Yonatan Bisk, Ruslan Salakhutdinov, Amos Azaria, Tom M. Mitchell, Yuanzhi Li

ICMLW 2023 The Framework Tax: Disparities Between Inference Efficiency in Research and Deployment Jared Fernandez, Jacob Kahn, Clara Na, Yonatan Bisk, Emma Strubell

TMLR 2023 Training Vision-Language Transformers from Captions Liangke Gui, Yingshan Chang, Qiuyuan Huang, Subhojit Som, Alexander G Hauptmann, Jianfeng Gao, Yonatan Bisk

NeurIPSW 2023 WebArena: A Realistic Web Environment for Building Autonomous Agents Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, Graham Neubig

NeurIPSW 2023 WebArena: A Realistic Web Environment for Building Autonomous Agents Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, Graham Neubig

ICML 2022 A Framework for Learning to Request Rich and Contextually Useful Information from Humans Khanh X Nguyen, Yonatan Bisk, Hal Daumé Iii

ICLR 2022 FILM: Following Instructions in Language with Modular Methods So Yeon Min, Devendra Singh Chaplot, Pradeep Kumar Ravikumar, Yonatan Bisk, Ruslan Salakhutdinov

NeurIPSW 2022 MAEA: Multimodal Attribution for Embodied AI Vidhi Jain, Sahiti Yerramilli, Jayant Sravan Tamarapalli, Yonatan Bisk

ICML 2022 Symmetric Machine Theory of Mind Melanie Sclar, Graham Neubig, Yonatan Bisk

NeurIPSW 2022 Tackling AlfWorld with Action Attention and Common Sense from Language Models Yue Wu, So Yeon Min, Yonatan Bisk, Ruslan Salakhutdinov, Shrimai Prabhumoye

CoRL 2022 Transformers Are Adaptable Task Planners Vidhi Jain, Yixin Lin, Eric Undersander, Yonatan Bisk, Akshara Rai

CVPR 2022 WebQA: Multihop and Multimodal QA Yingshan Chang, Mridu Narang, Hisami Suzuki, Guihong Cao, Jianfeng Gao, Yonatan Bisk

ICLR 2021 ALFWorld: Aligning Text and Embodied Environments for Interactive Learning Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Cote, Yonatan Bisk, Adam Trischler, Matthew Hausknecht

ICML 2021 Few-Shot Language Coordination by Modeling Theory of Mind Hao Zhu, Graham Neubig, Yonatan Bisk

AAAI 2021 Knowledge-Driven Data Construction for Zero-Shot Evaluation in Commonsense Question Answering Kaixin Ma, Filip Ilievski, Jonathan Francis, Yonatan Bisk, Eric Nyberg, Alessandro Oltramari

CoRL 2021 Language Grounding with 3D Objects Jesse Thomason, Mohit Shridhar, Yonatan Bisk, Chris Paxton, Luke Zettlemoyer

ICCV 2021 TACo: Token-Aware Cascade Contrastive Learning for Video-Text Alignment Jianwei Yang, Yonatan Bisk, Jianfeng Gao

AAAI 2020 PIQA: Reasoning About Physical Commonsense in Natural Language Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, Yejin Choi

NeurIPS 2019 Defending Against Neural Fake News Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, Yejin Choi

AAAI 2018 Learning Interpretable Spatial Operations in a Rich 3D Blocks World Yonatan Bisk, Kevin J. Shih, Yejin Choi, Daniel Marcu

ICLR 2018 Synthetic and Natural Noise Both Break Neural Machine Translation Yonatan Belinkov, Yonatan Bisk

AAAI 2012 Simple Robust Grammar Induction with Combinatory Categorial Grammars Yonatan Bisk, Julia Hockenmaier