Fergus, Rob

78 publications

ICLR 2025 BALROG: Benchmarking Agentic LLM and VLM Reasoning on Games Davide Paglieri, Bartłomiej Cupiał, Samuel Coward, Ulyana Piterbarg, Maciej Wolczyk, Akbir Khan, Eduardo Pignatelli, Łukasz Kuciński, Lerrel Pinto, Rob Fergus, Jakob Nicolaus Foerster, Jack Parker-Holder, Tim Rocktäschel

ICLRW 2025 D3: A Large Dataset for Training Code Language Models to Act Diff-by-Diff Ulyana Piterbarg, Kanishk Gandhi, Lerrel Pinto, Noah Goodman, Rob Fergus

ICLR 2025 Efficient Exploration and Discriminative World Model Learning with an Object-Centric Abstraction Anthony GX-Chen, Kenneth Marino, Rob Fergus

ICLR 2025 Training Language Models on Synthetic Edit Sequences Improves Code Synthesis Ulyana Piterbarg, Lerrel Pinto, Rob Fergus

NeurIPS 2025 Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Acceleration Ahmed Khaled, Satyen Kale, Arthur Douillard, Chi Jin, Rob Fergus, Manzil Zaheer

ICML 2024 A Fresh Take on Stale Embeddings: Improving Dense Retriever Training with Corrector Networks Nicholas Monath, Will Sussman Grathwohl, Michael Boratko, Rob Fergus, Andrew Mccallum, Manzil Zaheer

ICLR 2024 Adaptive Retrieval and Scalable Indexing for k-NN Search with Cross-Encoders Nishant Yadav, Nicholas Monath, Manzil Zaheer, Rob Fergus, Andrew McCallum

NeurIPS 2024 Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang, Rob Fergus, Yann LeCun, Saining Xie

ICML 2024 Diff History for Neural Language Agents Ulyana Piterbarg, Lerrel Pinto, Rob Fergus

ICML 2024 USTAD: Unified Single-Model Training Achieving Diverse Scores for Information Retrieval Seungyeon Kim, Ankit Singh Rawat, Manzil Zaheer, Wittawat Jitkrittum, Veeranjaneyulu Sadhanala, Sadeep Jayasumana, Aditya Krishna Menon, Rob Fergus, Sanjiv Kumar

ICMLW 2023 Accelerating Exploration and Representation Learning with Offline Pre-Training Bogdan Mazoure, Jake Bruce, Doina Precup, Rob Fergus, Ankit Anand

ICML 2023 Distilling Internet-Scale Vision-Language Models into Embodied Agents Theodore Sumers, Kenneth Marino, Arun Ahuja, Rob Fergus, Ishita Dasgupta

ICLR 2023 Learning About Progress from Experts Jake Bruce, Ankit Anand, Bogdan Mazoure, Rob Fergus

NeurIPS 2023 NetHack Is Hard to Hack Ulyana Piterbarg, Lerrel Pinto, Rob Fergus

ICML 2023 Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC Yilun Du, Conor Durkan, Robin Strudel, Joshua B. Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, Will Sussman Grathwohl

ICLR 2023 Teacher Guided Training: An Efficient Framework for Knowledge Transfer Manzil Zaheer, Ankit Singh Rawat, Seungyeon Kim, Chong You, Himanshu Jain, Andreas Veit, Rob Fergus, Sanjiv Kumar

NeurIPSW 2022 Collaborating with Language Models for Embodied Reasoning Ishita Dasgupta, Christine Kaeser-Chen, Kenneth Marino, Arun Ahuja, Sheila Babayan, Felix Hill, Rob Fergus

NeurIPSW 2022 Collaborating with Language Models for Embodied Reasoning Ishita Dasgupta, Christine Kaeser-Chen, Kenneth Marino, Arun Ahuja, Sheila Babayan, Felix Hill, Rob Fergus

NeurIPS 2022 Learning to Navigate Wikipedia by Taking Random Walks Manzil Zaheer, Kenneth Marino, Will Grathwohl, John Schultz, Wendy Shang, Sheila Babayan, Arun Ahuja, Ishita Dasgupta, Christine Kaeser-Chen, Rob Fergus

ICLR 2022 Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning Denis Yarats, Rob Fergus, Alessandro Lazaric, Lerrel Pinto

NeurIPS 2021 Automatic Data Augmentation for Generalization in Reinforcement Learning Roberta Raileanu, Maxwell Goldstein, Denis Yarats, Ilya Kostrikov, Rob Fergus

ICML 2021 Decoupling Value and Policy for Generalization in Reinforcement Learning Roberta Raileanu, Rob Fergus

ICLR 2021 Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels Denis Yarats, Ilya Kostrikov, Rob Fergus

ICML 2021 Imitation by Predicting Observations Andrew Jaegle, Yury Sulsky, Arun Ahuja, Jake Bruce, Rob Fergus, Greg Wayne

AAAI 2021 Improving Sample Efficiency in Model-Free Reinforcement Learning from Images Denis Yarats, Amy Zhang, Ilya Kostrikov, Brandon Amos, Joelle Pineau, Rob Fergus

NeurIPSW 2021 Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning Denis Yarats, Rob Fergus, Alessandro Lazaric, Lerrel Pinto

ICML 2021 Offline Reinforcement Learning with Fisher Divergence Critic Regularization Ilya Kostrikov, Rob Fergus, Jonathan Tompson, Ofir Nachum

ICML 2021 Reinforcement Learning with Prototypical Representations Denis Yarats, Rob Fergus, Alessandro Lazaric, Lerrel Pinto

ICLRW 2021 Reinforcement Learning with Prototypical Representations Denis Yarats, Rob Fergus, Alessandro Lazaric, Lerrel Pinto

ICLR 2020 Energy-Based Models for Atomic-Resolution Protein Conformations Yilun Du, Joshua Meier, Jerry Ma, Rob Fergus, Alexander Rives

ICML 2020 Fast Adaptation to New Environments via Policy-Dynamics Value Functions Roberta Raileanu, Max Goldstein, Arthur Szlam, Rob Fergus

ICLR 2019 Hierarchical RL Using an Ensemble of Proprioceptive Periodic Policies Kenneth Marino, Abhinav Gupta, Rob Fergus, Arthur Szlam

ICML 2018 Composable Planning with Attributes Amy Zhang, Sainbayar Sukhbaatar, Adam Lerer, Arthur Szlam, Rob Fergus

ICLR 2018 Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play Sainbayar Sukhbaatar, Zeming Lin, Ilya Kostrikov, Gabriel Synnaeve, Arthur Szlam, Rob Fergus

ICML 2018 Modeling Others Using Oneself in Multi-Agent Reinforcement Learning Roberta Raileanu, Emily Denton, Arthur Szlam, Rob Fergus

ICML 2018 Stochastic Video Generation with a Learned Prior Emily Denton, Rob Fergus

CVPRW 2016 Deep End2End Voxel2Voxel Prediction Du Tran, Lubomir D. Bourdev, Rob Fergus, Lorenzo Torresani, Manohar Paluri

NeurIPS 2016 Learning Multiagent Communication with Backpropagation Sainbayar Sukhbaatar, Arthur Szlam, Rob Fergus

ICML 2016 Learning Physical Intuition of Block Towers by Example Adam Lerer, Sam Gross, Rob Fergus

ICML 2016 Learning Simple Algorithms from Examples Wojciech Zaremba, Tomas Mikolov, Armand Joulin, Rob Fergus

CVPR 2015 Beyond Frontal Faces: Improving Person Recognition Using Multiple Cues Ning Zhang, Manohar Paluri, Yaniv Taigman, Rob Fergus, Lubomir Bourdev

NeurIPS 2015 Deep Generative Image Models Using a Laplacian Pyramid of Adversarial Networks Emily L Denton, Soumith Chintala, Arthur Szlam, Rob Fergus

CVPR 2015 End-to-End Integration of a Convolution Network, Deformable Parts Model and Non-Maximum Suppression Li Wan, David Eigen, Rob Fergus

NeurIPS 2015 End-to-End Memory Networks Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob Fergus

ICCV 2015 Improving Image Classification with Location Context Kevin Tang, Manohar Paluri, Li Fei-Fei, Rob Fergus, Lubomir Bourdev

ICCV 2015 Learning Spatiotemporal Features with 3D Convolutional Networks Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, Manohar Paluri

ICLR 2015 Learning from Noisy Labels with Deep Neural Networks Sainbayar Sukhbaatar, Rob Fergus

ICCV 2015 Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture David Eigen, Rob Fergus

CVPR 2015 Web Scale Photo Hash Clustering on a Single Machine Yunchao Gong, Marcin Pawlowski, Fei Yang, Louis Brandy, Lubomir Bourdev, Rob Fergus

NeurIPS 2014 Depth mAP Prediction from a Single Image Using a Multi-Scale Deep Network David Eigen, Christian Puhrsch, Rob Fergus

NeurIPS 2014 Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation Emily L Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, Rob Fergus

ECCV 2014 Instance Segmentation of Indoor Scenes Using a Coverage Loss Nathan Silberman, David A. Sontag, Rob Fergus

ICLR 2014 Intriguing Properties of Neural Networks Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, Rob Fergus

NeurIPS 2014 Learning to Discover Efficient Mathematical Identities Wojciech Zaremba, Karol Kurach, Rob Fergus

ICLR 2014 OverFeat: Integrated Recognition, Localization and Detection Using Convolutional Networks Pierre Sermanet, David Eigen, Xiang Zhang, Michaël Mathieu, Rob Fergus, Yann LeCun

ICLR 2014 Understanding Deep Architectures Using a Recursive Convolutional Network David Eigen, Jason Tyler Rolfe, Rob Fergus, Yann LeCun

ECCV 2014 Visualizing and Understanding Convolutional Networks Matthew D. Zeiler, Rob Fergus

ICML 2013 Regularization of Neural Networks Using DropConnect Li Wan, Matthew Zeiler, Sixin Zhang, Yann Le Cun, Rob Fergus

ICCV 2013 Restoring an Image Taken Through a Window Covered with Dirt or Rain David Eigen, Dilip Krishnan, Rob Fergus

ICLR 2013 Stochastic Pooling for Regularization of Deep Convolutional Neural Networks Matthew D. Zeiler, Rob Fergus

AISTATS 2012 A Hybrid Neural Network-Latent Topic Model Li Wan, Leo Zhu, Rob Fergus

ECCV 2012 Indoor Segmentation and Support Inference from RGBD Images Nathan Silberman, Derek Hoiem, Pushmeet Kohli, Rob Fergus

ECCV 2012 Multidimensional Spectral Hashing Yair Weiss, Rob Fergus, Antonio Torralba

CVPR 2012 Nonparametric Image Parsing Using Adaptive Neighbor Sets David Eigen, Rob Fergus

ICCV 2011 Adaptive Deconvolutional Networks for Mid and High Level Feature Learning Matthew D. Zeiler, Graham W. Taylor, Rob Fergus

CVPR 2011 Blind Deconvolution Using a Normalized Sparsity Measure Dilip Krishnan, Terence Tay, Rob Fergus

NeurIPS 2011 Facial Expression Transfer with Input-Output Temporal Restricted Boltzmann Machines Matthew D. Zeiler, Graham W. Taylor, Leonid Sigal, Iain Matthews, Rob Fergus

CVPR 2011 Learning Invariance Through Imitation Graham W. Taylor, Ian Spiro, Christoph Bregler, Rob Fergus

ECCV 2010 Convolutional Learning of Spatio-Temporal Features Graham W. Taylor, Rob Fergus, Yann LeCun, Christoph Bregler

NeurIPS 2010 Pose-Sensitive Embedding by Nonlinear NCA Regression Graham W. Taylor, Rob Fergus, George Williams, Ian Spiro, Christoph Bregler

NeurIPS 2009 Fast Image Deconvolution Using Hyper-Laplacian Priors Dilip Krishnan, Rob Fergus

CVPR 2009 Learning Invariant Features Through Topographic Filter Maps Koray Kavukcuoglu, Marc'Aurelio Ranzato, Rob Fergus, Yann LeCun

NeurIPS 2009 Semi-Supervised Learning in Gigantic Image Collections Rob Fergus, Yair Weiss, Antonio Torralba

NeurIPS 2008 Spectral Hashing Yair Weiss, Antonio Torralba, Rob Fergus

NeurIPS 2007 Object Recognition by Scene Alignment Bryan Russell, Antonio Torralba, Ce Liu, Rob Fergus, William T. Freeman

CVPR 2004 Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories Li Fei-Fei, Rob Fergus, Pietro Perona

CVPRW 2004 Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories Li Fei-Fei, Rob Fergus, Pietro Perona

NeurIPS 2004 Sampling Methods for Unsupervised Learning Rob Fergus, Andrew Zisserman, Pietro Perona