Frankle, Jonathan

27 publications

ICML 2024 Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws Nikhil Sardana, Jacob Portes, Sasha Doubov, Jonathan Frankle

CVPR 2024 CommonCanvas: Open Diffusion Models Trained on Creative-Commons Images Aaron Gokaslan, A. Feder Cooper, Jasmine Collins, Landan Seguin, Austin Jacobson, Mihir Patel, Jonathan Frankle, Cory Stephenson, Volodymyr Kuleshov

ICMLW 2024 Does Your Data Spark Joy? Performance Gains from Domain Upsampling at the End of Training Cody Blakeney, Mansheej Paul, Brett W. Larsen, Sean Owen, Jonathan Frankle

TMLR 2024 LoRA Learns Less and Forgets Less Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, John Patrick Cunningham

NeurIPS 2023 MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining Jacob Portes, Alexander Trott, Sam Havens, Daniel King, Abhinav Venigalla, Moin Nadeem, Nikhil Sardana, Daya Khudia, Jonathan Frankle

ICMLW 2023 MosaicBERT: How to Train BERT with a Lunch Money Budget Jacob Portes, Alexander R Trott, Sam Havens, Daniel King, Abhinav Venigalla, Moin Nadeem, Nikhil Sardana, Daya Khudia, Jonathan Frankle

ICMLW 2023 Predicting Task Forgetting in Large Language Models Anat Kleiman, Jonathan Frankle, Sham M. Kakade, Mansheej Paul

ICLR 2023 Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? Mansheej Paul, Feng Chen, Brett W. Larsen, Jonathan Frankle, Surya Ganguli, Gintare Karolina Dziugaite

NeurIPSW 2022 Fast Benchmarking of Accuracy vs. Training Time with Cyclic Learning Rates Jacob Portes, Davis Blalock, Cory Stephenson, Jonathan Frankle

ICMLW 2022 Knowledge Distillation for Efficient Sequences of Training Runs Xingyu Liu, Alexander Leonardi, Lu Yu, Christopher Gilmer-Hill, Matthew L Leavitt, Jonathan Frankle

NeurIPS 2022 Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks Mansheej Paul, Brett Larsen, Surya Ganguli, Jonathan Frankle, Gintare Karolina Dziugaite

NeurIPSW 2022 On Convexity and Linear Mode Connectivity in Neural Networks David Yunis, Kumar Kshitij Patel, Pedro Henrique Pamplona Savarese, Gal Vardi, Jonathan Frankle, Matthew Walter, Karen Livescu, Michael Maire

ICMLW 2022 Pre-Training on a Data Diet: Identifying Sufficient Examples for Early Training Mansheej Paul, Brett W Larsen, Surya Ganguli, Jonathan Frankle, Gintare Karolina Dziugaite

NeurIPS 2022 Pruning’s Effect on Generalization Through the Lens of Training and Regularization Tian Jin, Michael Carbin, Dan Roy, Jonathan Frankle, Gintare Karolina Dziugaite

NeurIPSW 2022 The Effect of Data Dimensionality on Neural Network Prunability Zachary Ankner, Alex Renda, Gintare Karolina Dziugaite, Jonathan Frankle, Tian Jin

NeurIPSW 2022 Unmasking the Lottery Ticket Hypothesis: Efficient Adaptive Pruning for Finding Winning Tickets Mansheej Paul, Feng Chen, Brett W. Larsen, Jonathan Frankle, Surya Ganguli, Gintare Karolina Dziugaite

ICML 2022 What Can Linear Interpolation of Neural Network Loss Landscapes Tell Us? Tiffany J Vlaar, Jonathan Frankle

ICML 2021 On the Predictability of Pruning Across Scales Jonathan S Rosenfeld, Jonathan Frankle, Michael Carbin, Nir Shavit

ICLR 2021 Pruning Neural Networks at Initialization: Why Are We Missing the Mark? Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, Michael Carbin

CVPR 2021 The Lottery Tickets Hypothesis for Supervised and Self-Supervised Pre-Training in Computer Vision Models Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Michael Carbin, Zhangyang Wang

ICLR 2021 Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs Jonathan Frankle, David J. Schwab, Ari S. Morcos

ICLR 2020 Comparing Rewinding and Fine-Tuning in Neural Network Pruning Alex Renda, Jonathan Frankle, Michael Carbin

ICML 2020 Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, Michael Carbin

NeurIPSW 2020 Revisiting "Qualitatively Characterizing Neural Network Optimization Problems" Jonathan Frankle

ICLR 2020 The Early Phase of Neural Network Training Jonathan Frankle, David J. Schwab, Ari S. Morcos

NeurIPS 2020 The Lottery Ticket Hypothesis for Pre-Trained BERT Networks Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Zhangyang Wang, Michael Carbin

ICLR 2019 The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle, Michael Carbin