Firat, Orhan

23 publications

NeurIPS 2024 Scaling Sign Language Translation Biao Zhang, Garrett Tanzer, Orhan Firat

ICLR 2024 When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method Biao Zhang, Zhongtao Liu, Colin Cherry, Orhan Firat

NeurIPS 2023 Binarized Neural Machine Translation Yichi Zhang, Ankush Garg, Yuan Cao, Lukasz Lew, Behrooz Ghorbani, Zhiru Zhang, Orhan Firat

NeurIPS 2023 Block-State Transformers Jonathan Pilault, Mahan Fathi, Orhan Firat, Chris Pal, Pierre-Luc Bacon, Ross Goroshin

ICMLW 2023 Interactive-Chain-Prompting: Ambiguity Resolution for Crosslingual Conditional Generation with Interaction Jonathan Pilault, Xavier Garcia, Arthur Brazinskas, Orhan Firat

NeurIPS 2023 MADLAD-400: A Multilingual and Document-Level Large Audited Dataset Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, Orhan Firat

NeurIPS 2023 Order Matters in the Presence of Dataset Imbalance for Multilingual Learning Dami Choi, Derrick Xin, Hamid Dadkhahi, Justin Gilmer, Ankush Garg, Orhan Firat, Chih-Kuan Yeh, Andrew M Dai, Behrooz Ghorbani

JMLR 2023 PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, Noah Fiedel

ICML 2023 Scaling Laws for Multilingual Neural Machine Translation Patrick Fernandes, Behrooz Ghorbani, Xavier Garcia, Markus Freitag, Orhan Firat

ICML 2023 The Unreasonable Effectiveness of Few-Shot Learning for Machine Translation Xavier Garcia, Yamini Bansal, Colin Cherry, George Foster, Maxim Krikun, Melvin Johnson, Orhan Firat

ICLR 2023 UniMax: Fairer and More Effective Language Sampling for Large-Scale Multilingual Pretraining Hyung Won Chung, Xavier Garcia, Adam Roberts, Yi Tay, Orhan Firat, Sharan Narang, Noah Constant

ICLR 2022 A Loss Curvature Perspective on Training Instabilities of Deep Learning Models Justin Gilmer, Behrooz Ghorbani, Ankush Garg, Sneha Kudugunta, Behnam Neyshabur, David Cardoze, George Edward Dahl, Zachary Nado, Orhan Firat

ICML 2022 Data Scaling Laws in NMT: The Effect of Noise and Architecture Yamini Bansal, Behrooz Ghorbani, Ankush Garg, Biao Zhang, Colin Cherry, Behnam Neyshabur, Orhan Firat

NeurIPS 2022 Do Current Multi-Task Optimization Methods in Deep Learning Even Help? Derrick Xin, Behrooz Ghorbani, Justin Gilmer, Ankush Garg, Orhan Firat

ICML 2022 Examining Scaling and Transfer of Language Model Architectures for Machine Translation Biao Zhang, Behrooz Ghorbani, Ankur Bapna, Yong Cheng, Xavier Garcia, Jonathan Shen, Orhan Firat

ICML 2022 GLaM: Efficient Scaling of Language Models with Mixture-of-Experts Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten P Bosma, Zongwei Zhou, Tao Wang, Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen Meier-Hellstern, Toju Duke, Lucas Dixon, Kun Zhang, Quoc Le, Yonghui Wu, Zhifeng Chen, Claire Cui

ICLR 2022 Scaling Laws for Neural Machine Translation Behrooz Ghorbani, Orhan Firat, Markus Freitag, Ankur Bapna, Maxim Krikun, Xavier Garcia, Ciprian Chelba, Colin Cherry

ICLR 2021 GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, Zhifeng Chen

ICLR 2021 Gradient Vaccine: Investigating and Improving Multi-Task Optimization in Massively Multilingual Models Zirui Wang, Yulia Tsvetkov, Orhan Firat, Yuan Cao

ICLR 2021 Share or Not? Learning to Schedule Language-Specific Capacity for Multilingual Translation Biao Zhang, Ankur Bapna, Rico Sennrich, Orhan Firat

AAAI 2020 Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation Aditya Siddhant, Melvin Johnson, Henry Tsai, Naveen Ari, Jason Riesa, Ankur Bapna, Orhan Firat, Karthik Raman

ICML 2020 XTREME: A Massively Multilingual Multi-Task Benchmark for Evaluating Cross-Lingual Generalisation Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, Melvin Johnson

NeurIPS 2019 GPipe: Efficient Training of Giant Neural Networks Using Pipeline Parallelism Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, Yonghui Wu, Zhifeng Chen