Ruder, Sebastian

16 publications

NeurIPS 2024 BAM! Just like That: Simple and Efficient Parameter Upcycling for Mixture of Experts Qizhen Zhang, Nikolas Gritsch, Dwaraknath Gnaneshwar, Simon Guo, David Cairuz, Bharat Venkitesh, Jakob Foerster, Phil Blunsom, Sebastian Ruder, Ahmet Üstün, Acyr Locatelli

ICMLW 2024 BAM! Just like That: Simple and Efficient Parameter Upcycling for Mixture of Experts Qizhen Zhang, Nikolas Gritsch, Dwaraknath Gnaneshwar, Simon Guo, David Cairuz, Bharat Venkitesh, Jakob Nicolaus Foerster, Phil Blunsom, Sebastian Ruder, Ahmet Üstün, Acyr Locatelli

ICLR 2023 Language Models Are Multilingual Chain-of-Thought Reasoners Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, Jason Wei

TMLR 2023 Modular Deep Learning Jonas Pfeiffer, Sebastian Ruder, Ivan Vulić, Edoardo Ponti

ICLR 2022 Charformer: Fast Character Transformers via Gradient-Based Subword Tokenization Yi Tay, Vinh Q. Tran, Sebastian Ruder, Jai Gupta, Hyung Won Chung, Dara Bahri, Zhen Qin, Simon Baumgartner, Cong Yu, Donald Metzler

ICLR 2022 ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q. Tran, Dara Bahri, Jianmo Ni, Jai Gupta, Kai Hui, Sebastian Ruder, Donald Metzler

AAAI 2021 Analogy Training Multilingual Encoders Nicolas Garneau, Mareike Hartmann, Anders Sandholm, Sebastian Ruder, Ivan Vulic, Anders Søgaard

NeurIPS 2021 Compacter: Efficient Low-Rank Hypercomplex Adapter Layers Rabeeh Karimi Mahabadi, James Henderson, Sebastian Ruder

ICLR 2021 Long Range Arena : A Benchmark for Efficient Transformers Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, Donald Metzler

NeurIPS 2021 Mind the Gap: Assessing Temporal Generalization in Neural Language Models Angeliki Lazaridou, Adhi Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Tomas Kocisky, Sebastian Ruder, Dani Yogatama, Kris Cao, Susannah Young, Phil Blunsom

ICLR 2021 Rethinking Embedding Coupling in Pre-Trained Language Models Hyung Won Chung, Thibault Fevry, Henry Tsai, Melvin Johnson, Sebastian Ruder

ICML 2020 XTREME: A Massively Multilingual Multi-Task Benchmark for Evaluating Cross-Lingual Generalisation Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, Melvin Johnson

AAAI 2019 A Hierarchical Multi-Task Approach for Learning Embeddings from Semantic Tasks Victor Sanh, Thomas Wolf, Sebastian Ruder

JAIR 2019 A Survey of Cross-Lingual Word Embedding Models Sebastian Ruder, Ivan Vulic, Anders Søgaard

NeurIPS 2019 Episodic Memory in Lifelong Language Learning Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong, Dani Yogatama

AAAI 2019 Latent Multi-Task Architecture Learning Sebastian Ruder, Joachim Bingel, Isabelle Augenstein, Anders Søgaard