Song, Xia

11 publications

ICML 2025 POROver: Improving Safety and Reducing Overrefusal in Large Language Models with Overgeneration and Preference Optimization Batuhan K. Karaman, Ishmam Zabir, Alon Benhaim, Vishrav Chaudhary, Mert R. Sabuncu, Xia Song

ICLRW 2025 S2-Attention: Hardware-Aware Context Sharding Among Attention Heads Xihui Lin, Yunan Zhang, Suyu Ge, Liliang Ren, Barun Patra, Vishrav Chaudhary, Hao Peng, Xia Song

ICLR 2025 Scaling Optimal LR Across Token Horizons Johan Bjorck, Alon Benhaim, Vishrav Chaudhary, Furu Wei, Xia Song

NeurIPSW 2024 WildFeedback: Aligning LLMs with In-Situ User Interactions and Feedback Taiwei Shi, Zhuoer Wang, Longqi Yang, Ying-Chun Lin, Zexue He, Mengting Wan, Pei Zhou, Sujay Kumar Jauhar, Xiaofeng Xu, Xia Song, Jennifer Neville

NeurIPS 2023 Language Is Not All You Need: Aligning Perception with Language Models Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, Qiang Liu, Kriti Aggarwal, Zewen Chi, Nils Bjorck, Vishrav Chaudhary, Subhojit Som, Xia Song, Furu Wei

ICML 2023 Magneto: A Foundation Transformer Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei

NeurIPS 2022 On the Representation Collapse of Sparse Mixture of Experts Zewen Chi, Li Dong, Shaohan Huang, Damai Dai, Shuming Ma, Barun Patra, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, Heyan Huang, Furu Wei

ICLR 2022 Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators Yu Meng, Chenyan Xiong, Payal Bajaj, Saurabh Tiwary, Paul N. Bennett, Jiawei Han, Xia Song

NeurIPS 2021 COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng, Chenyan Xiong, Payal Bajaj, Saurabh Tiwary, Paul Bennett, Jiawei Han, Xia Song

NeurIPS 2020 Pushing the Limits of Narrow Precision Inferencing at Cloud Scale with Microsoft Floating Point Bita Darvish Rouhani, Daniel Lo, Ritchie Zhao, Ming Liu, Jeremy Fowers, Kalin Ovtcharov, Anna Vinogradsky, Sarah Massengill, Lita Yang, Ray Bittner, Alessandro Forin, Haishan Zhu, Taesik Na, Prerak Patel, Shuai Che, Lok Chand Koppaka, Xia Song, Subhojit Som, Kaustav Das, Saurabh T, Steve Reinhardt, Sitaram Lanka, Eric Chung, Doug Burger

ICLR 2020 Transformer-XH: Multi-Evidence Reasoning with eXtra Hop Attention Chen Zhao, Chenyan Xiong, Corby Rosset, Xia Song, Paul Bennett, Saurabh Tiwary