Koh, Pang Wei

45 publications

ICLRW 2025 A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-Level Privacy Leakage Rui Xin, Niloofar Mireshghallah, Shuyue Stella Li, Michael Duan, Hyunwoo Kim, Yejin Choi, Yulia Tsvetkov, Sewoong Oh, Pang Wei Koh

ICML 2025 DataDecide: How to Predict Best Pretraining Data with Small Experiments Ian Magnusson, Nguyen Tai, Ben Bogin, David Heineman, Jena D. Hwang, Luca Soldaini, Akshita Bhagia, Jiacheng Liu, Dirk Groeneveld, Oyvind Tafjord, Noah A. Smith, Pang Wei Koh, Jesse Dodge

NeurIPS 2025 FlexOLMo: Open Language Models for Flexible Data Use Weijia Shi, Akshita Bhagia, Kevin Farhat, Niklas Muennighoff, Jacob Morrison, Evan Pete Walsh, Dustin Schwenk, Shayne Longpre, Jake Poznanski, Allyson Ettinger, Daogao Liu, Margaret Li, Mike Lewis, Wen-tau Yih, Dirk Groeneveld, Luca Soldaini, Kyle Lo, Noah A. Smith, Luke Zettlemoyer, Pang Wei Koh, Hannaneh Hajishirzi, Ali Farhadi, Sewon Min

ICLR 2025 Group-Robust Sample Reweighting for Subpopulation Shifts via Influence Functions Rui Qiao, Zhaoxuan Wu, Jingtan Wang, Pang Wei Koh, Bryan Kian Hsiang Low

ICLR 2025 Language Models Scale Reliably with Over-Training and on Downstream Tasks Samir Yitzhak Gadre, Georgios Smyrnis, Vaishaal Shankar, Suchin Gururangan, Mitchell Wortsman, Rulin Shao, Jean Mercat, Alex Fang, Jeffrey Li, Sedrick Keh, Rui Xin, Marianna Nezhurina, Igor Vasiljevic, Luca Soldaini, Jenia Jitsev, Alex Dimakis, Gabriel Ilharco, Pang Wei Koh, Shuran Song, Thomas Kollar, Yair Carmon, Achal Dave, Reinhard Heckel, Niklas Muennighoff, Ludwig Schmidt

ICML 2025 NICE Data Selection for Instruction Tuning in LLMs with Non-Differentiable Evaluation Metric Jingtan Wang, Xiaoqiang Lin, Rui Qiao, Pang Wei Koh, Chuan-Sheng Foo, Bryan Kian Hsiang Low

ICLRW 2025 NICE: Non-Differentiable Evaluation Metric-Based Data Selection for Instruction Tuning Jingtan Wang, Xiaoqiang Lin, Rui Qiao, Pang Wei Koh, Chuan-Sheng Foo, Bryan Kian Hsiang Low

ICLR 2025 OLMoE: Open Mixture-of-Experts Language Models Niklas Muennighoff, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Jacob Morrison, Sewon Min, Weijia Shi, Evan Pete Walsh, Oyvind Tafjord, Nathan Lambert, Yuling Gu, Shane Arora, Akshita Bhagia, Dustin Schwenk, David Wadden, Alexander Wettig, Binyuan Hui, Tim Dettmers, Douwe Kiela, Ali Farhadi, Noah A. Smith, Pang Wei Koh, Amanpreet Singh, Hannaneh Hajishirzi

CVPR 2025 PLeaS - Merging Models with Permutations and Least Squares Anshul Nasery, Jonathan Hayase, Pang Wei Koh, Sewoong Oh

NeurIPS 2025 Precise Information Control in Long-Form Text Generation Jacqueline He, Howard Yen, Margaret Li, Shuyue Stella Li, Zhiyuan Zeng, Weijia Shi, Yulia Tsvetkov, Danqi Chen, Pang Wei Koh, Luke Zettlemoyer

TMLR 2025 Reliable and Responsible Foundation Models Xinyu Yang, Junlin Han, Rishi Bommasani, Jinqi Luo, Wenjie Qu, Wangchunshu Zhou, Adel Bibi, Xiyao Wang, Jaehong Yoon, Elias Stengel-Eskin, Shengbang Tong, Lingfeng Shen, Rafael Rafailov, Runjia Li, Zhaoyang Wang, Yiyang Zhou, Chenhang Cui, Yu Wang, Wenhao Zheng, Huichi Zhou, Jindong Gu, Zhaorun Chen, Peng Xia, Tony Lee, Thomas P Zollo, Vikash Sehwag, Jixuan Leng, Jiuhai Chen, Yuxin Wen, Huan Zhang, Zhun Deng, Linjun Zhang, Pavel Izmailov, Pang Wei Koh, Yulia Tsvetkov, Andrew Gordon Wilson, Jiaheng Zhang, James Zou, Cihang Xie, Hao Wang, Philip Torr, Julian McAuley, David Alvarez-Melis, Florian Tramèr, Kaidi Xu, Suman Jana, Chris Callison-Burch, Rene Vidal, Filippos Kokkinos, Mohit Bansal, Beidi Chen, Huaxiu Yao

ICML 2025 S4S: Solving for a Fast Diffusion Model Solver Eric Frankel, Sitan Chen, Jerry Li, Pang Wei Koh, Lillian J. Ratliff, Sewoong Oh

ICLRW 2025 The Delta Learning Hypothesis: Preference Tuning on Weak Data Can Yield Strong Gains Scott Geng, Hamish Ivison, Chun-Liang Li, Maarten Sap, Jerry Li, Ranjay Krishna, Pang Wei Koh

NeurIPSW 2024 A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-Level Privacy Leakage Rui Xin, Niloofar Mireshghallah, Shuyue Stella Li, Hyunwoo Kim, Michael Duan, Yejin Choi, Yulia Tsvetkov, Sewoong Oh, Pang Wei Koh

NeurIPSW 2024 A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-Level Privacy Leakage Rui Xin, Niloofar Mireshghallah, Shuyue Stella Li, Michael Duan, Hyunwoo Kim, Yejin Choi, Yulia Tsvetkov, Sewoong Oh, Pang Wei Koh

NeurIPSW 2024 CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generation Tong Chen, Akari Asai, Niloofar Mireshghallah, Sewon Min, James Grimmelmann, Yejin Choi, Hannaneh Hajishirzi, Luke Zettlemoyer, Pang Wei Koh

NeurIPS 2024 DataComp-LM: In Search of the Next Generation of Training Sets for Language Models Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Keh, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, Reinhard Heckel, Jean Mercat, Mayee Chen, Suchin Gururangan, Mitchell Wortsman, Alon Albalak, Yonatan Bitton, Marianna Nezhurina, Amro Abbas, Cheng-Yu Hsieh, Dhruba Ghosh, Josh Gardner, Maciej Kilian, Hanlin Zhang, Rulin Shao, Sarah Pratt, Sunny Sanyal, Gabriel Ilharco, Giannis Daras, Kalyani Marathe, Aaron Gokaslan, Jieyu Zhang, Khyathi Chandu, Thao Nguyen, Igor Vasiljevic, Sham Kakade, Shuran Song, Sujay Sanghavi, Fartash Faghri, Sewoong Oh, Luke Zettlemoyer, Kyle Lo, Alaaeldin El-Nouby, Hadi Pouransari, Alexander Toshev, Stephanie Wang, Dirk Groeneveld, Luca Soldaini, Pang Wei Koh, Jenia Jitsev, Thomas Kollar, Alexandros G. Dimakis, Yair Carmon, Achal Dave, Ludwig Schmidt, Vaishaal Shankar

ICLR 2024 Improving Domain Generalization with Domain Relations Huaxiu Yao, Xinyu Yang, Xinyi Pan, Shengchao Liu, Pang Wei Koh, Chelsea Finn

NeurIPS 2024 MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan S. Ilgen, Emma Pierson, Pang Wei Koh, Yulia Tsvetkov

NeurIPS 2024 Multilingual Diversity Improves Vision-Language Representations Thao Nguyen, Matthew Wallingford, Sebastin Santy, Wei-Chiu Ma, Sewoong Oh, Ludwig Schmidt, Pang Wei Koh, Ranjay Krishna

NeurIPS 2024 Scaling Retrieval-Based Language Models with a Trillion-Token Datastore Rulin Shao, Jacqueline He, Akari Asai, Weijia Shi, Tim Dettmers, Sewon Min, Luke Zettlemoyer, Pang Wei Koh

ICLR 2024 The Generative AI Paradox: “What It Can Create, It May Not Understand” Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian Fisher, Abhilasha Ravichander, Khyathi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger, Yejin Choi

NeurIPS 2024 The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better Scott Geng, Cheng-Yu Hsieh, Vivek Ramanujan, Matthew Wallingford, Chun-Liang Li, Pang Wei Koh, Ranjay Krishna

NeurIPS 2024 Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in LLMs Zhiyuan Hu, Chumin Liu, Xidong Feng, Yilun Zhao, See-Kiong Ng, Anh Tuan Luu, Junxian He, Pang Wei Koh, Bryan Hooi

ICLRW 2024 Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Models Zhiyuan Hu, Chumin Liu, Xidong Feng, Yilun Zhao, See-Kiong Ng, Anh Tuan Luu, Junxian He, Pang Wei Koh, Bryan Hooi

NeurIPSW 2023 FActScore: Fine-Grained Atomic Evaluation of Factual Precision in Long Form Text Generation Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, Hannaneh Hajishirzi

ICML 2023 Out-of-Domain Robustness via Targeted Augmentations Irena Gao, Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, Percy Liang

NeurIPSW 2023 Retrieval-Based Language Models Using a Multi-Domain Datastore Rulin Shao, Sewon Min, Luke Zettlemoyer, Pang Wei Koh

ICLR 2022 Extending the WILDS Benchmark for Unsupervised Adaptation Shiori Sagawa, Pang Wei Koh, Tony Lee, Irena Gao, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunaga, Henrik Marklund, Sara Beery, Etienne David, Ian Stavness, Wei Guo, Jure Leskovec, Kate Saenko, Tatsunori Hashimoto, Sergey Levine, Chelsea Finn, Percy Liang

NeurIPSW 2022 Out-of-Distribution Robustness via Targeted Augmentations Irena Gao, Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, Percy Liang

MLJ 2022 Stronger Data Poisoning Attacks Break Data Sanitization Defenses Pang Wei Koh, Jacob Steinhardt, Percy Liang

ICMLW 2022 Wild-Time: A Benchmark of In-the-Wild Distribution Shift over Time Huaxiu Yao, Caroline Choi, Yoonho Lee, Pang Wei Koh, Chelsea Finn

ICML 2021 Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, Ludwig Schmidt

NeurIPSW 2021 Extending the WILDS Benchmark for Unsupervised Adaptation Shiori Sagawa, Pang Wei Koh, Tony Lee, Irena Gao, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunaga, Henrik Marklund, Sara Beery, Etienne David, Ian Stavness, Wei Guo, Jure Leskovec, Kate Saenko, Tatsunori Hashimoto, Sergey Levine, Chelsea Finn, Percy Liang

ICML 2021 Just Train Twice: Improving Group Robustness Without Training Group Information Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, Chelsea Finn

ICLR 2021 Selective Classification Can Magnify Disparities Across Groups Erik Jones, Shiori Sagawa, Pang Wei Koh, Ananya Kumar, Percy Liang

ICML 2020 An Investigation of Why Overparameterization Exacerbates Spurious Correlations Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, Percy Liang

ICML 2020 Concept Bottleneck Models Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, Percy Liang

ICLR 2020 Distributionally Robust Neural Networks Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, Percy Liang

NeurIPSW 2020 Selective Classification Can Magnify Disparities Across Groups Erik Jones, Shiori Sagawa, Pang Wei Koh, Ananya Kumar, Percy Liang

AISTATS 2019 Inferring Multidimensional Rates of Aging from Cross-Sectional Data Emma Pierson, Pang Wei Koh, Tatsunori Hashimoto, Daphne Koller, Jure Leskovec, Nick Eriksson, Percy Liang

ICML 2017 Understanding Black-Box Predictions via Influence Functions Pang Wei Koh, Percy Liang

ICML 2011 Learning Deep Energy Models Jiquan Ngiam, Zhenghao Chen, Pang Wei Koh, Andrew Y. Ng

ICML 2011 On Random Weights and Unsupervised Feature Learning Andrew M. Saxe, Pang Wei Koh, Zhenghao Chen, Maneesh Bhand, Bipin Suresh, Andrew Y. Ng