He, Pengcheng

22 publications

ICML 2025 Deep Reinforcement Learning from Hierarchical Preference Design Alexander Bukharin, Yixiao Li, Pengcheng He, Tuo Zhao

ICLR 2024 DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, Pengcheng He

ICLR 2024 Learning Stackable and Skippable LEGO Bricks for Efficient, Reconfigurable, and Variable-Resolution Diffusion Modeling Huangjie Zheng, Zhendong Wang, Jianbo Yuan, Guanghan Ning, Pengcheng He, Quanzeng You, Hongxia Yang, Mingyuan Zhou

ICLR 2024 LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models Yixiao Li, Yifan Yu, Chen Liang, Nikos Karampatziakis, Pengcheng He, Weizhu Chen, Tuo Zhao

ICLR 2024 Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective Ming Zhong, Chenxin An, Weizhu Chen, Jiawei Han, Pengcheng He

ICML 2024 Switchable Decision: Dynamic Neural Generation Networks Shujian Zhang, Korawat Tanwisuth, Chengyue Gong, Pengcheng He, Mingyuan Zhou

ICLR 2023 Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, Tuo Zhao

ICLR 2023 DeBERTaV3: Improving DeBERTa Using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing Pengcheng He, Jianfeng Gao, Weizhu Chen

ICLR 2023 Diffusion-GAN: Training GANs with Diffusion Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen, Mingyuan Zhou

NeurIPS 2023 Guiding Large Language Models via Directional Stimulus Prompting Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, Xifeng Yan

ICML 2023 HyperTuning: Toward Adapting Large Language Models Without Back-Propagation Jason Phang, Yi Mao, Pengcheng He, Weizhu Chen

NeurIPS 2023 In-Context Learning Unlocked for Diffusion Models Zhendong Wang, Yifan Jiang, Yadong Lu, Yelong Shen, Pengcheng He, Weizhu Chen, Zhangyang "Atlas" Wang, Mingyuan Zhou

ICML 2023 Less Is More: Task-Aware Layer-Wise Distillation for Language Model Compression Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen, Tuo Zhao

ICML 2023 LoSparse: Structured Compression of Large Language Models Based on Low-Rank and Sparse Approximation Yixiao Li, Yifan Yu, Qingru Zhang, Chen Liang, Pengcheng He, Weizhu Chen, Tuo Zhao

ICML 2023 POUF: Prompt-Oriented Unsupervised Fine-Tuning for Large Pre-Trained Models Korawat Tanwisuth, Shujian Zhang, Huangjie Zheng, Pengcheng He, Mingyuan Zhou

NeurIPS 2023 Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models Zhendong Wang, Yifan Jiang, Huangjie Zheng, Peihao Wang, Pengcheng He, Zhangyang "Atlas" Wang, Weizhu Chen, Mingyuan Zhou

ICLR 2023 Truncated Diffusion Probabilistic Models and Diffusion-Based Adversarial Auto-Encoders Huangjie Zheng, Pengcheng He, Weizhu Chen, Mingyuan Zhou

IJCAI 2022 Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang

ICLR 2022 No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

ICML 2022 PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance Qingru Zhang, Simiao Zuo, Chen Liang, Alexander Bukharin, Pengcheng He, Weizhu Chen, Tuo Zhao

ICLR 2021 DeBERTa: Decoding-Enhanced BERT with Disentangled Attention Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen

ICLR 2020 On the Variance of the Adaptive Learning Rate and Beyond Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han