ML Anthology

Sun, Ruoyu

43 publications

ICLR 2025 Adam-Mini: Use Fewer Learning Rates to Gain More Yushun Zhang, Congliang Chen, Ziniu Li, Tian Ding, Chenwei Wu, Diederik P Kingma, Yinyu Ye, Zhi-Quan Luo, Ruoyu Sun
TMLR 2025 Exploring and Improving Initialization for Deep Graph Neural Networks: A Signal Propagation Perspective Senmiao Wang, Yupeng Chen, Yushun Zhang, Ruoyu Sun, Tian Ding
TMLR 2025 MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen, Senmiao Wang, Yushun Zhang, Zhihang Lin, Haozhe Zhang, Weijian Sun, Tian Ding, Ruoyu Sun
TMLR 2025 On Representing Convex Quadratically Constrained Quadratic Programs via Graph Neural Networks Chenyang Wu, Qian Chen, Akang Wang, Tian Ding, Ruoyu Sun, Wenguo Yang, Qingjiang Shi
ICLRW 2025 On-Premises LLM Deployment Demands a Middle Path: Preserving Privacy Without Sacrificing Model Confidentiality Hanbo Huang, Yihan Li, Bowen Jiang, Lin Liu, Bo Jiang, Ruoyu Sun, Zhuotao Liu, Shiyu Liang
ICLR 2025 Preserving Diversity in Supervised Fine-Tuning of Large Language Models Ziniu Li, Congliang Chen, Tian Xu, Zeyu Qin, Jiancong Xiao, Zhi-Quan Luo, Ruoyu Sun
NeurIPS 2025 Teaching Language Models to Reason with Tools Chengpeng Li, Zhengyang Tang, Ziniu Li, Mingfeng Xue, Keqin Bao, Tian Ding, Ruoyu Sun, Benyou Wang, Xiang Wang, Junyang Lin, Dayiheng Liu
ICLR 2025 Towards Explaining the Power of Constant-Depth Graph Neural Networks for Structured Linear Programming Qian Li, Minghui Ouyang, Tian Ding, Yuyi Wang, Qingjiang Shi, Ruoyu Sun
ICLR 2025 When GNNs Meet Symmetry in ILPs: An Orbit-Based Feature Augmentation Approach Qian Chen, Lei Li, Qian Li, Jianghua Wu, Akang Wang, Ruoyu Sun, Xiaodong Luo, Tsung-Hui Chang, Qingjiang Shi
ICMLW 2024 Adam-Mini: Use Fewer Learning Rates to Gain More Yushun Zhang, Congliang Chen, Ziniu Li, Tian Ding, Chenwei Wu, Yinyu Ye, Zhi-Quan Luo, Ruoyu Sun
COLT 2024 Bridging the Gap: Rademacher Complexity in Robust and Standard Generalization Jiancong Xiao, Ruoyu Sun, Qi Long, Weijie Su
ICMLW 2024 Enhancing the Resilience of LLMs Against Grey-Box Extractions Hanbo Huang, Yihan Li, Bowen Jiang, Bo Jiang, Lin Liu, Zhuotao Liu, Ruoyu Sun, Shiyu Liang
NeurIPSW 2024 Entropic Distribution Matching for Supervised Fine-Tuning of LLMs: Less Overfitting and Better Diversity Ziniu Li, Congliang Chen, Tian Xu, Zeyu Qin, Jiancong Xiao, Ruoyu Sun, Zhi-Quan Luo
NeurIPSW 2024 GaLore-Mini: Low Rank Gradient Learning with Fewer Learning Rates Weihao Huang, Zhenyu Zhang, Yushun Zhang, Zhi-Quan Luo, Ruoyu Sun, Zhangyang Wang
ICML 2024 How Graph Neural Networks Learn: Lessons from Training Dynamics Chenxiao Yang, Qitian Wu, David Wipf, Ruoyu Sun, Junchi Yan
ICLR 2024 LEMON: Lossless Model Expansion Yite Wang, Jiahao Su, Hanlin Lu, Cong Xie, Tianyi Liu, Jianbo Yuan, Haibin Lin, Ruoyu Sun, Hongxia Yang
NeurIPS 2024 On the Power of Small-Size Graph Neural Networks for Linear Programming Qian Li, Tian Ding, Linxin Yang, Minghui Ouyang, Qingjiang Shi, Ruoyu Sun
ICML 2024 PDHG-Unrolled Learning-to-Optimize Method for Large-Scale Linear Programming Bingheng Li, Linxin Yang, Yupeng Chen, Senmiao Wang, Haitao Mao, Qian Chen, Yao Ma, Akang Wang, Tian Ding, Jiliang Tang, Ruoyu Sun
ICML 2024 ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models Ziniu Li, Tian Xu, Yushun Zhang, Zhihang Lin, Yang Yu, Ruoyu Sun, Zhi-Quan Luo
NeurIPS 2024 SymILO: A Symmetry-Aware Learning Framework for Integer Linear Optimization Qian Chen, Tianjian Zhang, Linxin Yang, Qingyu Han, Akang Wang, Ruoyu Sun, Xiaodong Luo, Tsung-Hui Chang
NeurIPS 2024 Why Transformers Need Adam: A Hessian Perspective Yushun Zhang, Congliang Chen, Tian Ding, Ziniu Li, Ruoyu Sun, Zhi-Quan Luo
ICMLW 2024 Why Transformers Need Adam: A Hessian Perspective Yushun Zhang, Congliang Chen, Tian Ding, Ziniu Li, Ruoyu Sun, Zhi-Quan Luo
ICLR 2023 A GNN-Guided Predict-and-Search Framework for Mixed-Integer Linear Programming Qingyu Han, Linxin Yang, Qian Chen, Xiang Zhou, Dong Zhang, Akang Wang, Ruoyu Sun, Xiaodong Luo
NeurIPS 2023 Balanced Training for Sparse GANs Yite Wang, Jing Wu, Naira Hovakimyan, Ruoyu Sun
ICMLW 2023 Breaking the Curse of Depth in Graph Convolutional Networks via Refined Initialization Strategy Senmiao Wang, Yupeng Chen, Yushun Zhang, Tian Ding, Ruoyu Sun
ICLR 2023 NTK-SAP: Improving Neural Network Pruning by Aligning Training Dynamics Yite Wang, Dawei Li, Ruoyu Sun
ICMLW 2023 PAC-Bayesian Adversarially Robust Generalization Bounds for Deep Neural Networks Jiancong Xiao, Ruoyu Sun, Zhi-Quan Luo
NeurIPS 2023 PAC-Bayesian Spectrally-Normalized Bounds for Adversarially Robust Generalization Jiancong Xiao, Ruoyu Sun, Zhi-Quan Luo
NeurIPS 2022 Adam Can Converge Without Any Modification on Update Rules Yushun Zhang, Congliang Chen, Naichen Shi, Ruoyu Sun, Zhi-Quan Luo
NeurIPS 2022 DigGAN: Discriminator gradIent Gap Regularization for GAN Training with Limited Data Tiantian Fang, Ruoyu Sun, Alex Schwing
NeurIPS 2022 Does Momentum Change the Implicit Regularization on Separable Data? Bohan Wang, Qi Meng, Huishuai Zhang, Ruoyu Sun, Wei Chen, Zhi-Ming Ma, Tie-Yan Liu
CVPR 2022 Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning Haoxiang Wang, Yite Wang, Ruoyu Sun, Bo Li
NeurIPS 2022 Stability Analysis and Generalization Bounds of Adversarial Training Jiancong Xiao, Yanbo Fan, Ruoyu Sun, Jue Wang, Zhi-Quan Luo
NeurIPS 2021 Faster Directional Convergence of Linear Neural Networks Under Spherically Symmetric Data Dachao Lin, Ruoyu Sun, Zhihua Zhang
AAAI 2021 PenDer: Incorporating Shape Constraints via Penalized Derivatives Akhil Gupta, Lavanya Marla, Ruoyu Sun, Naman Shukla, Arinbjörn Kolbeinsson
ICLR 2021 RMSProp Converges with Proper Hyper-Parameter Naichen Shi, Dawei Li, Mingyi Hong, Ruoyu Sun
NeurIPS 2021 When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work Jiawei Zhang, Yushun Zhang, Mingyi Hong, Ruoyu Sun, Zhi-Quan Luo
NeurIPS 2020 A Single-Loop Smoothed Gradient Descent-Ascent Algorithm for Nonconvex-Concave Min-Max Problems Jiawei Zhang, Peijun Xiao, Ruoyu Sun, Zhiquan Luo
NeurIPS 2020 Towards a Better Global Loss Landscape of GANs Ruoyu Sun, Tiantian Fang, Alexander Schwing
ICLR 2019 On the Convergence of a Class of Adam-Type Algorithms for Non-Convex Optimization Xiangyi Chen, Sijia Liu, Ruoyu Sun, Mingyi Hong
NeurIPS 2018 Adding One Neuron Can Eliminate All Bad Local Minima Shiyu Liang, Ruoyu Sun, Jason Lee, R. Srikant
ICML 2018 Understanding the Loss Surface of Neural Networks for Binary Classification Shiyu Liang, Ruoyu Sun, Yixuan Li, Rayadurgam Srikant
NeurIPS 2015 Improved Iteration Complexity Bounds of Cyclic Block Coordinate Descent for Convex Problems Ruoyu Sun, Mingyi Hong