Yin, Lu

34 publications

NeurIPS 2025 AlphaDecay: Module-Wise Weight Decay for Heavy-Tailed Balancing in LLMs Di He, Songjun Tu, Ajay Jaiswal, Li Shen, Ganzhao Yuan, Shiwei Liu, Lu Yin

ICML 2025 From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications Ajay Kumar Jaiswal, Yifan Wang, Lu Yin, Shiwei Liu, Runjin Chen, Jiawei Zhao, Ananth Grama, Yuandong Tian, Zhangyang Wang

NeurIPS 2025 GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling Tianhao Chen, Xin Xu, Zijing Liu, Pengxiang Li, Xinyuan Song, Ajay Kumar Jaiswal, Fan Zhang, Jishan Hu, Yang Wang, Hao Chen, Shizhe Diao, Shiwei Liu, Yu Li, Lu Yin, Can Yang

ICML 2025 LIFT the Veil for the Truth: Principal Weights Emerge After Rank Reduction for Reasoning-Focused Supervised Fine-Tuning Zihang Liu, Tianyu Pang, Oleg Balabanov, Chaoqun Yang, Tianjin Huang, Lu Yin, Yaoqing Yang, Shiwei Liu

ICCV 2025 MagShield: Towards Better Robustness in Sparse Inertial Motion Capture Under Magnetic Disturbances Yunzhe Shao, Xinyu Yi, Lu Yin, Shihui Guo, Junhai Yong, Feng Xu

ICLR 2025 Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN Pengxiang Li, Lu Yin, Shiwei Liu

TMLR 2025 Pushing the Limits of Sparsity: A Bag of Tricks for Extreme Pruning Andy Li, Aiden Durrant, Milan Markovic, Tianjin Huang, Souvik Kundu, Tianlong Chen, Lu Yin, Georgios Leontidis

CPAL 2025 Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients Zhenyu Zhang, Ajay Kumar Jaiswal, Lu Yin, Shiwei Liu, Jiawei Zhao, Yuandong Tian, Zhangyang Wang

ICLR 2025 SEBRA : Debiasing Through Self-Guided Bias Ranking Adarsh Kappiyath, Abhra Chaudhuri, Ajay Kumar Jaiswal, Ziquan Liu, Yunpeng Li, Xiatian Zhu, Lu Yin

TMLR 2025 TFAR: A Training-Free Framework for Autonomous Reliable Reasoning in Visual Question Answering Zhuo Zhi, Chen Feng, Adam Daneshmend, Mine Orlu, Andreas Demosthenous, Lu Yin, Da Li, Ziquan Liu, Miguel R. D. Rodrigues

ICLR 2025 TODO: Enhancing LLM Alignment with Ternary Preferences Yuxiang Guo, Lu Yin, Bo Jiang, Jiaqi Zhang

NeurIPS 2025 The Curse of Depth in Large Language Models Wenfang Sun, Xinyuan Song, Pengxiang Li, Lu Yin, Yefeng Zheng, Shiwei Liu

ICLRW 2025 The Curse of Depth in Large Language Models Wenfang Sun, Xinyuan Song, Pengxiang Li, Lu Yin, Yefeng Zheng, Shiwei Liu

ICLRW 2024 $\mathcal{D}^2$-Sparse: Navigating the Low Data Learning Regime with Coupled Sparse Networks Diganta Misra, Niklas Nolte, Sparsha Mishra, Lu Yin

NeurIPS 2024 Accurate and Steady Inertial Pose Estimation Through Sequence Structure Learning and Modulation Yinghao Wu, Chaoran Wang, Lu Yin, Shihui Guo, Yipeng Qin

ICML 2024 Advancing Dynamic Sparse Training by Exploring Optimization Opportunities Jie Ji, Gen Li, Lu Yin, Minghai Qin, Geng Yuan, Linke Guo, Shiwei Liu, Xiaolong Ma

NeurIPS 2024 E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation Boqian Wu, Qiao Xiao, Shiwei Liu, Lu Yin, Mykola Pechenizkiy, Decebal Constantin Mocanu, Maurice van Keulen, Elena Mocanu

ICML 2024 Junk DNA Hypothesis: Pruning Small Pre-Trained Weights $\textit{Irreversibly}$ and $\textit{Monotonically}$ Impairs “Difficult" Downstream Tasks in LLMs Lu Yin, Ajay Kumar Jaiswal, Shiwei Liu, Souvik Kundu, Zhangyang Wang

ICLR 2024 NeurRev: Train Better Sparse Neural Network Practically via Neuron Revitalization Gen Li, Lu Yin, Jie Ji, Wei Niu, Minghai Qin, Bin Ren, Linke Guo, Shiwei Liu, Xiaolong Ma

ICML 2024 Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Gen Li, Ajay Kumar Jaiswal, Mykola Pechenizkiy, Yi Liang, Michael Bendersky, Zhangyang Wang, Shiwei Liu

ICLRW 2024 Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Gen Li, Ajay Kumar Jaiswal, Mykola Pechenizkiy, Yi Liang, Michael Bendersky, Zhangyang Wang, Shiwei Liu

ICML 2023 Are Large Kernels Better Teachers than Transformers for ConvNets? Tianjin Huang, Lu Yin, Zhenyu Zhang, Li Shen, Meng Fang, Mykola Pechenizkiy, Zhangyang Wang, Shiwei Liu

NeurIPS 2023 Dynamic Sparsity Is Channel-Level Sparsity Learner Lu Yin, Gen Li, Meng Fang, Li Shen, Tianjin Huang, Zhangyang "Atlas" Wang, Vlado Menkovski, Xiaolong Ma, Mykola Pechenizkiy, Shiwei Liu

ECML-PKDD 2023 Enhancing Adversarial Training via Reweighting Optimization Trajectory Tianjin Huang, Shiwei Liu, Tianlong Chen, Meng Fang, Li Shen, Vlado Menkovski, Lu Yin, Yulong Pei, Mykola Pechenizkiy

AAAI 2023 Lottery Pools: Winning More by Interpolating Tickets Without Increasing Training or Inference Cost Lu Yin, Shiwei Liu, Meng Fang, Tianjin Huang, Vlado Menkovski, Mykola Pechenizkiy

ECML-PKDD 2023 REST: Enhancing Group Robustness in DNNs Through Reweighted Sparse Training Jiaxu Zhao, Lu Yin, Shiwei Liu, Meng Fang, Mykola Pechenizkiy

TMLR 2023 Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks Zahra Atashgahi, Xuhao Zhang, Neil Kichler, Shiwei Liu, Lu Yin, Mykola Pechenizkiy, Raymond Veldhuis, Decebal Constantin Mocanu

UAI 2022 Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training Lu Yin, Vlado Menkovski, Meng Fang, Tianjin Huang, Yulong Pei, Mykola Pechenizkiy

LoG 2022 You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets Tianjin Huang, Tianlong Chen, Meng Fang, Vlado Menkovski, Jiaxu Zhao, Lu Yin, Yulong Pei, Decebal Constantin Mocanu, Zhangyang Wang, Mykola Pechenizkiy, Shiwei Liu

ICML 2021 Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training Shiwei Liu, Lu Yin, Decebal Constantin Mocanu, Mykola Pechenizkiy

ACML 2021 Hierarchical Semantic Segmentation Using Psychometric Learning Lu Yin, Vlado Menkovski, Shwei Liu, Mykola Pechenizkiy

NeurIPS 2021 Sparse Training via Boosting Pruning Plasticity with Neuroregeneration Shiwei Liu, Tianlong Chen, Xiaohan Chen, Zahra Atashgahi, Lu Yin, Huanyu Kou, Li Shen, Mykola Pechenizkiy, Zhangyang Wang, Decebal Constantin Mocanu

IJCAI 2020 Beyond Labels: Knowledge Elicitation Using Deep Metric Learning and Psychometric Testing Lu Yin

ECML-PKDD 2020 Knowledge Elicitation Using Deep Metric Learning and Psychometric Testing Lu Yin, Vlado Menkovski, Mykola Pechenizkiy