Han, Song

79 publications

ICLR 2025 COAT: Compressing Optimizer States and Activations for Memory-Efficient FP8 Training Haocheng Xi, Han Cai, Ligeng Zhu, Yao Lu, Kurt Keutzer, Jianfei Chen, Song Han

CVPR 2025 CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han, Chelsea Finn, Ankur Handa, Tsung-Yi Lin, Gordon Wetzstein, Ming-Yu Liu, Donglai Xiang

ICCV 2025 DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Space Junyu Chen, Dongyun Zou, Wenkun He, Junsong Chen, Enze Xie, Song Han, Han Cai

ICCV 2025 DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer Yecheng Wu, Han Cai, Junyu Chen, Zhuoyang Zhang, Enze Xie, Jincheng Yu, Junsong Chen, Jinyi Hu, Yao Lu, Song Han

ICLR 2025 Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models Junyu Chen, Han Cai, Junsong Chen, Enze Xie, Shang Yang, Haotian Tang, Muyang Li, Song Han

ICLR 2025 DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads Guangxuan Xiao, Jiaming Tang, Jingwei Zuo, Junxian Guo, Shang Yang, Haotian Tang, Yao Fu, Song Han

ICLR 2025 HART: Efficient Visual Generation with Hybrid Autoregressive Transformer Haotian Tang, Yecheng Wu, Shang Yang, Enze Xie, Junsong Chen, Junyu Chen, Zhuoyang Zhang, Han Cai, Yao Lu, Song Han

NeurIPS 2025 Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search Yuxian Gu, Qinghao Hu, Haocheng Xi, Junyu Chen, Shang Yang, Song Han, Han Cai

ICLR 2025 LongVILA: Scaling Long-Context Visual Language Models for Long Videos Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, Yihui He, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu, Song Han

CVPR 2025 NVILA: Efficient Frontier Visual Language Models Zhijian Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang, Haocheng Xi, Shiyi Cao, Yuxian Gu, Dacheng Li, Xiuyu Li, Haotian Tang, Yunhao Fang, Yukang Chen, Cheng-Yu Hsieh, De-An Huang, An-Chieh Cheng, Jinyi Hu, Sifei Liu, Ranjay Krishna, Pavlo Molchanov, Jan Kautz, Hongxu Yin, Song Han, Yao Lu

NeurIPS 2025 Radial Attention: $\mathcal{O}(n\log N)$ Sparse Attention with Energy Decay for Long Video Generation Xingyang Li, Muyang Li, Tianle Cai, Haocheng Xi, Shuo Yang, Yujun Lin, Lvmin Zhang, Songlin Yang, Jinbo Hu, Kelly Peng, Maneesh Agrawala, Ion Stoica, Kurt Keutzer, Song Han

ICML 2025 SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, Han Cai, Bingchen Liu, Daquan Zhou, Song Han

ICCV 2025 SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation Junsong Chen, Shuchen Xue, Yuyang Zhao, Jincheng Yu, Sayak Paul, Junyu Chen, Han Cai, Song Han, Enze Xie

ICLR 2025 SANA: Efficient High-Resolution Text-to-Image Synthesis with Linear Diffusion Transformers Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, Song Han

ICLR 2025 SVDQuant: Absorbing Outliers by Low-Rank Component for 4-Bit Diffusion Models Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun-Yan Zhu, Song Han

NeurIPS 2025 Scaling RL to Long Videos Yukang Chen, Wei Huang, Baifeng Shi, Qinghao Hu, Hanrong Ye, Ligeng Zhu, Zhijian Liu, Pavlo Molchanov, Jan Kautz, Xiaojuan Qi, Sifei Liu, Hongxu Yin, Yao Lu, Song Han

CVPR 2025 Scaling Vision Pre-Training to 4k Resolution Baifeng Shi, Boyi Li, Han Cai, Yao Lu, Sifei Liu, Marco Pavone, Jan Kautz, Song Han, Trevor Darrell, Pavlo Molchanov, Hongxu Yin

ICML 2025 Sparse Video-Gen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity Haocheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, Jianfei Chen, Ion Stoica, Kurt Keutzer, Song Han

NeurIPS 2025 Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation Shuo Yang, Haocheng Xi, Yilong Zhao, Muyang Li, Jintao Zhang, Han Cai, Yujun Lin, Xiuyu Li, Chenfeng Xu, Kelly Peng, Jianfei Chen, Song Han, Kurt Keutzer, Ion Stoica

ICML 2025 SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity Samir Khaki, Xiuyu Li, Junxian Guo, Ligeng Zhu, Konstantinos N. Plataniotis, Amir Yazdanbakhsh, Kurt Keutzer, Song Han, Zhijian Liu

ICCV 2025 SparseVILA: Decoupling Visual Sparsity for Efficient VLM Inference Samir Khaki, Junxian Guo, Jiaming Tang, Shang Yang, Yukang Chen, Konstantinos N. Plataniotis, Yao Lu, Song Han, Zhijian Liu

NeurIPS 2025 Twilight: Adaptive Attention Sparsity with Hierarchical Top-$p$ Pruning Chaofan Lin, Jiaming Tang, Shuo Yang, Hanshuo Wang, Tian Tang, Boyu Tian, Ion Stoica, Song Han, Mingyu Gao

ICLR 2025 VILA-U: A Unified Foundation Model Integrating Visual Understanding and Generation Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, Song Han, Yao Lu

TMLR 2025 Wolf: Dense Video Captioning with a World Summarization Framework Boyi Li, Ligeng Zhu, Ran Tian, Shuhan Tan, Yuxiao Chen, Yao Lu, Yin Cui, Sushant Veer, Max Ehrlich, Jonah Philion, Xinshuo Weng, Fuzhao Xue, Linxi Fan, Yuke Zhu, Jan Kautz, Andrew Tao, Ming-Yu Liu, Sanja Fidler, Boris Ivanovic, Trevor Darrell, Jitendra Malik, Song Han, Marco Pavone

NeurIPS 2025 WorldModelBench: Judging Video Generation Models as World Models Dacheng Li, Yunhao Fang, Yukang Chen, Shuo Yang, Shiyi Cao, Justin Wong, Michael Luo, Xiaolong Wang, Hongxu Yin, Joseph E. Gonzalez, Ion Stoica, Song Han, Yao Lu

ICML 2025 XAttention: Block Sparse Attention with Antidiagonal Scoring Ruyi Xu, Guangxuan Xiao, Haofeng Huang, Junxian Guo, Song Han

NeurIPS 2024 BitDelta: Your Fine-Tune May Only Be Worth One Bit James Liu, Guangxuan Xiao, Kai Li, Jason D. Lee, Song Han, Tri Dao, Tianle Cai

CVPR 2024 Condition-Aware Neural Network for Controlled Image Generation Han Cai, Muyang Li, Qinsheng Zhang, Ming-Yu Liu, Song Han

CVPR 2024 DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models Muyang Li, Tianle Cai, Jiaxin Cao, Qinsheng Zhang, Han Cai, Junjie Bai, Yangqing Jia, Kai Li, Song Han

ICLR 2024 Efficient Streaming Language Models with Attention Sinks Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis

CVPRW 2024 EfficientViT-SAM: Accelerated Segment Anything Model Without Performance Loss Zhuoyang Zhang, Han Cai, Song Han

ICLR 2024 LongLoRA: Efficient Fine-Tuning of Long-Context Large Language Models Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, Jiaya Jia

ICML 2024 QUEST: Query-Aware Sparsity for Efficient Long-Context LLM Inference Jiaming Tang, Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikci, Song Han

ECCV 2024 Sparse Refinement for Efficient High-Resolution Semantic Segmentation Zhijian Liu, Zhuoyang Zhang, Samir Khaki, Shang Yang, Haotian Tang, Chenfeng Xu, Kurt Keutzer, Song Han

CVPR 2024 VILA: On Pre-Training for Visual Language Models Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, Song Han

NeurIPSW 2024 Wolf: Captioning Everything with a World Summarization Framework Boyi Li, Ligeng Zhu, Ran Tian, Shuhan Tan, Yuxiao Chen, Yao Lu, Yin Cui, Sushant Veer, Max Ehrlich, Jonah Philion, Xinshuo Weng, Fuzhao Xue, Andrew Tao, Ming-Yu Liu, Sanja Fidler, Boris Ivanovic, Trevor Darrell, Jitendra Malik, Song Han, Marco Pavone

ICCV 2023 EfficientViT: Lightweight Multi-Scale Attention for High-Resolution Dense Prediction Han Cai, Junyan Li, Muyan Hu, Chuang Gan, Song Han

CVPR 2023 FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer Zhijian Liu, Xinyu Yang, Haotian Tang, Shang Yang, Song Han

CHIL 2023 Machine Learning for Arterial Blood Pressure Prediction Jessica Zheng, Hanrui Wang, Anand Chandrasekhar, Aaron D Aguirre, Song Han, Hae-Seung Lee, Charles G Sodini

ICML 2023 SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, Song Han

CVPR 2023 SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer Xuanyao Chen, Zhijian Liu, Haotian Tang, Li Yi, Hang Zhao, Song Han

CVPRW 2023 TorchSparse++: Efficient Point Cloud Engine Haotian Tang, Shang Yang, Zhijian Liu, Ke Hong, Zhongming Yu, Xiuyu Li, Guohao Dai, Yu Wang, Song Han

NeurIPS 2022 Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models Muyang Li, Ji Lin, Chenlin Meng, Stefano Ermon, Song Han, Jun-Yan Zhu

CVPR 2022 Lite Pose: Efficient Architecture Design for 2D Human Pose Estimation Yihan Wang, Muyang Li, Han Cai, Wei-Ming Chen, Song Han

ICLR 2022 Network Augmentation for Tiny Deep Learning Han Cai, Chuang Gan, Ji Lin, Song Han

NeurIPS 2022 On-Device Training Under 256KB Memory Ji Lin, Ligeng Zhu, Wei-Ming Chen, Wei-Chen Wang, Chuang Gan, Song Han

CVPR 2021 Anycost GANs for Interactive Image Synthesis and Editing Ji Lin, Richard Zhang, Frieder Ganz, Song Han, Jun-Yan Zhu

NeurIPS 2021 Delayed Gradient Averaging: Tolerate the Communication Latency for Federated Learning Ligeng Zhu, Hongzhou Lin, Yao Lu, Yujun Lin, Song Han

ICCV 2021 LocTex: Learning Data-Efficient Visual Representations from Localized Textual Supervision Zhijian Liu, Simon Stent, Jie Li, John Gideon, Song Han

NeurIPS 2021 Memory-Efficient Patch-Based Inference for Tiny Deep Learning Ji Lin, Wei-Ming Chen, Han Cai, Chuang Gan, Song Han

ECCV 2020 DataMix: Efficient Privacy-Preserving Edge-Cloud Inference Zhijian Liu, Zhanghao Wu, Chuang Gan, Ligeng Zhu, Song Han

NeurIPS 2020 Differentiable Augmentation for Data-Efficient GAN Training Shengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu, Song Han

ICLR 2020 Lite Transformer with Long-Short Range Attention Zhanghao Wu, Zhijian Liu, Ji Lin, Yujun Lin, Song Han

NeurIPS 2020 MCUNet: Tiny Deep Learning on IoT Devices Ji Lin, Wei-Ming Chen, Yujun Lin, John Cohn, Chuang Gan, Song Han

ICLR 2020 Once for All: Train One Network and Specialize It for Efficient Deployment Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, Song Han

ECCV 2020 Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution Haotian Tang, Zhijian Liu, Shengyu Zhao, Yujun Lin, Ji Lin, Hanrui Wang, Song Han

NeurIPS 2020 TinyTL: Reduce Memory, Not Parameters for Efficient On-Device Learning Han Cai, Chuang Gan, Ligeng Zhu, Song Han

AAAI 2019 Communication-Optimal Distributed Dynamic Graph Clustering Chun Jiang Zhu, Tan Zhu, Kam-yiu Lam, Song Han, Jinbo Bi

NeurIPS 2019 Deep Leakage from Gradients Ligeng Zhu, Zhijian Liu, Song Han

ICLR 2019 Defensive Quantization: When Efficiency Meets Robustness Ji Lin, Chuang Gan, Song Han

ICML 2019 Improved Dynamic Graph Learning Through Fault-Tolerant Sparsification Chunjiang Zhu, Sabine Storandt, Kam-Yiu Lam, Song Han, Jinbo Bi

ICCVW 2019 On-Device Image Classification with Proxyless Neural Architecture Search and Quantization-Aware Fine-Tuning Han Cai, Tianzhe Wang, Zhanghao Wu, Kuan Wang, Ji Lin, Song Han

ICMLW 2019 Park: An Open Platform for Learning Augmented Computer Systems Hongzi Mao, Parimarjan Negi, Akshay Narayan, Hanrui Wang, Jiacheng Yang, Haonan Wang, Ryan Marcus, Ravichandra Addanki, Mehrdad Khani, Songtao He, Vikram Nathan, Frank Cangialosi, Shaileshh Bojja Venkatakrishnan, Wei-Hung Weng, Song Han, Tim Kraska, Mohammad Alizadeh

NeurIPS 2019 Park: An Open Platform for Learning-Augmented Computer Systems Hongzi Mao, Parimarjan Negi, Akshay Narayan, Hanrui Wang, Jiacheng Yang, Haonan Wang, Ryan Marcus, Ravichandra Addanki, Mehrdad Khani Shirkoohi, Songtao He, Vikram Nathan, Frank Cangialosi, Shaileshh Venkatakrishnan, Wei-Hung Weng, Song Han, Tim Kraska, Dr.Mohammad Alizadeh

NeurIPS 2019 Point-Voxel CNN for Efficient 3D Deep Learning Zhijian Liu, Haotian Tang, Yujun Lin, Song Han

ICLR 2019 ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware Han Cai, Ligeng Zhu, Song Han

ECCV 2018 AMC: AutoML for Model Compression and Acceleration on Mobile Devices Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, Song Han

ICLR 2018 Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training Yujun Lin, Song Han, Huizi Mao, Yu Wang, Bill Dally

ICLR 2018 Efficient Sparse-Winograd Convolutional Neural Networks Xingyu Liu, Jeff Pool, Song Han, William J. Dally

ICML 2018 Path-Level Network Transformation for Efficient Architecture Search Han Cai, Jiacheng Yang, Weinan Zhang, Song Han, Yong Yu

ICLR 2017 DSD: Dense-Sparse-Dense Training for Deep Neural Networks Song Han, Jeff Pool, Sharan Narang, Huizi Mao, Enhao Gong, Shijian Tang, Erich Elsen, Peter Vajda, Manohar Paluri, John Tran, Bryan Catanzaro, William J. Dally

ICLR 2017 Efficient Sparse-Winograd Convolutional Neural Networks Xingyu Liu, Song Han, Huizi Mao, William J. Dally

CVPRW 2017 Exploring the Granularity of Sparsity in Convolutional Neural Networks Huizi Mao, Song Han, Jeff Pool, Wenshuo Li, Xingyu Liu, Yu Wang, William J. Dally

ICLR 2017 Trained Ternary Quantization Chenzhuo Zhu, Song Han, Huizi Mao, William J. Dally

ICLR 2016 Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding Song Han, Huizi Mao, William J. Dally

NeurIPS 2015 Learning Both Weights and Connections for Efficient Neural Network Song Han, Jeff Pool, John Tran, William Dally

CVPR 2010 Removal of 3D Facial Expressions: A Learning-Based Approach Gang Pan, Song Han, Zhaohui Wu, Yuting Zhang

CVPR 2008 Hallucinating 3D Facial Shapes Gang Pan, Song Han, Zhaohui Wu

ICCV 1993 Using Hyperquadrics for Shape Recovery from Range Data Song Han, Dmitry B. Goldgof, Kevin W. Bowyer