Zhang, Ruiyi

54 publications

WACV 2025 ARTIST: Improving the Generation of Text-Rich Images with Disentangled Diffusion Models and Large Language Models Jianyi Zhang, Yufan Zhou, Jiuxiang Gu, Curtis Wigington, Tong Yu, Yiran Chen, Tong Sun, Ruiyi Zhang

TMLR 2025 BiDoRA: Bi-Level Optimization-Based Weight-Decomposed Low-Rank Adaptation Peijia Qin, Ruiyi Zhang, Pengtao Xie

TMLR 2025 Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimization Han Guo, Ramtin Hosseini, Ruiyi Zhang, Sai Ashish Somayajula, Ranak Roy Chowdhury, Rajesh K. Gupta, Pengtao Xie

NeurIPS 2025 DreamPRM: Domain-Reweighted Process Reward Model for Multimodal Reasoning Qi Cao, Ruiyi Wang, Ruiyi Zhang, Sai Ashish Somayajula, Pengtao Xie

ICCV 2025 Multimodal LLMs as Customized Reward Models for Text-to-Image Generation Shijie Zhou, Ruiyi Zhang, Huaisheng Zhu, Branislav Kveton, Yufan Zhou, Jiuxiang Gu, Jian Chen, Changyou Chen

AAAI 2025 Numerical Pruning for Efficient Autoregressive Models Xuan Shen, Zhao Song, Yufa Zhou, Bo Chen, Jing Liu, Ruiyi Zhang, Ryan A. Rossi, Hao Tan, Tong Yu, Xiang Chen, Yufan Zhou, Tong Sun, Pu Zhao, Yanzhi Wang, Jiuxiang Gu

TMLR 2025 Personalization of Large Language Models: A Survey Zhehao Zhang, Ryan A. Rossi, Branislav Kveton, Yijia Shao, Diyi Yang, Hamed Zamani, Franck Dernoncourt, Joe Barrow, Tong Yu, Sungchul Kim, Ruiyi Zhang, Jiuxiang Gu, Tyler Derr, Hongjie Chen, Junda Wu, Xiang Chen, Zichao Wang, Subrata Mitra, Nedim Lipka, Nesreen K. Ahmed, Yu Wang

ICLR 2025 SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understanding Jian Chen, Ruiyi Zhang, Yufan Zhou, Tong Yu, Franck Dernoncourt, Jiuxiang Gu, Ryan A. Rossi, Changyou Chen, Tong Sun

ICLRW 2025 TRIG-Bench: A Benchmark for Text-Rich Image Grounding Ming Li, Ruiyi Zhang, Jian Chen, Tianyi Zhou

TMLR 2025 TapWeight: Reweighting Pretraining Objectives for Task-Adaptive Pretraining Ruiyi Zhang, Sai Ashish Somayajula, Pengtao Xie

ICLRW 2025 Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information Zhengmian Hu, Gang Wu, Saayan Mitra, Ruiyi Zhang, Tong Sun, Heng Huang, Viswanathan Swaminathan

ICLRW 2025 VisR-Bench: A Visual Retrieval Benchmark for Visually-Rich Documents Jian Chen, Ruiyi Zhang, Ming Li, Shijie Zhou, Changyou Chen

ICLR 2024 ADOPD: A Large-Scale Document Page Decomposition Dataset Jiuxiang Gu, Xiangxi Shi, Jason Kuen, Lu Qi, Ruiyi Zhang, Anqi Liu, Ani Nenkova, Tong Sun

ICMLW 2024 Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Models Bang An, Sicheng Zhu, Ruiyi Zhang, Michael-Andrei Panaitescu-Liess, Yuancheng Xu, Furong Huang

ICML 2024 BLO-SAM: Bi-Level Optimization Based Finetuning of the Segment Anything Model for Overfitting-Preventing Semantic Segmentation Li Zhang, Youwei Liang, Ruiyi Zhang, Amirhosein Javadi, Pengtao Xie

CVPR 2024 Customization Assistant for Text-to-Image Generation Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, Tong Sun

ICMLW 2024 Improve Temporal Awareness of LLMs for Domain-General Sequential Recommendation Zhendong Chu, Zichao Wang, Ruiyi Zhang, Yangfeng Ji, Hongning Wang, Tong Sun

AAAI 2024 Knowledge Graph Prompting for Multi-Document Question Answering Yu Wang, Nedim Lipka, Ryan A. Rossi, Alexa F. Siu, Ruiyi Zhang, Tyler Derr

ICLR 2024 SOHES: Self-Supervised Open-World Hierarchical Entity Segmentation Shengcao Cao, Jiuxiang Gu, Jason Kuen, Hao Tan, Ruiyi Zhang, Handong Zhao, Ani Nenkova, Liangyan Gui, Tong Sun, Yu-Xiong Wang

CVPR 2024 TRINS: Towards Multimodal Language Models That Can Read Ruiyi Zhang, Yanzhe Zhang, Jian Chen, Yufan Zhou, Jiuxiang Gu, Changyou Chen, Tong Sun

ICLR 2024 Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints Jian Chen, Ruiyi Zhang, Yufan Zhou, Changyou Chen

TMLR 2024 Transformer Architecture Search for Improving Out-of-Domain Generalization in Machine Translation Yiheng He, Ruiyi Zhang, Sai Ashish Somayajula, Pengtao Xie

NeurIPSW 2023 AutoDAN: Automatic and Interpretable Adversarial Attacks on Large Language Models Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, Tong Sun

NeurIPSW 2023 Enhanced Visual Instruction Tuning for Text-Rich Image Understanding Yanzhe Zhang, Ruiyi Zhang, Jiuxiang Gu, Yufan Zhou, Nedim Lipka, Diyi Yang, Tong Sun

AAAI 2023 Few-Shot Composition Learning for Image Retrieval with Prompt Tuning Junda Wu, Rui Wang, Handong Zhao, Ruiyi Zhang, Chaochao Lu, Shuai Li, Ricardo Henao

NeurIPS 2023 InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding Junda Wu, Tong Yu, Rui Wang, Zhao Song, Ruiyi Zhang, Handong Zhao, Chaochao Lu, Shuai Li, Ricardo Henao

NeurIPSW 2023 Knowledge Graph Prompting for Multi-Document Question Answering Yu Wang, Nedim Lipka, Ryan Rossi, Alexa Siu, Ruiyi Zhang, Tyler Derr

NeurIPS 2023 Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels Jian Chen, Ruiyi Zhang, Tong Yu, Rohan Sharma, Zhiqiang Xu, Tong Sun, Changyou Chen

ICCV 2023 Learning Navigational Visual Representations with Semantic mAP Supervision Yicong Hong, Yang Zhou, Ruiyi Zhang, Franck Dernoncourt, Trung Bui, Stephen Gould, Hao Tan

NeurIPSW 2023 Towards Building the FederatedGPT: Federated Instruction Tuning Jianyi Zhang, Saeed Vahidian, Martin Kuo, Chunyuan Li, Ruiyi Zhang, Tong Yu, Guoyin Wang, Yiran Chen

AAAI 2022 Text-Based Interactive Recommendation via Offline Reinforcement Learning Ruiyi Zhang, Tong Yu, Yilin Shen, Hongxia Jin

AAAI 2022 TiGAN: Text-Based Interactive Image Generation and Manipulation Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, Chris Tensmeyer, Tong Yu, Changyou Chen, Jinhui Xu, Tong Sun

CVPR 2022 Towards Language-Free Training for Text-to-Image Generation Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, Tong Sun

ICLR 2021 Improving Zero-Shot Voice Style Transfer via Disentangled Representation Learning Siyang Yuan, Pengyu Cheng, Ruiyi Zhang, Weituo Hao, Zhe Gan, Lawrence Carin

AAAI 2021 Unsupervised Abstractive Dialogue Summarization for Tete-a-Tetes Xinyuan Zhang, Ruiyi Zhang, Manzil Zaheer, Amr Ahmed

ICLR 2020 Bayesian Meta Sampling for Fast Uncertainty Adaptation Zhenyi Wang, Yang Zhao, Ping Yu, Ruiyi Zhang, Changyou Chen

ICLR 2020 Collaborative Filtering with a Synthetic Feedback Loop Wenlin Wang, Hongteng Xu, Ruiyi Zhang, Wenqi Wang, Lawrence Carin

WACV 2020 Figure Captioning with Relation Maps for Reasoning Charles Chen, Ruiyi Zhang, Eunyee Koh, Sungchul Kim, Scott Cohen, Ryan Rossi

ICLR 2020 GenDICE: Generalized Offline Estimation of Stationary Values Ruiyi Zhang, Bo Dai, Lihong Li, Dale Schuurmans

ICML 2020 Graphical Models Meet Bandits: A Variational Thompson Sampling Approach Tong Yu, Branislav Kveton, Zheng Wen, Ruiyi Zhang, Ole J. Mengshoel

AAAI 2020 Learning Diverse Stochastic Human-Action Generators by Learning Smooth Latent Transitions Zhenyi Wang, Ping Yu, Yang Zhao, Ruiyi Zhang, Yufan Zhou, Junsong Yuan, Changyou Chen

AISTATS 2020 Nested-Wasserstein Self-Imitation Learning for Sequence Generation Ruiyi Zhang, Changyou Chen, Zhe Gan, Zheng Wen, Wenlin Wang, Lawrence Carin

AISTATS 2020 Stochastic Particle-Optimization Sampling and the Non-Asymptotic Convergence Theory Jianyi Zhang, Ruiyi Zhang, Lawrence Carin, Changyou Chen

ICLR 2019 Improving Sequence-to-Sequence Learning via Optimal Transport Liqun Chen, Yizhe Zhang, Ruiyi Zhang, Chenyang Tao, Zhe Gan, Haichao Zhang, Bai Li, Dinghan Shen, Changyou Chen, Lawrence Carin

NeurIPS 2019 Improving Textual Network Learning with Variational Homophilic Embeddings Wenlin Wang, Chenyang Tao, Zhe Gan, Guoyin Wang, Liqun Chen, Xinyuan Zhang, Ruiyi Zhang, Qian Yang, Ricardo Henao, Lawrence Carin

AISTATS 2019 Scalable Thompson Sampling via Optimal Transport Ruiyi Zhang, Zheng Wen, Changyou Chen, Chen Fang, Tong Yu, Lawrence Carin

NeurIPS 2019 Text-Based Interactive Recommendation via Constraint-Augmented Reinforcement Learning Ruiyi Zhang, Tong Yu, Yilin Shen, Hongxia Jin, Changyou Chen

ICML 2019 Understanding and Accelerating Particle-Based Variational Inference Chang Liu, Jingwei Zhuo, Pengyu Cheng, Ruiyi Zhang, Jun Zhu

ICML 2019 Variational Annealing of GANs: A Langevin Perspective Chenyang Tao, Shuyang Dai, Liqun Chen, Ke Bai, Junya Chen, Chang Liu, Ruiyi Zhang, Georgiy Bobashev, Lawrence Carin Duke

UAI 2018 A Unified Particle-Optimization Framework for Scalable Bayesian Sampling Changyou Chen, Ruiyi Zhang, Wenlin Wang, Bai Li, Liqun Chen

NeurIPS 2018 Adversarial Text Generation via Feature-Mover's Distance Liqun Chen, Shuyang Dai, Chenyang Tao, Haichao Zhang, Zhe Gan, Dinghan Shen, Yizhe Zhang, Guoyin Wang, Ruiyi Zhang, Lawrence Carin

AISTATS 2018 Learning Structural Weight Uncertainty for Sequential Decision-Making Ruiyi Zhang, Chunyuan Li, Changyou Chen, Lawrence Carin

ICML 2018 Policy Optimization as Wasserstein Gradient Flows Ruiyi Zhang, Changyou Chen, Chunyuan Li, Lawrence Carin

ICML 2018 Variational Inference and Model Selection with Generalized Evidence Bounds Liqun Chen, Chenyang Tao, Ruiyi Zhang, Ricardo Henao, Lawrence Carin Duke