Gradient-Guided Discrete Walk-Jump Sampling for Biological Sequence Generation

Abstract

In this work, we propose gradient-guided discrete walk-jump sampling (gg-dWJS), a novel discrete sequence generation method for biological sequence optimization. Leveraging gradient guidance in the noisy manifold, we sample from the smoothed data manifold by applying discretized Markov chain Monte Carlo (MCMC) using a denoising model with the gradient-guidance from a discriminative model. This is followed by jumping to the discrete data manifold using a conditional one-step denoising. We showcase our method in two different modalities: discrete image and biological sequence involving antibody and peptide sequence generation tasks in the single objective and multi-objective setting. Through evaluation on these tasks, we show that our method generates high-quality samples that are well-optimized for specific tasks.