Powering Robust Fashion Retrieval with Information Rich Feature Embeddings

Abstract

Visual content based product retrieval has become increasingly important for e-commerce. Fashion retrieval, in particular, is a challenging problem owing to a wide range of deformations of clothing items along with visual distortions in their product images. In this paper, we propose a Grid Search Network (GSN) for learning feature embeddings for fashion retrieval. The proposed approach posits the training procedure as a search problem, focused on locating matches for a reference query image in a grid containing both positive and negative images w.r.t the query. The proposed framework significantly outperforms existing state-of-art methods on benchmark fashion datasets. We also utilize a reinforcement learning based strategy to learn a specialized transformation function which further improves retrieval performance when applied over the feature embeddings. We also extend the reinforcement learning based strategy to learn custom kernel functions for SVM based classification over FashionMNIST and MNIST datasets, showing improved performance. We highlight the generalization capabilities of this search strategy by showing performance improvement in search and attribution tasks in domains beyond fashion.