Fashion Attribute Extraction Under an Evolving Ontology

Abstract

Fashion trends evolve rapidly, and industries operating within the fashion domain need to constantly update the underlying models to absorb newer trends. The current approach for model updation involves time-consuming and costly retraining procedures due to the incrementally growing dataset size. Thus, continual learning approaches that incrementally accommodate changes in data distributions without exhaustive retraining hold promise for efficient model updation in the fashion domain. In this paper, we study incremental learning for fashion attribute extraction (FAE), an essential task with many downstream applications. Previous studies on FAE have been on purely static settings, and no fashion-domain datasets exist to study the incremental FAE task. In order to address this, we propose an algorithm to transform a static dataset into an evolving dataset and apply the proposed method to existing static FAE datasets. We observe that both generative and discriminative Visual Language Models (VLMs) have dominated static FAE benchmarks. Hence, we choose to extend continual learning frameworks proposed for VLMs to create three discriminative and two generative baselines for the incremental FAE task. We show that, under an evolving ontology, the generative baselines exhibit 50% lower catastrophic forgetting (CF) compared to the discriminative setting for the incremental DeepFashion dataset with a complex ontology. In comparison, we see equivalent performance on the incremental FashionAI dataset with a relatively simple ontology. Finally, we compare our generative VLM baselines with zero and few-shot Gemini baselines on the incremental FAE task, demonstrating that the marginal performance gain does not justify its significantly higher computational cost and inference time (approximately 140x more compute and 150x slower inference), encouraging further research into efficient specialist models for incremental FAE.