Boosting Transductive Few-Shot Fine-Tuning with Margin-Based Uncertainty Weighting and Probability Regularization
Abstract
Few-Shot Learning (FSL) has been rapidly developed in recent years, potentially eliminating the requirement for significant data acquisition. Few-shot fine-tuning has been demonstrated to be practically efficient and helpful, especially for out-of-distribution datum. In this work, we first observe that the few-shot fine-tuned methods are learned with the imbalanced class marginal distribution. This observation further motivates us to propose the Transductive Fine-tuning with Margin-based uncertainty weighting and Probability regularization (TF-MP), which learns a more balanced class marginal distribution. We first conduct sample weighting on the testing data with margin-based uncertainty scores and further regularize each test sample's categorical probability. TF-MP achieves state-of-the-art performance on in- / out-of-distribution evaluations of Meta-Dataset and surpasses previous transductive methods by a large margin.
Cite
Text
Tao et al. "Boosting Transductive Few-Shot Fine-Tuning with Margin-Based Uncertainty Weighting and Probability Regularization." Conference on Computer Vision and Pattern Recognition, 2023. doi:10.1109/CVPR52729.2023.01512Markdown
[Tao et al. "Boosting Transductive Few-Shot Fine-Tuning with Margin-Based Uncertainty Weighting and Probability Regularization." Conference on Computer Vision and Pattern Recognition, 2023.](https://mlanthology.org/cvpr/2023/tao2023cvpr-boosting/) doi:10.1109/CVPR52729.2023.01512BibTeX
@inproceedings{tao2023cvpr-boosting,
title = {{Boosting Transductive Few-Shot Fine-Tuning with Margin-Based Uncertainty Weighting and Probability Regularization}},
author = {Tao, Ran and Chen, Hao and Savvides, Marios},
booktitle = {Conference on Computer Vision and Pattern Recognition},
year = {2023},
pages = {15752-15761},
doi = {10.1109/CVPR52729.2023.01512},
url = {https://mlanthology.org/cvpr/2023/tao2023cvpr-boosting/}
}