NICO Challenge: Out-of-Distribution Generalization for Image Recognition Challenges

Abstract

NICO challenge of out-of-distribution (OOD) generalization for image recognition features two tracks: common context generalization and hybrid context generalization, based on a newly proposed OOD dataset called NICO $^{++}$ + + . Strong distribution shifts between the training and test data are constructed for both tracks. In contrast to the current OOD generalization benchmarks where models are tested on a single domain, NICO challenge tests models on multiple domains for a thorough and comprehensive evaluation. To prevent the leakage of target context knowledge and encourage novel and creative solutions instead of leveraging additional training data, we prohibit the model initialization with pretrained parameters, which is not noticed in the previous benchmarks for OOD generalization. To ensure the random initialization of models, we verify and retrain models from all top-10 teams and test them on the private test data. We empirically show that pretraining on ImageNet introduces considerable bias on test performance. We summarize the insights in top-4 solutions, which outperform the official baselines significantly, and the approach of jury award for each track.