Feature extractor stacking for cross-domain few-shot learning

The University of Waikato 2024

Cross-domain few-shot learning (CDFSL) addresses learning problems where knowledge needs to be transferred from one or more source domains into an instance-scarce target domain with an explicitly different distribution. Recently published CDFSL methods generally construct a universal model that combines knowledge of multiple source domains into one feature extractor. This enables efficient inference but necessitates re-computation of the extractor whenever a new source domain is added. Some of these methods are also incompatible with heterogeneous source domain extractor architectures. The first part of this thesis proposes feature extractor stacking (FES), a new CDFSL method for combining information from a collection of extractors, that can utilise heterogeneous pretrained extractors out of the box and does not maintain a universal model that needs to be re-computed when its extractor collection is updated. We present the basic FES algorithm, which is inspired by the classic stacked generalisation approach, and also introduce two variants: convolutional FES (ConFES) and regularised FES (ReFES). Given a target-domain task, these algorithms fine-tune each extractor independently, use cross-validation to extract training data for stacked generalisation from the support set, and learn a simple linear stacking classifier from this data. We evaluate our FES methods on the well-known Meta-Dataset benchmark, targeting image classification with convolutional neural networks, and show that they can achieve state-of-the-art performance. The second part of this thesis proposes an efficient semi-supervised learning method that applies self-training to the classification head only and show that it can yield very consistent improvements in average performance in the Meta-Dataset benchmark for cross-domain few-shot learning when applied with FES and other contemporary methods utilising centroid-based classification. The third part of this thesis proposes a bidirectional snapshot

cross-domain few-shot learning; pretrained feature extractors; stacking; semi-supervised learning; self-training; feature subset selection; Thesis

Open access content. Open access content
All items in Research Commons are provided for private study and research purposes and are protected by copyright with all rights reserved unless otherwise indicated.

English

UV1 oai:researchcommons.waikato.ac.nz:10289/16548
1450027722

UNIV OF WAIKATO LIBR
From OAIster®, provided by the OCLC Cooperative.

edsoai.on1450027722