TransVLAD: Focusing on Locally Aggregated Descriptors for Few-Shot Learning

Haoquan Li1; Laoming Zhang2; Daoan Zhang1; Lang Fu2; Peng Yang2,3; Jianguo Zhang1,4

10.1007/978-3-031-20044-1_30

2022

17th European Conference on Computer Vision (ECCV)

0302-9743

1611-3349

978-3-031-20043-4

LECT NOTES ARTIF INT

13680

OCT 23-27, 2022

null,Tel Aviv,ISRAEL

GEWERBESTRASSE 11, CHAM, CH-6330, SWITZERLAND

SPRINGER INTERNATIONAL PUBLISHING AG

This paper presents a transformer framework for few-shot learning, termed TransVLAD, with one focus showing the power of locally aggregated descriptors for few-shot learning. Our TransVLAD model is simple: a standard transformer encoder following a NeXtVLAD aggregation module to output the locally aggregated descriptors. In contrast to the prevailing use of CNN as part of the feature extractor, we are the first to prove self-supervised learning like masked autoencoders (MAE) can deal with the overfitting of transformers in few-shot image classification. Besides, few-shot learning can benefit from this general-purpose pre-training. Then, we propose two methods to mitigate few-shot biases, supervision bias and simple-characteristic bias. The first method is introducing masking operation into fine-tuning, by which we accelerate fine-tuning (by more than 3x) and improve accuracy. The second one is adapting focal loss into soft focal loss to focus on hard characteristics learning. Our TransVLAD finally tops 10 benchmarks on five popular few-shot datasets by an average of more than 2%.

Few-shot learning Transformers Self-supervised learning NeXtVLAD Focal loss

First ; Corresponding

English

CPCI-S

National Key Research and Development Program of China[2021YFF1200800] ; Stable Support Plan Program of Shenzhen Natural Science Fund[20200925154942002]

Computer Science ; Imaging Science & Photographic Technology

Computer Science, Artificial Intelligence ; Imaging Science & Photographic Technology

WOS:000904098900030

Web of Science

1.Research Institute of Trustworthy Autonomous Systems, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China
2.Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China
3.Department of Statistics and Data Science, Southern University of Science and Technology, Shenzhen, China
4.Peng Cheng Lab, Shenzhen, China

Haoquan Li,Laoming Zhang,Daoan Zhang,et al. TransVLAD: Focusing on Locally Aggregated Descriptors for Few-Shot Learning[C]. GEWERBESTRASSE 11, CHAM, CH-6330, SWITZERLAND:SPRINGER INTERNATIONAL PUBLISHING AG,2022.