Filter pruning with uniqueness mechanism in the frequency domain for efficient neural networks
Filter pruning has drawn extensive attention due to its advantage in reducing computational costs and memory requirements of deep convolutional neural networks. However, most existing methods only prune filters based on their intrinsic properties or spatial feature maps, ignoring the correlation between filters. In this paper, we suggest the correlation is valuable and consider it from a novel view: the frequency domain. Specifically, we first transfer features to the frequency domain by Discrete Cosine Transform (DCT). Then, for each feature map, we compute a uniqueness score, which measures its probability of being replaced by others. This way allows to prune the filters corresponding to the low-uniqueness maps without significant performance degradation. Compared to the methods focusing on intrinsic properties, our proposed method introduces a more comprehensive criterion to prune filters, further improving the network compactness while preserving good performance. In addition, our method is more robust against noise than the spatial ones since the critical clues for pruning are more concentrated after DCT. Experimental results demonstrate the superiority of our method. To be specific, our method outperforms the baseline ResNet-56 by 0.38% on CIFAR-10 while reducing the floating-point operations (FLOPs) by 47.4%. In addition, a consistent improvement can be observed when pruning the baseline ResNet-110: 0.23% performance increase and up to 71% FLOPs drop. Finally, on ImageNet, our method reduces the FLOPs of the baseline ResNet-50 by 48.7% with only 0.32% accuracy loss.
|WOS Research Area|
Computer Science, Artificial Intelligence
|WOS Accession No|
|ESI Research Field|
Cited Times [WOS]:0
|Document Type||Journal Article|
|Department||Department of Computer Science and Engineering|
1.School of Computing and Communications,Lancaster University,Lancaster,LA1 4WA,United Kingdom
2.WMG Data Science,The University of Warwick,Coventry,CV4 7AL,United Kingdom
3.Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen,518055,China
4.Department of Computer Science,The University of Sheffield,Sheffield,Regent Court, 211 Portobello,S1 4DP,United Kingdom
Zhang，Shuo,Gao，Mingqi,Ni，Qiang,et al. Filter pruning with uniqueness mechanism in the frequency domain for efficient neural networks[J]. NEUROCOMPUTING,2023,530:116-124.
Zhang，Shuo,Gao，Mingqi,Ni，Qiang,&Han，Jungong.(2023).Filter pruning with uniqueness mechanism in the frequency domain for efficient neural networks.NEUROCOMPUTING,530,116-124.
Zhang，Shuo,et al."Filter pruning with uniqueness mechanism in the frequency domain for efficient neural networks".NEUROCOMPUTING 530(2023):116-124.
|Files in This Item:||There are no files associated with this item.|
|Recommend this item|
|Export to Endnote|
|Export to Excel|
|Export to Csv|
|Similar articles in Google Scholar|
|Similar articles in Baidu Scholar|
|Similar articles in Bing Scholar|
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.