FAT: Frequency-Aware Transformation for Bridging Full-Precision and Low-Precision Deep Representations

Tao, Chaofan1; Lin, Rui1; Chen, Quan2; Zhang, Zhaoyang3; Luo, Ping4; Wong, Ngai1

2022-07-01

10.1109/TNNLS.2022.3190607

IEEE Transactions on Neural Networks and Learning Systems   Impact Factor & Quartile

2162-237X

2162-2388

Learning low-bitwidth convolutional neural networks (CNNs) is challenging because performance may drop significantly after quantization. Prior arts often quantize the network weights by carefully tuning hyperparameters such as nonuniform stepsize and layerwise bitwidths, which are complicated since the full-and low-precision representations have large discrepancies. This work presents a novel quantization pipeline, named frequency-aware transformation (FAT), that features important benefits: 1) instead of designing complicated quantizers, FAT learns to transform network weights in the frequency domain to remove redundant information before quantization, making them amenable to training in low bitwidth with simple quantizers; 2) FAT readily embeds CNNs in low bitwidths using standard quantizers without tedious hyperparameter tuning and theoretical analyses show that FAT minimizes the quantization errors in both uniform and nonuniform quantizations; and 3) FAT can be easily plugged into various CNN architectures. Using FAT with a simple uniform/logarithmic quantizer can achieve the state-of-the-art performance in different bitwidths on various model architectures. Consequently, FAT serves to provide a novel frequency-based perspective for model quantization.

Quantization (signal) Fats Transforms Training Adaptation models Standards Frequency-domain analysis Efficient neural network model compression quantization representation learning

SCI

English

Others

General Research Fund (GRF)["17206020","17209721"]

Computer Science ; Engineering

Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic

WOS:000833054400001

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

Web of Science

1.Univ Hong Kong, Dept Elect & Elect Engn, Hong Kong, Peoples R China
2.Southern Univ Sci & Technol, Sch Microelect, Shenzhen 518055, Peoples R China
3.Chinese Univ Hong Kong, Dept Elect Engn, Hong Kong, Peoples R China
4.Univ Hong Kong, Dept Comp Sci, Hong Kong, Peoples R China

Tao, Chaofan,Lin, Rui,Chen, Quan,et al. FAT: Frequency-Aware Transformation for Bridging Full-Precision and Low-Precision Deep Representations[J]. IEEE Transactions on Neural Networks and Learning Systems,2022,PP(99):1-15.

Tao, Chaofan,Lin, Rui,Chen, Quan,Zhang, Zhaoyang,Luo, Ping,&Wong, Ngai.(2022).FAT: Frequency-Aware Transformation for Bridging Full-Precision and Low-Precision Deep Representations.IEEE Transactions on Neural Networks and Learning Systems,PP(99),1-15.

Tao, Chaofan,et al."FAT: Frequency-Aware Transformation for Bridging Full-Precision and Low-Precision Deep Representations".IEEE Transactions on Neural Networks and Learning Systems PP.99(2022):1-15.