Tiny adversarial multi-objective one-shot neural architecture search

Xie，Guoyang1,2; Wang，Jinbao1; Yu，Guo3; Lyu，Jiayi4; Zheng，Feng5; Jin，Yaochu6,7

2023

10.1007/s40747-023-01139-8

Complex and Intelligent Systems   Impact Factor & Quartile

2199-4536

2198-6053

The widely employed tiny neural networks (TNNs) in mobile devices are vulnerable to adversarial attacks. However, more advanced research on the robustness of TNNs is highly in demand. This work focuses on improving the robustness of TNNs without sacrificing the model’s accuracy. To find the optimal trade-off networks in terms of the adversarial accuracy, clean accuracy, and model size, we present TAM-NAS, a tiny adversarial multi-objective one-shot network architecture search method. First, we build a novel search space comprised of new tiny blocks and channels to establish a balance between the model size and adversarial performance. Then, we demonstrate how the supernet facilitates the acquisition of the optimal subnet under white-box adversarial attacks, provided that the supernet significantly impacts the subnet’s performance. Concretely, we investigate a new adversarial training paradigm by evaluating the adversarial transferability, the width of the supernet, and the distinction between training subnets from scratch and fine-tuning. Finally, we undertake statistical analysis for the layer-wise combination of specific blocks and channels on the first non-dominated front, which can be utilized as a design guideline for the design of TNNs.

Adversarial attack Multi-objective optimization One-shot learning Tiny neural network architecture search

SCI

English

First

National Natural Science Foundation of China["62136003","61972188","62122035","62206122","62103150"] ; China Postdoctoral Science Foundation[2021M691012]

Computer Science

Computer Science, Artificial Intelligence

WOS:001026364300001

SPRINGER HEIDELBERG

2-s2.0-85164511940

Scopus

1.Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen,518055,China
2.Department of Computer Science,University of Surrey,Guildford,Surrey,GU2 7XH,United Kingdom
3.Institute of Intelligent Manufacturing,Nanjing Tech University,Nanjing,211816,China
4.School of Engineering Science,University of Chinese Academy of Sciences,Beijing,China
5.Department of Computer Science,Southern University of Science and Technology,Shenzhen,518055,China
6.Faculty of Technology,Bielefeld University,Bielefeld,33619,Germany
7.Department of Computer Science,University of Surrey,Guildford,GU2 7XH,United Kingdom

Xie，Guoyang,Wang，Jinbao,Yu，Guo,et al. Tiny adversarial multi-objective one-shot neural architecture search[J]. Complex and Intelligent Systems,2023,9(6).

Xie，Guoyang,Wang，Jinbao,Yu，Guo,Lyu，Jiayi,Zheng，Feng,&Jin，Yaochu.(2023).Tiny adversarial multi-objective one-shot neural architecture search.Complex and Intelligent Systems,9(6).

Xie，Guoyang,et al."Tiny adversarial multi-objective one-shot neural architecture search".Complex and Intelligent Systems 9.6(2023).