Effective, Efficient and Robust Neural Architecture Search

Zhixiong Yue1; Baijiong Lin2; Yu Zhang3; Christy Liang1

10.1109/IJCNN55064.2022.9892654

2022

IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) / IEEE World Congress on Computational Intelligence (IEEE WCCI) / International Joint Conference on Neural Networks (IJCNN) / IEEE Congress on Evolutionary Computation (IEEE CEC)

2161-4393

978-1-6654-9526-4

2022 International Joint Conference on Neural Networks (IJCNN)

1-8

18-23 July 2022

Padua, Italy

345 E 47TH ST, NEW YORK, NY 10017 USA

IEEE

Designing neural network architecture for embedded devices is practical but challenging because the models are expected to be not only accurate but also enough lightweight and robust. However, it is challenging to balance those trade-offs manually because of the large search space. To solve this problem, we propose an Effective, Efficient, and Robust Neural Architecture Search (E2RNAS) method to automatically search a neural network architecture that balances the performance, robustness, and resource consumption. Unlike previous studies, the objective function of the proposed E2RNAS method is formulated as a multi-objective bi-level optimization problem with the upper-level subproblem as a multi-objective optimization problem that considers the performance, robustness, and resource consumption. To solve the proposed objective function, we integrate the multiple-gradient descent algorithm, a widely studied gradient-based multi-objective optimization algorithm, with the bi-level optimization. Experiments on benchmark datasets show that the proposed E2RNAS method can find robust architecture with low resource consumption and comparable classification accuracy.

neural architecture search adversarial robustness out-of-distribution

Others

English

CPCI-S

NSFC["62136005","62076118"]

Computer Science ; Engineering ; Neurosciences & Neurology

Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Engineering, Electrical & Electronic ; Neurosciences

WOS:000867070906038

IEEE

1.School of Computer Science, University of Technology Sydney
2.Department of Computer Science and Engineering, Southern University of Science and Technology
3.Peng Cheng Laboratory

Zhixiong Yue,Baijiong Lin,Yu Zhang,et al. Effective, Efficient and Robust Neural Architecture Search[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:IEEE,2022:1-8.