Difference of convex algorithms for bilevel programs with applications in hyperparameter selection

Ye, Jane J.1; Yuan, Xiaoming2; Zeng, Shangzhi1; Zhang, Jin3

2022-09-01

10.1007/s10107-022-01888-3

MATHEMATICAL PROGRAMMING   Impact Factor & Quartile

0025-5610

1436-4646

In this paper, we present difference of convex algorithms for solving bilevel programs in which the upper level objective functions are difference of convex functions, and the lower level programs are fully convex. This nontrivial class of bilevel programs provides a powerful modelling framework for dealing with applications arising from hyperparameter selection in machine learning. Thanks to the full convexity of the lower level program, the value function of the lower level program turns out to be convex and hence the bilevel program can be reformulated as a difference of convex bilevel program. We propose two algorithms for solving the reformulated difference of convex program and show their convergence to stationary points under very mild assumptions. Finally we conduct numerical experiments to a bilevel model of support vector machine classification.

Bilevel program Difference of convex algorithm Hyperparameter selection Bilevel model of support vector machine classification

SCI

English

Corresponding

NSFC[12222106] ; Guangdong Basic and Applied Basic Research Foundation[2022B1515020082] ; Shenzhen Science and Technology Program[RCYX20200714114700072]

Computer Science ; Operations Research & Management Science ; Mathematics

Computer Science, Software Engineering ; Operations Research & Management Science ; Mathematics, Applied

WOS:000854426300001

SPRINGER HEIDELBERG

COMPUTER SCIENCE

Web of Science

1.Univ Victoria, Dept Math & Stat, Victoria, BC, Canada
2.Univ Hong Kong, Dept Math, Hong Kong, Peoples R China
3.Southern Univ Sci & Technol, SUSTech Int Ctr Math, Natl Ctr Appl Math Shenzhen, Peng Cheng Lab,Dept Math, Shenzhen, Peoples R China

Ye, Jane J.,Yuan, Xiaoming,Zeng, Shangzhi,et al. Difference of convex algorithms for bilevel programs with applications in hyperparameter selection[J]. MATHEMATICAL PROGRAMMING,2022.

Ye, Jane J.,Yuan, Xiaoming,Zeng, Shangzhi,&Zhang, Jin.(2022).Difference of convex algorithms for bilevel programs with applications in hyperparameter selection.MATHEMATICAL PROGRAMMING.

Ye, Jane J.,et al."Difference of convex algorithms for bilevel programs with applications in hyperparameter selection".MATHEMATICAL PROGRAMMING (2022).