| Title | Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems |
| Author | |
| Corresponding Author | Zhang, Jin |
| Publication Years | 2022
|
| Conference Name | 38th International Conference on Machine Learning (ICML)
|
| ISSN | 2640-3498
|
| Source Title | |
| Conference Date | JUL 17-23, 2022
|
| Conference Place | null,Baltimore,MD
|
| Publication Place | 1269 LAW ST, SAN DIEGO, CA, UNITED STATES
|
| Publisher | |
| Abstract | Gradient-based optimization methods for hyper-parameter tuning guarantee theoretical convergence to stationary solutions when for fixed upperlevel variable values, the lower level of the bilevel program is strongly convex (LLSC) and smooth (LLS). This condition is not satisfied for bilevel programs arising from tuning hyperparameters in many machine learning algorithms. In this work, we develop a sequentially convergent Value Function based Difference-of-Convex Algorithm with inexactness (VF-iDCA). We show that this algorithm achieves stationary solutions without LLSC and LLS assumptions for bilevel programs from a broad class of hyperparameter tuning applications. Our extensive experiments confirm our theoretical findings and show that the proposed VF-iDCA yields superior performance when applied to tune hyperparameters. |
| SUSTech Authorship | Corresponding
|
| Language | English
|
| URL | [Source Record] |
| Indexed By | |
| Funding Project | National Natural Science Foundation of China[11971220]
; Shenzhen Science and Technology Program[RCYX20200714114700072]
; Guangdong Basic and Applied Basic Research Foundation[2022B1515020082]
|
| WOS Research Area | Computer Science
|
| WOS Subject | Computer Science, Artificial Intelligence
|
| WOS Accession No | WOS:000922378802009
|
| Data Source | Web of Science
|
| Citation statistics |
Cited Times [WOS]:0
|
| Document Type | Conference paper |
| Identifier | http://kc.sustech.edu.cn/handle/2SGJ60CL/502107 |
| Department | Department of Mathematics |
| Affiliation | 1.Univ Waterloo, Dept Stat & Actuarial Sci, Waterloo, ON, Canada 2.Univ Victoria, Dept Math & Stat, Victoria, BC, Canada 3.Southern Univ Sci & Technol, Dept Math, SUSTech Int Natl Ctr Math, Shenzhen, Guangdong, Peoples R China 4.Natl Ctr Appl Math Shenzhen, Shenzhen, Guangdong, Peoples R China |
| Corresponding Author Affilication | Department of Mathematics |
| Recommended Citation GB/T 7714 |
Gao, Lucy,Ye, Jane J.,Yin, Haian,et al. Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems[C]. 1269 LAW ST, SAN DIEGO, CA, UNITED STATES:JMLR-JOURNAL MACHINE LEARNING RESEARCH,2022.
|
| Files in This Item: | There are no files associated with this item. | |||||
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment