Gradient Learning With the Mode-Induced Loss: Consistency Analysis and Applications

Hong Chen1; Youcheng Fu1; Xue Jiang2; Yanhong Chen3; Weifu Li1; Yicong Zhou4; Feng Zheng2

2023

10.1109/TNNLS.2023.3236345

IEEE Transactions on Neural Networks and Learning Systems   Impact Factor & Quartile

2162-2388

2162-2388

Variable selection methods aim to select the key covariates related to the response variable for learning problems with high-dimensional data. Typical methods of variable selection are formulated in terms of sparse mean regression with a parametric hypothesis class, such as linear functions or additive functions. Despite rapid progress, the existing methods depend heavily on the chosen parametric function class and are incapable of handling variable selection for problems where the data noise is heavy-tailed or skewed. To circumvent these drawbacks, we propose sparse gradient learning with the mode-induced loss (SGLML) for robust model-free (MF) variable selection. The theoretical analysis is established for SGLML on the upper bound of excess risk and the consistency of variable selection, which guarantees its ability for gradient estimation from the lens of gradient risk and informative variable identification under mild conditions. Experimental analysis on the simulated and real data demonstrates the competitive performance of our method over the previous gradient learning (GL) methods.

Gradient learning (GL) learning theory mode-induced loss Rademacher complexity variable selection

SCI

English

Others

National Natural Science Foundation of China["12071166","62122035","61972188"] ; Fundamental Research Funds for the Central Universities of China["2662020LXQD002","2662021JC008"] ; Science and Technology Development Fund, Macau SAR[0049/2022/A1] ; University of Macau[MYRG2022-00072-FST]

Computer Science ; Engineering

Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic

WOS:000920995400001

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

IEEE

1.Engineering Research Center of Intelligent Technology for Agriculture, Ministry of Education, Wuhan, China
2.Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China
3.National Space Science Center, Chinese Academy of Sciences, Beijing, China
4.Department of Computer and Information Science, University of Macau, Macau, China

Hong Chen,Youcheng Fu,Xue Jiang,et al. Gradient Learning With the Mode-Induced Loss: Consistency Analysis and Applications[J]. IEEE Transactions on Neural Networks and Learning Systems,2023,PP(99):1-14.

Hong Chen.,Youcheng Fu.,Xue Jiang.,Yanhong Chen.,Weifu Li.,...&Feng Zheng.(2023).Gradient Learning With the Mode-Induced Loss: Consistency Analysis and Applications.IEEE Transactions on Neural Networks and Learning Systems,PP(99),1-14.

Hong Chen,et al."Gradient Learning With the Mode-Induced Loss: Consistency Analysis and Applications".IEEE Transactions on Neural Networks and Learning Systems PP.99(2023):1-14.