中文版 | English
Title

Generalization Error Analysis of Neural Networks with Gradient Based Regularization

Author
Corresponding AuthorTai, Xue-Cheng
Publication Years
2022-10-01
DOI
Source Title
ISSN
1815-2406
EISSN
1991-7120
Volume32Issue:4
Abstract
In this work, we study gradient-based regularization methods for neural networks. We mainly focus on two regularization methods: the total variation and the Tikhonov regularization. Adding the regularization term to the training loss is equiv-alent to using neural networks to solve some variational problems, mostly in high di-mensions in practical applications. We introduce a general framework to analyze the error between neural network solutions and true solutions to variational problems. The error consists of three parts: the approximation errors of neural networks, the quadrature errors of numerical integration, and the optimization error. We also apply the proposed framework to two-layer networks to derive a priori error estimate when the true solution belongs to the so-called Barron space. Moreover, we conduct some numerical experiments to show that neural networks can solve corresponding varia-tional problems sufficiently well. The networks with gradient-based regularization are much more robust in image applications.
Keywords
URL[Source Record]
Indexed By
Language
English
SUSTech Authorship
Others
Funding Project
National Science Foundation of China and Hong Kong RGC Joint Research Scheme (NSFC/RGC )[2019B030301001] ; Guangdong Provincial Key Laboratory of Computational Science[NSFC-11871264] ; National Science Foundation of China[RCJC20210609103819018] ; [11961160718]
WOS Research Area
Physics
WOS Subject
Physics, Mathematical
WOS Accession No
WOS:000882773300004
Publisher
Data Source
Web of Science
Citation statistics
Cited Times [WOS]:0
Document TypeJournal Article
Identifierhttp://kc.sustech.edu.cn/handle/2SGJ60CL/412184
DepartmentDepartment of Mathematics
Affiliation
1.Hong Kong Ctr Cerebrocardiovasc Hlth Engn, Shatin, 19W, Hong Kong Sci Pk, Hong Kong, Peoples R China
2.Southern Univ Sci & Technol, Dept Math, Shenzhen, Peoples R China
3.Southern Univ Sci & Technol, SUSTech Int Ctr Math, Shenzhen, Peoples R China
Recommended Citation
GB/T 7714
Li, Lingfeng,Tai, Xue-Cheng,Yang, Jiang. Generalization Error Analysis of Neural Networks with Gradient Based Regularization[J]. Communications in Computational Physics,2022,32(4).
APA
Li, Lingfeng,Tai, Xue-Cheng,&Yang, Jiang.(2022).Generalization Error Analysis of Neural Networks with Gradient Based Regularization.Communications in Computational Physics,32(4).
MLA
Li, Lingfeng,et al."Generalization Error Analysis of Neural Networks with Gradient Based Regularization".Communications in Computational Physics 32.4(2022).
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Export to Excel
Export to Csv
Altmetrics Score
Google Scholar
Similar articles in Google Scholar
[Li, Lingfeng]'s Articles
[Tai, Xue-Cheng]'s Articles
[Yang, Jiang]'s Articles
Baidu Scholar
Similar articles in Baidu Scholar
[Li, Lingfeng]'s Articles
[Tai, Xue-Cheng]'s Articles
[Yang, Jiang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Li, Lingfeng]'s Articles
[Tai, Xue-Cheng]'s Articles
[Yang, Jiang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.