Title | Learning Linear and Nonlinear Low-Rank Structure in Multi-Task Learning |
Author | |
Publication Years | 2022
|
DOI | |
Source Title | |
ISSN | 1041-4347
|
EISSN | 1558-2191
|
Volume | PPIssue:99Pages:1-12 |
Abstract | As the trace norm can discover low-rank structures in a matrix, it has been widely used in multi-task learning to recover the low-rank structure contained in the parameter matrix. Recently, with the emerging of big complex datasets and the popularity of deep learning techniques, tensor trace norms have been used for deep multi-task models. However, existing tensor trace norms exhibit some limitations. For example, they cannot discover all the low-rank structures in a tensor, they require users to manually specify the importance of each component in the corresponding tensor trace norm, and they only capture the linear low-rank structure. To solve the first issue, in this paper, we propose a Generalized Tensor Trace Norm (GTTN). The GTTN is defined as a convex combination of matrix trace norms of all possible tensor flattenings and hence it can discover all the possible low-rank structures. For the second issue, in the induced objective function with the GTTN, we propose four strategies to learn combination coefficients in the GTTN. Furthermore, we propose the Nonlinear GTTN (NGTTN) to capture nonlinear low-rank structure among all the tasks. Experiments on benchmark datasets demonstrate the effectiveness of the proposed GTTN and NGTTN. |
Keywords | |
URL | [Source Record] |
Indexed By | |
Language | English
|
SUSTech Authorship | First
|
EI Accession Number | 20223812754010
|
EI Keywords | Deep Learning
; Job Analysis
; Matrix Algebra
; Neural Networks
|
ESI Classification Code | Ergonomics And Human Factors Engineering:461.4
; Algebra:921.1
|
ESI Research Field | ENGINEERING
|
Scopus EID | 2-s2.0-85137905337
|
Data Source | Scopus
|
PDF url | https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9875058 |
Citation statistics |
Cited Times [WOS]:0
|
Document Type | Journal Article |
Identifier | http://kc.sustech.edu.cn/handle/2SGJ60CL/402402 |
Department | Department of Computer Science and Engineering 工学院_斯发基斯可信自主研究院 |
Affiliation | 1.Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, China 2.Department of Computer Science and Engineering and Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology, China 3.National Key Laboratory for Novel Software Technology, Nanjing University, China |
First Author Affilication | Department of Computer Science and Engineering |
First Author's First Affilication | Department of Computer Science and Engineering |
Recommended Citation GB/T 7714 |
Zhang,Yi,Zhang,Yu,Wang,Wei. Learning Linear and Nonlinear Low-Rank Structure in Multi-Task Learning[J]. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,2022,PP(99):1-12.
|
APA |
Zhang,Yi,Zhang,Yu,&Wang,Wei.(2022).Learning Linear and Nonlinear Low-Rank Structure in Multi-Task Learning.IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,PP(99),1-12.
|
MLA |
Zhang,Yi,et al."Learning Linear and Nonlinear Low-Rank Structure in Multi-Task Learning".IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING PP.99(2022):1-12.
|
Files in This Item: | There are no files associated with this item. |
|
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment