Learning Linear and Nonlinear Low-Rank Structure in Multi-Task Learning
As the trace norm can discover low-rank structures in a matrix, it has been widely used in multi-task learning to recover the low-rank structure contained in the parameter matrix. Recently, with the emerging of big complex datasets and the popularity of deep learning techniques, tensor trace norms have been used for deep multi-task models. However, existing tensor trace norms exhibit some limitations. For example, they cannot discover all the low-rank structures in a tensor, they require users to manually specify the importance of each component in the corresponding tensor trace norm, and they only capture the linear low-rank structure. To solve the first issue, in this paper, we propose a Generalized Tensor Trace Norm (GTTN). The GTTN is defined as a convex combination of matrix trace norms of all possible tensor flattenings and hence it can discover all the possible low-rank structures. For the second issue, in the induced objective function with the GTTN, we propose four strategies to learn combination coefficients in the GTTN. Furthermore, we propose the Nonlinear GTTN (NGTTN) to capture nonlinear low-rank structure among all the tasks. Experiments on benchmark datasets demonstrate the effectiveness of the proposed GTTN and NGTTN.
|EI Accession Number|
Deep Learning ; Job Analysis ; Matrix Algebra ; Neural Networks
|ESI Classification Code|
Ergonomics And Human Factors Engineering:461.4 ; Algebra:921.1
|ESI Research Field|
Cited Times [WOS]:0
|Document Type||Journal Article|
|Department||Department of Computer Science and Engineering|
1.Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, China
2.Department of Computer Science and Engineering and Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology, China
3.National Key Laboratory for Novel Software Technology, Nanjing University, China
|First Author Affilication||Department of Computer Science and Engineering|
|First Author's First Affilication||Department of Computer Science and Engineering|
Zhang，Yi,Zhang，Yu,Wang，Wei. Learning Linear and Nonlinear Low-Rank Structure in Multi-Task Learning[J]. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,2022,PP(99):1-12.
Zhang，Yi,Zhang，Yu,&Wang，Wei.(2022).Learning Linear and Nonlinear Low-Rank Structure in Multi-Task Learning.IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,PP(99),1-12.
Zhang，Yi,et al."Learning Linear and Nonlinear Low-Rank Structure in Multi-Task Learning".IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING PP.99(2022):1-12.
|Files in This Item:||There are no files associated with this item.|
|Recommend this item|
|Export to Endnote|
|Export to Excel|
|Export to Csv|
|Similar articles in Google Scholar|
|Similar articles in Baidu Scholar|
|Similar articles in Bing Scholar|
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.