Multi-Task Learning via Time-Aware Neural ODE
Multi-Task Learning (MTL) is a well-established paradigm for learning shared models for a diverse set of tasks. Moreover, MTL improves data efficiency by jointly training all tasks simultaneously. However, directly optimizing the losses of all the tasks may lead to imbalanced performance on all the tasks due to the competition among tasks for the shared parameters in MTL models. Many MTL methods try to mitigate this problem by dynamically weighting task losses or manipulating task gradients. Different from existing studies, in this paper, we propose a Neural Ordinal diffeRential equation based Multi-tAsk Learning (NORMAL) method to alleviate this issue by modeling task-specific feature transformations from the perspective of dynamic flows built on the Neural Ordinary Differential Equation (NODE). Specifically, the proposed NORMAL model designs a time-aware neural ODE block to learn task-specific time information, which determines task positions of feature transformations in the dynamic flow, in NODE automatically via gradient descent methods. In this way, the proposed NORMAL model handles the problem of competing shared parameters by learning task positions. Moreover, the learned task positions can be used to measure the relevance among different tasks. Extensive experiments show that the proposed NORMAL model outperforms state-of-the-art MTL models.
First ; Corresponding
National Natural Science Foundation of China;National Natural Science Foundation of China;
|Document Type||Conference paper|
|Department||Department of Computer Science and Engineering|
1.Department of Computer Science and Engineering,Southern University of Science and Technology,China
2.Australian Artificial Intelligence Institute,University of Technology Sydney,Australia
3.Centre for Frontier AI Research,A*STAR,Singapore
4.Institute of High Performance Computing,A*STAR,Singapore
5.Peng Cheng Laboratory,China
|First Author Affilication||Department of Computer Science and Engineering|
|Corresponding Author Affilication||Department of Computer Science and Engineering|
|First Author's First Affilication||Department of Computer Science and Engineering|
Ye，Feiyang,Wang，Xuehao,Zhang，Yu,et al. Multi-Task Learning via Time-Aware Neural ODE[C],2023:4495-4503.
|Files in This Item:||There are no files associated with this item.|
|Recommend this item|
|Export to Endnote|
|Export to Excel|
|Export to Csv|
|Similar articles in Google Scholar|
|Similar articles in Baidu Scholar|
|Similar articles in Bing Scholar|
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.