Multi-Task Learning via Time-Aware Neural ODE

Ye，Feiyang1,2,3,4; Wang，Xuehao1; Zhang，Yu1,5; Tsang，Ivor W.2,3,4

2023

1045-0823

IJCAI International Joint Conference on Artificial Intelligence

2023-August

4495-4503

Multi-Task Learning (MTL) is a well-established paradigm for learning shared models for a diverse set of tasks. Moreover, MTL improves data efficiency by jointly training all tasks simultaneously. However, directly optimizing the losses of all the tasks may lead to imbalanced performance on all the tasks due to the competition among tasks for the shared parameters in MTL models. Many MTL methods try to mitigate this problem by dynamically weighting task losses or manipulating task gradients. Different from existing studies, in this paper, we propose a Neural Ordinal diffeRential equation based Multi-tAsk Learning (NORMAL) method to alleviate this issue by modeling task-specific feature transformations from the perspective of dynamic flows built on the Neural Ordinary Differential Equation (NODE). Specifically, the proposed NORMAL model designs a time-aware neural ODE block to learn task-specific time information, which determines task positions of feature transformations in the dynamic flow, in NODE automatically via gradient descent methods. In this way, the proposed NORMAL model handles the problem of competing shared parameters by learning task positions. Moreover, the learned task positions can be used to measure the relevance among different tasks. Extensive experiments show that the proposed NORMAL model outperforms state-of-the-art MTL models.

First ; Corresponding

English

National Natural Science Foundation of China[62076118];National Natural Science Foundation of China[62136005];

2-s2.0-85170403979

Scopus

1.Department of Computer Science and Engineering,Southern University of Science and Technology,China
2.Australian Artificial Intelligence Institute,University of Technology Sydney,Australia
3.Centre for Frontier AI Research,A*STAR,Singapore
4.Institute of High Performance Computing,A*STAR,Singapore
5.Peng Cheng Laboratory,China

Ye，Feiyang,Wang，Xuehao,Zhang，Yu,et al. Multi-Task Learning via Time-Aware Neural ODE[C],2023:4495-4503.