Easy Begun Is Half Done: Spatial-Temporal Graph Modeling with ST-Curriculum Dropout

Wang，Hongjun1; Chen，Jiyuan1; Pan，Tong3; Fan，Zipei4; Song，Xuan1; Jiang，Renhe4,5; Zhang，Lingyu1,2,7; Xie，Yi6; Wang，Zhongyi6; Zhang，Boyuan1,2

2023-06-27

Proceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI 2023

37

4668-4675

Spatial-temporal (ST) graph modeling, such as traffic speed forecasting and taxi demand prediction, is an important task in deep learning area. However, for the nodes in graph, their ST patterns can vary greatly in difficulties for modeling, owning to the heterogeneous nature of ST data. We argue that unveiling the nodes to the model in a meaningful order, from easy to complex, can provide performance improvements over traditional training procedure. The idea has its root in Curriculum Learning which suggests in the early stage of training models can be sensitive to noise and difficult samples. In this paper, we propose ST-Curriculum Dropout, a novel and easy-to-implement strategy for spatial-temporal graph modeling. Specifically, we evaluate the learning difficulty of each node in high-level feature space and drop those difficult ones out to ensure the model only needs to handle fundamental ST relations at the beginning, before gradually moving to hard ones. Our strategy can be applied to any canonical deep learning architecture without extra trainable parameters, and extensive experiments on a wide range of datasets are conducted to illustrate that, by controlling the difficulty level of ST relations as the training progresses, the model is able to capture better representation of the data and thus yields better generalization.

First ; Corresponding

English

2-s2.0-85167871199

Scopus

1.SUSTech-UTokyo Joint Research Center on Super Smart City,Southern University of Science and Technology,China
2.Research Institute of Trustworthy Autonomous Systems,Southern University of Science and Technology,China
3.Department of Physics,The Chinese University of Hong Kong,Hong Kong
4.Center for Spatial Information Science,University of Tokyo,Japan
5.Information Technology Center,University of Tokyo,Japan
6.Huawei Technologies CO.LTD,China
7.Didichuxing Inc,China

Wang，Hongjun,Chen，Jiyuan,Pan，Tong,et al. Easy Begun Is Half Done: Spatial-Temporal Graph Modeling with ST-Curriculum Dropout[C],2023:4668-4675.