Multi-Domain Active Learning: Literature Review and Comparative Study

He，Rui1; Liu，Shengcai1; He，Shan2; Tang，Ke1

2022

10.1109/TETCI.2022.3217753

IEEE Transactions on Emerging Topics in Computational Intelligence   Impact Factor & Quartile

2471-285X

2471-285X

Multi-domain learning (MDL) refers to learning a set of models simultaneously, where each model is specialized to perform a task in a particular domain. Generally, a high labeling effort is required in MDL, as data needs to be labeled by human experts for every domain. Active learning (AL) can be utilized in MDL to reduce the labeling effort by only using the most informative data. The resultant paradigm is termed multi-domain active learning (MDAL). In this work, we provide an exhaustive literature review for MDAL on the relevant fields, including AL, cross-domain information sharing schemes, and cross-domain instance evaluation approaches. It is found that the few studies which have been directly conducted on MDAL cannot serve as off-the-shelf solutions on more general MDAL tasks. To fill this gap, we construct a pipeline of MDAL and present a comprehensive comparative study of thirty different algorithms, which are established by combining six representative MDL models and five commonly used AL strategies. We evaluate the algorithms on six datasets involving textual and visual classification tasks. In most cases, AL brings notable improvements to MDL, and the naive BvSB (best vs. second best) Uncertainty strategy can perform competitively with the state-of-the-art AL strategies. Besides, BvSB with the MAN (multinomial adversarial networks) model can consistently achieve top or above-average performance on all the datasets. Furthermore, we qualitatively analyze the behaviors of the well-performed strategies and models, shedding light on their superior performance in the comparison. Finally, we recommend using BvSB with the MAN model in the application of MDAL due to their good performance in the experiments.

Active learning Bibliographies comparative study Computational modeling Data models Labeling multi-domain learning Task analysis Training Uncertainty

SCI ; EI

English

First

Guangdong Provincial Key Laboratory[2020B121201001] ; Program for Guangdong Introducing Innovative and Entrepreneurial Teams[2017ZT07X386]

Computer Science

Computer Science, Artificial Intelligence

WOS:000881958100001

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

20224613123214

Artificial intelligence ; Classification (of information) ; Data structures ; Job analysis ; Labeling ; Learning systems

Packaging, General:694.1 ; Information Theory and Signal Processing:716.1 ; Data Processing and Image Processing:723.2 ; Artificial Intelligence:723.4 ; Information Sources and Analysis:903.1 ; Probability Theory:922.1

2-s2.0-85141562255

Scopus

1.Guangdong Provincial Key Laboratory of Brain-Inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China
2.School of Computer Science, University of Birmingham, Birmingham, U.K

He，Rui,Liu，Shengcai,He，Shan,et al. Multi-Domain Active Learning: Literature Review and Comparative Study[J]. IEEE Transactions on Emerging Topics in Computational Intelligence,2022,PP(99):1-14.

He，Rui,Liu，Shengcai,He，Shan,&Tang，Ke.(2022).Multi-Domain Active Learning: Literature Review and Comparative Study.IEEE Transactions on Emerging Topics in Computational Intelligence,PP(99),1-14.

He，Rui,et al."Multi-Domain Active Learning: Literature Review and Comparative Study".IEEE Transactions on Emerging Topics in Computational Intelligence PP.99(2022):1-14.