Theoretical guarantee for crowdsourcing learning with unsure option
Crowdsourcing learning, in which labels are collected from multiple workers through crowdsourcing platforms, has attracted much attention during the past decade. This learning paradigm would reduce the labeling cost since crowdsourcing workers may be non-expert and hence less costly. On the other hand, crowdsourcing learning algorithms also suffer from being misled by incorrect labels introduced by imperfect workers. To control such risks, recently, it has been suggested to provide workers an additional unsure option during the labeling process. Although the benefits of the unsure option have been empirically demonstrated, theoretical analysis is still limited. In this article, a theoretical analysis of crowdsourcing learning with the unsure option is presented. Specifically, an upper bound of minimally sufficient number of crowd labels required for learning a probably approximately correct (PAC) classification model with and without the unsure option are given respectively. Next, a condition under which providing (or not providing) an unsure option to workers is derived. Then, the theoretical results are extended to guide non-identical label options (with or without unsure options) to different workers. Last, several useful applications are proposed based on theoretical results.
|WOS Accession No|
|ESI Research Field|
Cited Times [WOS]:0
|Document Type||Journal Article|
|Department||Department of Computer Science and Engineering|
1.State Key Laboratory of Processors (SKLP),School of Computer Science and Technology,University of Science and Technology of China (USTC),Hefei,Anhui,230026,China
2.Guangdong Provincial Key Laboratory of Brain-Inspired Intelligent Computation,Department of Computer Science and Engineering,Southern University of Science and Technology (SUSTech),Shenzhen,Guangdong,518055,China
|Corresponding Author Affilication||Department of Computer Science and Engineering|
Pan，Yigong,Tang，Ke,Sun，Guangzhong. Theoretical guarantee for crowdsourcing learning with unsure option[J]. PATTERN RECOGNITION,2023,137.
Pan，Yigong,Tang，Ke,&Sun，Guangzhong.(2023).Theoretical guarantee for crowdsourcing learning with unsure option.PATTERN RECOGNITION,137.
Pan，Yigong,et al."Theoretical guarantee for crowdsourcing learning with unsure option".PATTERN RECOGNITION 137(2023).
|Files in This Item:||There are no files associated with this item.|
|Recommend this item|
|Export to Endnote|
|Export to Excel|
|Export to Csv|
|Similar articles in Google Scholar|
|Similar articles in Baidu Scholar|
|Similar articles in Bing Scholar|
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.