FedPDD: A Privacy-preserving Double Distillation Framework for Cross-silo Federated Recommendation

Wan，Sheng1; Gao，Dashan1; Gu，Hanlin2; Hu，Daning3

10.1109/IJCNN54540.2023.10191850

2023

International Joint Conference on Neural Networks (IJCNN)

2161-4393

Proceedings of the International Joint Conference on Neural Networks

2023-June

JUN 18-23, 2023

null,Broadbeach,AUSTRALIA

345 E 47TH ST, NEW YORK, NY 10017 USA

IEEE

Cross-platform recommendation aims to improve recommendation accuracy by gathering heterogeneous features from different platforms. However, such cross-silo collaborations between platforms are restricted by increasingly stringent privacy protection regulations, thus data cannot be aggregated for training. Federated learning (FL) is a practical solution to deal with the data silo problem in recommendation scenarios. Existing cross-silo FL methods transmit model information to collaboratively build a global model by leveraging the data of overlapped users. However, in reality, the number of overlapped users is often very small, thus largely limiting the performance of such approaches. Moreover, transmitting model information during training requires high communication costs and may cause serious privacy leakage. In this paper, we propose a novel privacy-preserving double distillation framework named FedPDD for cross-silo federated recommendation, which efficiently transfers knowledge when overlapped users are limited. Specifically, our double distillation strategy enables local models to learn not only explicit knowledge from the other party but also implicit knowledge from its past predictions. Moreover, to ensure privacy and high efficiency, we employ an offline training scheme to reduce communication needs and privacy leakage risk. In addition, we adopt differential privacy to further protect the transmitted information. The experiments on two real-world recommendation datasets, HetRec-MovieLens and Criteo, demonstrate the effectiveness of FedPDD compared to the state-of-the-art approaches.

First ; Corresponding

English

CPCI-S

Guangdong Province Focus Research Project[2019KZDZX2014] ; Guangdong Province Research Fund[2019QN01X277] ; National Natural Science Foundation of China["71971106","72001099"]

Computer Science ; Engineering

Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Engineering, Electrical & Electronic

WOS:001046198706045

2-s2.0-85169540296

Scopus

1.SUSTech,Hkust,Dept. of Cse,Hong Kong
2.Webank,Shenzhen,China
3.SUSTech,Dept. of Finance,Shenzhen,China

Wan，Sheng,Gao，Dashan,Gu，Hanlin,et al. FedPDD: A Privacy-preserving Double Distillation Framework for Cross-silo Federated Recommendation[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:IEEE,2023.