Journal Article10.1109/TCYB.2021.3139076
Multiparty Dual Learning
6
TL;DR: In this paper , the authors proposed a multiparty dual learning (MPDL) framework to alleviate the problem of limited data with poor quality in an isolated party, and explicitly exploited the probabilistic correlation and structural relationship between dual tasks to regularize the training process.
read more
Abstract: The performance of machine learning algorithms heavily relies on the availability of a large amount of training data. However, in reality, data usually reside in distributed parties such as different institutions and may not be directly gathered and integrated due to various data policy constraints. As a result, some parties may suffer from insufficient data available for training machine learning models. In this article, we propose a multiparty dual learning (MPDL) framework to alleviate the problem of limited data with poor quality in an isolated party. Since the knowledge-sharing processes for multiple parties always emerge in dual forms, we show that dual learning is naturally suitable to handle the challenge of missing data, and explicitly exploits the probabilistic correlation and structural relationship between dual tasks to regularize the training process. We introduce a feature-oriented differential privacy with mathematical proof, in order to avoid possible privacy leakage of raw features in the dual inference process. The approach requires minimal modifications to the existing multiparty learning structure, and each party can build flexible and powerful models separately, whose accuracy is no less than nondistributed self-learning approaches. The MPDL framework achieves significant improvement compared with state-of-the-art multiparty learning methods, as we demonstrated through simulations on real-world datasets.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Federated Learning on Multimodal Data: A Comprehensive Survey
TL;DR: In this article , the authors explore the area of multimodal federated learning (MMFL) to address the fundamental challenges of training models with privacy protection in mind, which allows building models in a collaborative but private way without exchanging data.
Continual Horizontal Federated Learning for Heterogeneous Data
Junki Mori,Isamu Teranishi,Ryo Furukawa +2 more
- 04 Mar 2022
TL;DR: This paper proposes a HFL method using neural networks named continual horizontal federated learning (CHFL), a continual learning approach to improve the performance of HFL by taking advantage of unique features of each client.
8
A Collaborative Multimodal Learning-Based Framework for COVID-19 Diagnosis.
TL;DR: Wang et al. as mentioned in this paper proposed a novel privacy-preserving cross-site framework for COVID-19 diagnosis with multimodal data, which leverages heterogeneous data from multiple parties while preserving patients' privacy.
3
Breaking Hardware Boundaries of IoT Devices via Inverse Feature Completion
TL;DR: In this paper , the authors propose a novel collaborative learning framework that is tolerant of IoT devices with unaligned feature spaces, where local bilevel optimizations for both model parameters and input features are performed iteratively in the training phase, in which the internal correlations of local sensor data provide additional guidance for the feature inference and completion.
1
Survey of Privacy Threats and Countermeasures in Federated Learning
M. Hayashitani,Junki Mori,Isamu Teranishi +2 more
TL;DR: Privacy threats and countermeasures for the typical types of federated learning; horizontal federated learning, vertical federated learning, and transfer federated learning are described.
References
A Survey on Transfer Learning
Sinno Jialin Pan,Qiang Yang +1 more
TL;DR: The relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift are discussed.
Calibrating noise to sensitivity in private data analysis
Cynthia Dwork,Frank McSherry,Kobbi Nissim,Adam Smith +3 more
- 04 Mar 2006
TL;DR: In this article, the authors show that for several particular applications substantially less noise is needed than was previously understood to be the case, and also show the separation results showing the increased value of interactive sanitization mechanisms over non-interactive.
•Posted Content
ADADELTA: An Adaptive Learning Rate Method
TL;DR: A novel per-dimension learning rate method for gradient descent called ADADELTA that dynamically adapts over time using only first order information and has minimal computational overhead beyond vanilla stochastic gradient descent is presented.
7.5K
Deep Learning with Differential Privacy
TL;DR: This work develops new algorithmic techniques for learning and a refined analysis of privacy costs within the framework of differential privacy, and demonstrates that deep neural networks can be trained with non-convex objectives, under a modest privacy budget, and at a manageable cost in software complexity, training efficiency, and model quality.
4.3K
Practical Secure Aggregation for Privacy-Preserving Machine Learning
Keith Bonawitz,Vladimir Ivanov,Ben Kreuter,Antonio Marcedone,H. Brendan McMahan,Sarvar Patel,Daniel Ramage,Aaron Segal,Karn Seth +8 more
- 30 Oct 2017
TL;DR: In this paper, the authors proposed a secure aggregation of high-dimensional data for federated deep neural networks, which allows a server to compute the sum of large, user-held data vectors from mobile devices in a secure manner without learning each user's individual contribution.
3.4K