Heterogenous Federated Learning: from Algorithms to Applications
Open Access
- Author:
- Wang, Jiaqi
- Graduate Program:
- Informatics
- Degree:
- Doctor of Philosophy
- Document Type:
- Dissertation
- Date of Defense:
- March 07, 2025
- Committee Members:
- Dongwon Lee, Professor in Charge/Director of Graduate Studies
Fenglong Ma, Chair & Dissertation Advisor
Jinghui Chen, Major Field Member
Lu Lin, Major Field Member
Sencun Zhu, Outside Unit & Field Member - Keywords:
- Federated learning
Data Heterogeneity
Model Heterogeneity
IoT
Medical Foundation Model - Abstract:
- Federated Learning (FL) is a collaborative machine learning mechanism that enables disparate parties to train models without sharing data directly. This approach has gained traction across various research domains and real-world applications. However, heterogeneity in data and models poses significant challenges, often degrading performance and complicating traditional model aggregation methods. This thesis delves into advanced federated learning algorithms to address these heterogeneity challenges. It also introduces novel applications in real-world contexts such as the Internet of Things (IoT) and healthcare. The first part of the thesis presents a novel model reassembly approach that addresses data heterogeneity through model personalization. After local models are uploaded to a central server, they are segmented and reassembled into personalized teacher models that guide local training. The second part proposes an asymmetrical reciprocity approach to federated learning, aimed at heterogeneous model aggregation. This method employs dual knowledge distillation to facilitate information exchange between local private models and compact proxy models, thereby protecting full model details and reducing communication costs. The final part of the thesis explores two real-world applications: a semi-supervised federated learning approach in the IoT and a federated knowledge injection method for medical foundation models. In the IoT case, the thesis demonstrates how distributed, unlabeled data in lightweight IoT clients can be leveraged through personalized model compression and uncertainty-based data selection. In healthcare, it proposes a novel knowledge injection strategy to enhance the capabilities of medical foundation models across different data modalities and tasks using federated learning. Overall, this dissertation systematically investigates heterogeneous federated learning from algorithmic innovations to practical applications, significantly advancing collaborative machine learning research and its implementation in both academic and industrial domains.