FedCC: Towards Addressing Label Distribution Skews in Distillation-Based Federated Learning

Wenxuan Ye ¹², Onur Ayan¹, Xueli An¹, Georg Carle² Affiliation: ¹ Huawei Heisenberg Research Center, Huawei Technologies Duesseldorf GmbH
² TUM School of Computation, Information and Technology, Technical University of Munich Affiliation: wenxuan.ye@tum.de, {onur.ayan, xueli.an}@huawei.com, carle@net.in.tum.de

Abstract

Federated Learning (FL) enables distributed clients to collaboratively train models without sharing raw data, making it promising for leveraging massive devices in communication networks. In distillation-based FL, each client applies its local model on an unlabeled public dataset, and shares only prediction results with the server. While heterogeneous local data introduces label distribution skew, thus biasing client models toward majority classes and leading to potentially inaccurate predictions. The lack of ground-truth labels in the public dataset hampers the server’s ability to calibrate predictions, which ultimately degrades overall performance. To address this, we propose FedCC, a simple and effective algorithm for mitigating client misclassification. Instead of being forced to classify and risking error propagation, clients are allowed to tag ambiguous samples as ‘unknown’. This additional class, together with calibrated pseudo-labels on the public data, balances confidence in majority classes against uncertainty in under-represented ones. Extensive experiments demonstrate that FedCC significantly outperforms existing methods, especially under severe label skew. In the extreme scenario where each client holds samples from only one of ten classes, FedCC achieves 67.3% accuracy, while baselines collapse to near-random results.

Index Terms:

Federated learning, Distillation-based FL, Label distribution skew, Pseudo-labeling

I Introduction

Federated Learning (FL) [1] enables collaborative model training across decentralized clients without exposing their raw data. In the classical parameter-based approach, each client refines the global model with its private data, uploads the updated parameters to the server, and the server aggregates these updates to produce the next-round global model. While effective, this approach incurs heavy communication overhead on parameter transmission, and assumes a homogeneous model architecture, overlooking the diverse bandwidth, computation, and data distributions across real-world devices [2, 3, 4, 5].

Drawing on Knowledge Distillation (KD) [6], Distillation-based FL offers an alternative for client collaboration [7]. Each client runs its local model on a public dataset and uploads only the resulting prediction vectors (i.e., class-score logits); the server then aggregates these vectors into a global prediction vector and redistributes it, reducing communication overhead by orders of magnitude. In addition, because logits are architecture-agnostic, the approach natively supports heterogeneous models, enabling flexible deployment across varied networked devices. Regarding the public dataset, various strategies have been employed, including the use of a labeled dataset [8, 9], or the generation of synthetic data derived from local datasets or local model parameters [10]. Recent studies [11] have relaxed the requirement by adopting unlabeled ones, avoiding costly labeling processes and alleviating privacy concerns. Accordingly, this paper adopts an unlabeled public dataset as an integral component of the scenario setting.

Despite its promise, distillation-based FL is vulnerable to label distribution skew: heterogeneous client data cause local models to overfit the majority classes, and the resulting biases are reflected in the prediction vectors [2]. The challenge is further compounded by the absence of ground-truth labels in the public dataset, which prevents the server from calibrating logits and results in distorted supervision that degrades global performance. Existing approaches in parameter-based FL (e.g., weighted model aggregation [12], model regularization [13]) presume access to client model weights. This assumption breaks down in distillation-based FL, where only logit vectors are exchanged, providing far sparser signals. Recent efforts focus on weighting client updates by estimated reliability during server-side aggregation [14], moving beyond naive averaging as in FedDF[11]. Others aim at amplifying minority-class information and aligning local models with the global objective [15]. Nevertheless, label-skew bias still creeps in, and existing schemes cannot consistently filter out misleading updates during aggregation. Empirical evidence [16] demonstrates that, under label-distribution skew, local models overwhelmingly predict their majority classes, thereby injecting errors into the aggregated prediction (as in Fig. 1a).

Fig. 1: Comparison of distillation-based FL under label distribution skews. a) FedDF [11] averages biased local predictions, leading to suboptimal outcomes. b) Our FedCC introduces an ‘unknown’ class (highlighted in blue shadow), allowing clients to acknowledge their unknownness and thereby yielding more informative aggregation.

In response, we propose FedCC, a simple and effective algorithm designed to mitigate Client misClassification. Treating the mismatch between a client’s skewed data and the ideal balanced distribution as an open-set problem [17], we introduce an ‘unknown’ class to absorb minority and missing classes. Instead of being forced to issue a class prediction and risking error propagation, clients are allowed to acknowledge their limitations and mark ambiguous inputs as ‘unknown’. Illustrated in Fig. 1b, this ‘unknown’ class (shown in blue shadow) enables clients to express uncertainty, yielding more informative aggregated predictions. To implement this, we incorporate the unlabeled dataset into the local objective function and employ a calibrated pseudo-label generation method, to weigh confidence in majority classes with the uncertainty hidden in unknown classes. This weight is adaptively adjusted per client and per sample, optimizing the model’s generalization under diverse data conditions.

To evaluate performance, we conduct experiments on three widely used datasets: CIFAR-10, CIFAR-100, and TinyImageNet, across various label skew scenarios. FedCC consistently outperforms state-of-the-art FL methods by at least 3% points in most scenarios, and the accuracy gap grows as label skew becomes more severe. In the extreme scenario where each client holds samples from only one of ten classes, FedCC achieves an accuracy of 67.3%, while other baselines fail to generate meaningful predictions. Beyond delivering strong global generalization, FedCC enhances local model performance on its minority classes by leveraging uncertainty-aware predictions.

To summarize, our key contributions are as follows:

•

We introduce FedCC, a simple and effective method for mitigating client misclassification. We frame label skew as an open-set recognition problem, and extend the existing class framework with an ‘unknown’ class to absorb minority and missing classes.
•

FedCC incorporates the unlabeled dataset into the local objective function, and proposes a novel pseudo-label generation method, enabling the model to recognize and account for unknownness. This method can be interpreted as adding a regularizer, mitigating overfitting.
•

Extensive experiments demonstrate that FedCC significantly outperforms current FL methods, and the performance gap increases as label skew becomes more severe.

II Related Works

Parameter-based FL with label distribution skews: While FL safeguards privacy by keeping data local, the skewed label distributions across clients undermine model generalization. To stabilize heterogeneous training, methods like FedProx [13] and FedSAM [18] employ regularization, variance reduction, or gradient filtering. Other approaches target class imbalance, e.g., via rescaling logits (FedRS [19]), or calibrating predictions (FedLC [20], FedMR [15], FedVLS [21]). Alternatively, FedConcat [22] departs from model averaging by concatenating local encoders to collaboratively train a global classifier. While FedOV [16] uses open-set recognition to alleviate label skew, its reliance on synthesizing outlier samples with implicit labels limits its applicability to unlabeled data.

Distillation-based FL with label distribution skews: To curb label skew under distillation-based FL, FedNed [23] emphasizes distilling minority-class knowledge, LightTS [24] re-weights clients by strength, and Co-Boosting [10] alternates synthetic data generation with ensemble updates. However, their reliance on labeled public datasets or large-scale synthesis raises annotation and privacy concerns. When public data is unlabeled, the lack of ground-truth labels complicates bias correction. Existing solutions address this by retaining high-confidence predictions [8], clustering clients [25], or communicating only hard labels [26]. While emphasizing reliable local knowledge, these methods struggle to filter misinformation stemming from inherent local biases (As shown in Fig. 1a). Thus, effective solutions for fair client-side classification remain an open challenge.

III Scenario Setting

Consider an FL scenario with $K$ clients and $C$ classes (label set $\mathbb{C}=\{1,\dots,C\}$ ). Each client $k$ holds a private local dataset $\mathbb{D}_{l}^{(k)}=\{(x,y)\}$ . Additionally, a global unlabeled public dataset $\mathbb{D}_{u}=\{x_{u}\}$ is accessible to all clients and the server. Due to label skew, client label priors $P^{(k)}(y)$ vary, though we assume the conditional distribution $P(y|x)$ is consistent across clients [2]. We partition $\mathbb{C}$ into three mutually exclusive subsets ( $\mathbb{C}_{J}^{(k)}\cup\mathbb{C}_{N}^{(k)}\cup\mathbb{C}_{S}^{(k)}=\mathbb{C}$ ) for each client $k$ : majority classes $\mathbb{C}_{J}^{(k)}$ (sufficiently represented), minority classes $\mathbb{C}_{N}^{(k)}$ (sparse, noisy estimates), and missing classes $\mathbb{C}_{S}^{(k)}$ (absent).

Collaborative distillation protocol: Client $k$ deploys a heterogeneous classifier $f^{(k)}(\cdot;\theta^{(k)})$ . In each communication round, client $k$ computes logits $f^{(k)}(x_{u})\in\mathbb{R}^{C}$ for public samples $x_{u}\in\mathbb{D}_{u}$ . These are converted to probabilities $\hat{\mathbf{y}}^{(k)}(x_{u}):=\sigma^{(k)}(x_{u})$ via a temperature-scaled softmax (with temperature $\tau=1$ omitted for brevity).

The server aggregates these probabilities into an ensemble teacher:

\hat{\mathbf{y}}^{(s)}(x_{u}):=\sum\nolimits_{k}\mu_{k}\,\hat{\mathbf{y}}^{(k)}(x_{u}),\quad\mu_{k}\geq 0,\;\sum\nolimits_{k}\mu_{k}=1.

(1)

Clients then synchronize their models by minimizing the Kullback-Leibler divergence with the teacher distribution:

\mathbb{E}_{x_{u}\sim\mathbb{D}_{u}}\left[\text{D}_{\text{KL}}\left(\hat{\mathbf{y}}^{(s)}(x_{u})\parallel\sigma^{(k)}(x_{u})\right)\right].

(2)

Post-synchronization, clients refine their models on local private data before sharing predictions in the next round.

IV Approach

FedCC targets client-side misclassification, and by lowering local errors, it ultimately improves global generalization. Sections IV-A-IV-B detail the proposed client-level algorithm, and Section IV-C outlines the overall federated training pipeline. In Sections IV-A–IV-B we focus on a single client, and therefore omit the client index for notational brevity (e.g., $f$ , $\theta$ and $\sigma$ instead of $f^{(k)}$ , $\theta^{(k)}$ and $\sigma^{(k)}$ ).

IV-A Local learning objective

Semi-supervised learning: We begin by reviewing Semi-Supervised Learning (SSL), a paradigm that leverages both labeled and unlabeled data to build effective models. A classical SSL strategy is self-training [27], which bootstraps a model with its own high-confidence predictions. Firstly, a classifier $f(\cdot\,;\theta_{0})$ is fitted on labeled data $\mathbb{D}_{l}=\{(x,y)\}$ . The trained model $\theta_{0}$ then assigns pseudo-labels to the unlabeled data $x_{u}\in\mathbb{D}_{u}$ by selecting the class of maximum posterior probability: $\hat{y}(x_{u})=\arg\max_{c\in\mathbb{C}}f_{c}(x_{u};\theta_{0})$ . Merging the original and pseudo-labeled examples yields an enlarged dataset on which the model is re-optimized. The overall training objective is to minimize the following loss function:

	$\displaystyle\mathcal{L}(\theta):=$	$\displaystyle\mathbb{E}_{(x,y)\sim\mathbb{D}_{l}}\left[\ell_{\text{CE}}(f(x;\theta),y)\right]$		(3)
		$\displaystyle+\lambda\>\mathbb{E}_{x_{u}\sim\mathbb{D}_{u}}\left[\,\ell_{\text{CE}}(f(x_{u};\theta),\hat{y}(x_{u}))\,\right]$		(3)

where the cross-entropy loss $\ell_{\text{CE}}(f(x),y)=-\log\sigma_{y}(x)$ quantifies the discrepancy between the label $y$ and prediction logits $f(x)$ , and $\sigma_{y}(x)$ is given as the probability after the softmax operation; $\lambda>0$ balances the labeled and unlabeled terms.

Our method: As described in the scenario setting, client $k$ maintains a labeled dataset $\mathbb{D}_{l}^{(k)}$ and unlabeled dataset $\mathbb{D}_{u}$ , mirroring the typical SSL structure. To optimally utilize both datasets, we design the objective function based on Eq. 3 as:

	$\displaystyle\mathcal{L}(\theta):=$	$\displaystyle\underbrace{\mathbb{E}_{(x,y)\sim\mathbb{D}_{l}^{(k)}}\left[\ell_{\text{CE}}(f(x;\theta),y)\right]}_{:\mathcal{L}_{l}(\theta)}$		(4)
		$\displaystyle+\lambda\>\underbrace{\mathbb{E}_{x_{u}\sim\mathbb{D}_{u}}\left[\ell_{\text{CE}}(f(x_{u};\theta),\hat{y}(x_{u}))\right]}_{:\mathcal{L}_{u}(\theta)}$		(4)

where $\mathcal{L}_{l}(\theta)$ and $\mathcal{L}_{u}(\theta)$ denote the losses on the labeled and unlabeled datasets, respectively.

As introduced before, client $k$ first fits a model $\theta_{0}$ on its labeled data, detailed as $\theta_{0}:=\arg\min\nolimits_{\theta}\mathcal{L}_{l}(\theta)$ . Ideally, $\theta_{0}$ would master classes it has encountered, i.e., $\mathbb{C}_{J}^{(k)}\bigcup\mathbb{C}_{N}^{(k)}$ . However, in practice, the severe under-representation of minority classes $\mathbb{C}_{N}^{(k)}$ leads to poorly calibrated posterior probabilities, which often drift toward majority classes [20]. Furthermore, $\theta_{0}$ lacks the ability to recognize samples from missing classes as it has never learned them during training.

To mitigate misclassification of minority and missing classes, we enlarge the label space with an auxiliary ‘unknown’ class $u$ , which acts as a catch-all for samples with unreliable predictions. With the label space augmented to $\mathbb{C}\,\cup\,\{u\}$ , the model then satisfies $\sum_{y\in\mathbb{C}}\sigma_{y}(x_{u})+\sigma_{u}(x_{u})=1$ .

Building on this, we ideally partition the unlabeled dataset $\mathbb{D}_{u}$ into samples from majority classes $\mathbb{C}_{J}^{(k)}$ , and all others. If the sample’s true class belongs to client $k$ ’s majority set $y(x_{u})\in\mathbb{C}_{J}^{(k)}$ , the pseudo-label is assigned to: $m(x_{u}):=\arg\max_{c\in\mathbb{C}_{J}^{(k)}}(f_{c}(x_{u};\theta_{0}))$ , as abundant local examples enable the client to identify these classes reliably [28]; if $y(x_{u})\not\in\mathbb{C}_{J}^{(k)}$ , the sample is intended to be classified as $u$ . Then, the ideal pseudo-label assignment is

\hat{y}(x_{u})={m(x_{u})}\,\text{ if }\,y(x_{u})\in\mathbb{C}_{J}^{(k)}\,\text{ else }\,u

(5)

The unlabeled loss $\mathcal{L}_{u}(\theta)$ is therefore reformulated as:

	$\displaystyle\mathbb{E}_{x_{u}\sim\mathbb{D}_{u}}\big[$	$\displaystyle\mathbf{1}[y(x_{u})\in\mathbb{C}_{J}^{(k)}]\>\ell_{\text{CE}}(f(x_{u};\theta),m(x_{u}))$		(6)
		$\displaystyle+(1-\mathbf{1}[y(x_{u})\in\mathbb{C}_{J}^{(k)}])\>\ell_{\text{CE}}(f(x_{u};\theta),u)\,\big]$		(6)

where the indicator $\mathbf{1}[P]$ equals 1 if the event $P$ is true and 0 otherwise. Since the truth labels $y(x_{u})$ are unavailable, we approximate this indicator with a soft weight $\alpha(x_{u})\in[0,1]$ :

	$\displaystyle\mathcal{L}_{u}(\theta)\approx\mathbb{E}_{x_{u}\sim\mathbb{D}_{u}}\big[$	$\displaystyle\alpha(x_{u})\,\ell_{\text{CE}}(f(x_{u};\theta),m(x_{u}))$		(7)
		$\displaystyle+(1-\alpha(x_{u}))\,\ell_{\text{CE}}(f(x_{u};\theta),u)\,\big]$		(7)

The comparison between Eq. 4 and Eq. 7 yields the pseudo-label in probabilistic-vector form:

\hat{\mathbf{y}}(x_{u}):=\alpha(x_{u})\cdot\mathbf{e}_{m(x_{u})}+(1-\alpha(x_{u}))\cdot\mathbf{e}_{u}

(8)

where $\mathbf{e}_{m(x_{u})}$ is the one-hot vector for the predicted class $m(x_{u})$ , and $\mathbf{e}_{u}$ represents that of class $u$ . This pseudo-label distributes probability mass over only two classes: the predicted class $m(x_{u})$ and the unknown class $u$ . We detail the concrete design of $\alpha$ in the following subsection.

IV-B Weight design

The weight $\alpha(x_{u})$ re-weighs the local loss on unlabeled data so that learning is guided by global rather than local class prevalence. Instead of relying on class probabilities, which risk privacy leakage, we derive the weights solely from entropies, since they are inexpensive to compute and reflect uncertainty.

Algorithm 1 Algorithm of FedCC pipeline

K

clients, global rounds

T

, local labeled dataset

\{\mathbb{D}_{l}^{(k)}\}_{k=1}^{K}

, public unlabeled dataset

\mathbb{D}_{u}

2: Training:

3: for

t=1

T

4: Client

k

(in parallel):

5: Obtain model

\theta_{sync}^{(k),t}

by solving the optimization problem specified in Eq. 2. (Omit when

t=1

)

\theta_{0}^{(k),t}\leftarrow\arg\min\nolimits_{\theta^{(k)}}\mathcal{L}_{l}(\theta^{k})

\triangleright

Initialize at

\theta_{sync}^{(k),t}

\theta^{(k),t}\leftarrow\textsc{FedCC\_Local}(\theta_{0}^{(k),t},\mathbb{D}_{l}^{(k)},\mathbb{D}_{u})

8: Compute pseudo-predictions:

\hat{\mathbf{y}}^{(k),t}(x_{u})\leftarrow\sigma_{\tau}^{(k)}(x_{u};\theta^{(k),t})\;\forall\,x_{u}\in\mathbb{D}_{u}

10: Upload

\{\hat{\mathbf{y}}^{(k),t}(x_{u})\}

to server

11: Server:

12:

\mu^{(k),t}(x_{u})\leftarrow\text{normalize }\{1-\hat{y}_{u}^{(k),t}(x_{u})\}_{k}

13:

\tilde{\mathbf{y}}^{(s),t}(x_{u})\leftarrow\sum\nolimits_{k}\mu^{(k),t}(x_{u})\,\hat{\mathbf{y}}^{(k),t}(x_{u})

14: Set the unknown-class entry of

\tilde{\mathbf{y}}^{(s),t}(x_{u})

to zero, and renormalize the remaining entries to get

\hat{\mathbf{y}}^{(s),t}(x_{u})\in\mathbb{R}^{C}

15: Broadcast

\{\hat{\mathbf{y}}^{(s),t}(x_{u})\}

back to all clients

16: end for

17: return

\{\theta^{(k),T}\}_{k=1}^{K}

18: procedure FedCC_Local(

\theta_{0}^{(k),t},\mathbb{D}_{l}^{(k)},\mathbb{D}_{u}

)

19: Generate pseudo-labels via Eq. 8 with

\alpha^{(k),t}(x_{u})

from Eq. 10

20: Obtain model

\theta^{(k),t}

by optimizing Eq. 4

21: return

\theta^{(k),t}

22: end procedure

23:

24: Inference: Given a new input

x

25: Generate

\tilde{\mathbf{y}}^{(s),T}(x_{u})

via Eq. 11

26:

\hat{y}=\arg\max\nolimits_{c\in\mathbb{C}}\tilde{\mathbf{y}}^{(s),T}_{c}(x_{u})

Weight towards majority classes ( $w_{l}$ ): Following [29], we regard a solid-color image $x_{0}$ as featureless, and probe the classifier’s intrinsic bias by feeding it into the pre-trained model, recording $\mathbf{p}^{(k)}:=\sigma(x_{0};\theta_{0})$ . To keep our entropy-based weighting scheme responsive even for near-certain predictions, we apply label smoothing [30]:

\tilde{\mathbf{p}}^{(k)}=(1-\delta)\,\mathbf{p}^{(k)}+\delta\frac{\mathbf{1}}{|\mathbb{C}|}

(9)

The weight is then $w_{l}:=H\bigl(\tilde{\mathbf{p}}^{(k)}\bigr)$ , where $H(\cdot)$ denotes entropy. The smoothed entropy quantifies how strongly the model already favors its local label set and stays strictly positive even under a one-class bias.

Weight over unknown classes ( $w_{u}$ ): Classes under-represented by client $k$ are collected in $\mathbb{C}_{N}^{(k)}\bigcup\mathbb{C}_{S}^{(k)}$ with number being $|\mathbb{C}|-|\mathbb{C}_{J}^{(k)}|$ . Since the client lacks prior knowledge for those classes, we assign them the maximum uncertainty, defined as the entropy of a uniform distribution. This yields $w_{u}:=\log(|\mathbb{C}|-|\mathbb{C}_{J}^{(k)}|+1)$ . Here, we add one virtual category to account for residual ambiguity introduced by the majority classes, reflecting that the client remains uncertain even with only one under-represented class.

Since $w_{l}$ quantifies the client’s concentration of probability over majority classes and $w_{u}$ captures its uncertainty about unknown classes, a natural way to trade off these two effects is through their ratio: $\alpha_{base}=\frac{w_{l}}{w_{l}+w_{u}}\in[0,1]$ . Note that $w_{l}$ and $w_{u}$ are client-specific and vary across clients.

To further account for sample-specific prediction uncertainty, we quantify the pretrained model’s confidence for each unlabeled instance $x_{u}$ by $w_{x_{u}}:=\min\big(w_{l},H(\sigma(x_{u};\theta_{0}))\big)$ . As high entropy indicates low confidence in assigning the sample to a class in $\mathbb{C}$ , we penalize $\alpha_{base}$ with $w_{x_{u}}$ , leading to the final $\alpha(x_{u})$ expression:

\alpha(x_{u}):=\frac{w_{l}-w_{x_{u}}}{w_{l}+w_{u}}

(10)

This weighting factor $\alpha$ limits the extent to which a local model’s bias affects pseudo-labeling. $\alpha$ reserves a non-zero probability for the ‘unknown’ class, allowing the model to consider an alternative label rather than forcing a majority-class assignment. By deferring these uncertain cases, $\alpha$ curbs early mislabelling of minority samples and prevents such errors from cascading through subsequent training rounds.

After generating pseudo-labels, each client updates its local model via Eq. 4 to obtain $\theta$ , which is then used to generate predictions shared with the server.

IV-C Federated training pipeline

The training process runs over $T$ communication rounds. For notational clarity, we use the superscript $(k)$ and $(s)$ to denote the client index and the server-aggregated information, respectively, and $t$ to indicate the communication round.

At the beginning of round $t$ , each client $k$ downloads the aggregated soft targets from the previous round, denoted as $\hat{y}^{(s),t-1}(x_{u}),\forall x_{u}\in\mathbb{D}_{u}$ , and then use Eq. 2 to synchronize its local model. Then each client adapts its model to its private labeled dataset $\mathbb{D}_{l}^{(k)}$ , resulting in the intermediate model $\theta_{0}^{(k),t}$ . Unlike prior distillation-based FL schemes that share predictions immediately, FedCC inserts an extra refinement step. Each client generates the pseudo-label via Eq. 8 and obtains the final round-specific parameters $\theta^{(k),t}$ by optimizing the objective in Eq. 4. Each client then evaluates all samples in $\mathbb{D}_{u}$ and produces temperature-scaled prediction vectors, $\hat{\mathbf{y}}^{(k),t}(x_{u})=\sigma_{\tau}^{(k)}(x_{u};\theta^{(k),t})\in\mathbb{R}^{C+1}$ , where the last entry corresponds to the unknown class. These prediction vectors are uploaded to the server. Clients’ contributions are weighted by known-class confidence: we take $1-\hat{y}_{u}^{(k),t}(x_{u})$ as a preliminary weight, then normalize it across clients to produce $\mu^{(k),t}(x_{u})$ . The server then aggregates the received predictions as

\tilde{\mathbf{y}}^{(s),t}(x_{u})=\sum\nolimits_{k}\mu^{(k),t}(x_{u})\>\hat{\mathbf{y}}^{(k),t}(x_{u})

(11)

It then zeroes out the unknown-class component and renormalizes the remaining $C$ entries to yield a valid probability vector $\hat{\mathbf{y}}^{(s),t}(x_{u})\in\mathbb{R}^{C}$ . The resulting soft targets are then redistributed to all clients to initiate round $t+1$ .

During inference, we adopt the ensemble strategy used in FedMD [31] and FedOV [16]. After completing all $T$ rounds, clients transmit their model $f^{(k)}$ with parameters $\theta^{(k),T}$ to the server. The server then aggregates the predictions from all client models using Eq. 11 and selecting the most probable label among the original classes $\mathbb{C}$ . Formally, the predicted label is given by $\hat{y}=\arg\max\nolimits_{c\in\mathbb{C}}\tilde{\mathbf{y}}^{(s),T}_{c}(x_{u})$ .

Algorithm 1 details the complete FedCC procedure.

V Experiments

V-A Experimental details

TABLE I: Performance comparison under various label skews. FedCC demonstrates superior performance across almost all scenarios, and an increasing performance gap as label skew intensifies.

Method TinyImageNet CIFAR-100 CIFAR-10 $p\sim\text{Dir}(\eta)$ $|\mathbb{C}_{J}^{(k)}|=N$ $p\sim\text{Dir}(\eta)$ $|\mathbb{C}_{J}^{(k)}|=N$ $p\sim\text{Dir}(\eta)$ $|\mathbb{C}_{J}^{(k)}|=N$ $0.5$ $0.05$ $40$ $20$ $0.5$ $0.05$ $20$ $10$ $0.5$ $0.05$ $2$ $1$ FedAvg $36.2_{\pm 1.6}$ $24.1_{\pm 1.3}$ $31.1_{\pm 1.9}$ $14.3_{\pm 0.8}$ ${\color[rgb]{0.804,0.4,0}49.3_{\pm 1.2}}$ $34.2_{\pm 2.1}$ $33.5_{\pm 1.3}$ $18.2_{\pm 1.3}$ $72.6_{\pm 2.6}$ $44.3_{\pm 4.2}$ $37.5_{\pm 1.0}$ $11.0_{\pm 1.1}$ FedProx $35.0_{\pm 3.0}$ $26.4_{\pm 1.0}$ $28.4_{\pm 0.5}$ $13.1_{\pm 2.0}$ $48.9_{\pm 0.9}$ $29.3_{\pm 0.7}$ $30.6_{\pm 1.8}$ $16.1_{\pm 0.6}$ $70.1_{\pm 2.3}$ $47.2_{\pm 2.0}$ $41.0_{\pm 1.3}$ $10.6_{\pm 0.4}$ FedNova $34.9_{\pm 1.1}$ $25.7_{\pm 0.6}$ $27.2_{\pm 1.3}$ $14.1_{\pm 1.0}$ $47.8_{\pm 1.7}$ $33.7_{\pm 1.7}$ $37.9_{\pm 1.5}$ $20.9_{\pm 1.1}$ $75.3_{\pm 1.1}$ $42.5_{\pm 0.7}$ $35.2_{\pm 0.8}$ $10.0_{\pm 0.0}$ FedRS $32.5_{\pm 1.3}$ $22.1_{\pm 1.0}$ $27.7_{\pm 1.4}$ $12.9_{\pm 1.5}$ $46.3_{\pm 0.8}$ $27.8_{\pm 0.9}$ $34.2_{\pm 1.4}$ $17.7_{\pm 1.2}$ ${\color[rgb]{0,0.545,0}76.6_{\pm 1.9}}$ $53.0_{\pm 1.0}$ $35.4_{\pm 1.5}$ $10.0_{\pm 0.0}$ FedSAM $35.5_{\pm 0.8}$ $30.3_{\pm 1.1}$ $29.4_{\pm 1.2}$ $14.1_{\pm 0.9}$ $48.7_{\pm 1.3}$ $33.7_{\pm 1.3}$ $35.3_{\pm 1.7}$ $16.5_{\pm 0.8}$ $74.6_{\pm 0.6}$ $43.8_{\pm 1.6}$ $32.8_{\pm 1.3}$ $10.0_{\pm 0.0}$ FedLC $39.2_{\pm 0.9}$ $28.0_{\pm 1.0}$ $28.8_{\pm 1.0}$ $12.5_{\pm 1.3}$ $48.5_{\pm 1.1}$ $34.6_{\pm 0.7}$ $39.4_{\pm 1.1}$ $21.4_{\pm 0.4}$ $76.0_{\pm 1.2}$ $41.5_{\pm 0.6}$ $36.2_{\pm 1.6}$ $10.0_{\pm 0.0}$ FedMR $34.8_{\pm 0.9}$ $24.7_{\pm 0.5}$ $29.2_{\pm 0.8}$ $15.1_{\pm 1.3}$ ${\color[rgb]{0,0.545,0}50.1_{\pm 1.2}}$ ${\color[rgb]{0.804,0.4,0}34.6_{\pm 0.9}}$ $38.6_{\pm 1.5}$ $21.8_{\pm 1.0}$ $76.2_{\pm 0.9}$ $57.8_{\pm 1.0}$ $38.4_{\pm 0.9}$ $11.7_{\pm 0.2}$ FedConcat $28.3_{\pm 2.5}$ $18.7_{\pm 3.9}$ $22.3_{\pm 2.3}$ $9.9_{\pm 1.2}$ $40.3_{\pm 2.4}$ $21.2_{\pm 1.7}$ $20.6_{\pm 0.4}$ $14.7_{\pm 1.7}$ $61.3_{\pm 1.0}$ $34.0_{\pm 2.1}$ $28.5_{\pm 1.2}$ $12.7_{\pm 0.7}$ FedVLS $26.4_{\pm 1.4}$ $20.7_{\pm 0.4}$ $22.4_{\pm 0.7}$ $15.3_{\pm 2.6}$ $43.5_{\pm 1.6}$ $27.3_{\pm 1.2}$ $27.1_{\pm 3.5}$ $18.3_{\pm 0.3}$ $66.0_{\pm 2.8}$ $39.5_{\pm 0.8}$ $32.9_{\pm 3.6}$ $10.0_{\pm 0.0}$ FedMD $30.1_{\pm 1.7}$ $24.8_{\pm 1.5}$ $30.2_{\pm 0.8}$ $23.1_{\pm 1.6}$ $36.4_{\pm 0.7}$ $28.1_{\pm 0.7}$ $34.9_{\pm 1.2}$ $26.4_{\pm 1.6}$ $69.0_{\pm 0.7}$ $34.2_{\pm 2.6}$ $54.5_{\pm 1.0}$ $10.7_{\pm 0.9}$ FedDF $26.8_{\pm 2.0}$ $16.9_{\pm 2.7}$ $20.7_{\pm 0.6}$ $10.3_{\pm 2.8}$ $33.6_{\pm 1.2}$ $22.1_{\pm 0.8}$ $27.6_{\pm 3.1}$ $14.1_{\pm 1.1}$ $63.9_{\pm 1.9}$ $39.9_{\pm 3.7}$ $50.8_{\pm 1.4}$ $11.7_{\pm 1.0}$ LSR $26.2_{\pm 0.9}$ $16.7_{\pm 1.1}$ $23.8_{\pm 1.2}$ $10.2_{\pm 0.6}$ $31.8_{\pm 1.6}$ $21.3_{\pm 0.8}$ $27.8_{\pm 1.4}$ $16.5_{\pm 0.3}$ $58.6_{\pm 1.0}$ $31.4_{\pm 0.7}$ $27.2_{\pm 1.4}$ $10.0_{\pm 0.0}$ FedET $23.8_{\pm 3.8}$ $18.5_{\pm 2.0}$ $21.3_{\pm 0.8}$ $11.6_{\pm 1.0}$ $27.2_{\pm 0.9}$ $22.3_{\pm 1.7}$ $28.0_{\pm 1.1}$ $22.1_{\pm 0.8}$ $62.1_{\pm 2.0}$ $38.4_{\pm 2.7}$ $51.2_{\pm 0.6}$ $10.0_{\pm 0.0}$ FedHKT $27.7_{\pm 1.0}$ $21.3_{\pm 0.9}$ $26.9_{\pm 1.7}$ $14.6_{\pm 0.4}$ $34.4_{\pm 1.9}$ $25.6_{\pm 0.5}$ $31.8_{\pm 3.6}$ $21.3_{\pm 0.9}$ $67.6_{\pm 1.4}$ $44.9_{\pm 0.9}$ $49.1_{\pm 0.4}$ $11.1_{\pm 1.2}$ FedCT $25.9_{\pm 0.5}$ $17.7_{\pm 1.3}$ $21.2_{\pm 0.7}$ $9.9_{\pm 2.6}$ $33.2_{\pm 0.4}$ $21.8_{\pm 1.6}$ $26.7_{\pm 0.8}$ $12.2_{\pm 2.3}$ $65.0_{\pm 1.2}$ $29.7_{\pm 1.7}$ $24.1_{\pm 0.2}$ $10.0_{\pm 0.0}$ FedCC ${\color[rgb]{0,0.545,0}40.6_{\pm 0.8}}$ ${\color[rgb]{0,0.545,0}34.8_{\pm 1.0}}$ ${\color[rgb]{0,0.545,0}36.9_{\pm 2.0}}$ ${\color[rgb]{0,0.545,0}34.3_{\pm 1.3}}$ $48.9_{\pm 1.3}$ ${\color[rgb]{0,0.545,0}40.5_{\pm 1.8}}$ ${\color[rgb]{0,0.545,0}45.3_{\pm 1.4}}$ ${\color[rgb]{0,0.545,0}42.6_{\pm 1.3}}$ $74.7_{\pm 1.6}$ ${\color[rgb]{0,0.545,0}60.9_{\pm 2.2}}$ ${\color[rgb]{0,0.545,0}68.2_{\pm 1.1}}$ ${\color[rgb]{0,0.545,0}67.3_{\pm 1.8}}$

TABLE II: Local model performance over local and global test data, where FedCC leads in almost every case. (CIFAR-100 subset)

Test data Local model $p\sim\text{Dir}(\eta)$ $|\mathbb{C}_{J}^{(k)}|=N$ $0.5$ $0.05$ $20$ $10$ FedAvg ${\color[rgb]{0.804,0.4,0}37.9_{\pm 2.2}}$ $60.0_{\pm 2.7}$ ${\color[rgb]{0.804,0.4,0}57.4_{\pm 3.1}}$ ${\color[rgb]{0.804,0.4,0}72.3_{\pm 4.9}}$ Local FedLC $36.1_{\pm 1.4}$ ${\color[rgb]{0.804,0.4,0}60.5_{\pm 4.4}}$ $56.9_{\pm 5.1}$ $72.2_{\pm 6.0}$ FedMR $36.7_{\pm 2.2}$ $60.2_{\pm 5.1}$ $56.7_{\pm 4.7}$ $70.6_{\pm 5.3}$ SSL $34.6_{\pm 2.0}$ $58.0_{\pm 2.9}$ $54.6_{\pm 2.0}$ $70.5_{\pm 5.2}$ FedCC ${\color[rgb]{0,0.545,0}38.2_{\pm 2.3}}$ ${\color[rgb]{0,0.545,0}64.4_{\pm 2.6}}$ ${\color[rgb]{0,0.545,0}60.0_{\pm 4.6}}$ ${\color[rgb]{0,0.545,0}76.6_{\pm 3.3}}$ FedAvg $17.6_{\pm 1.4}$ $10.1_{\pm 0.9}$ $11.3_{\pm 0.8}$ $7.2_{\pm 0.6}$ Global FedLC $19.0_{\pm 1.0}$ ${\color[rgb]{0.804,0.4,0}11.0_{\pm 1.6}}$ ${\color[rgb]{0.804,0.4,0}11.8_{\pm 1.0}}$ $7.0_{\pm 0.8}$ FedMR ${\color[rgb]{0.804,0.4,0}19.3_{\pm 1.5}}$ $10.7_{\pm 1.7}$ $11.6_{\pm 1.1}$ $7.0_{\pm 0.8}$ SSL $15.4_{\pm 1.1}$ $10.2_{\pm 1.3}$ $11.4_{\pm 0.9}$ ${\color[rgb]{0.804,0.4,0}7.3_{\pm 0.5}}$ FedCC ${\color[rgb]{0,0.545,0}71.1_{\pm 0.3}}$ ${\color[rgb]{0,0.545,0}74.7_{\pm 1.2}}$ ${\color[rgb]{0,0.545,0}71.1_{\pm 2.4}}$ ${\color[rgb]{0,0.545,0}77.9_{\pm 3.8}}$

Dataset settings: Following the previous work [20, 21], we conduct classification tasks using CIFAR-10, CIFAR-100 and TinyImageNet datasets. We randomly divide each dataset into two parts: a public dataset and a private dataset. The public dataset, which comprises $10\%$ of the total training samples, has all labels removed and is shared among all participants. To simulate different label skews in the private dataset, we use two partition methods: (1) Dirichlet distribution $p\sim\text{Dir}(\eta)$ : for each class $c$ , we sample $p_{c}\sim\text{Dir}_{K}(\eta)$ , then assign client $k$ a fraction $p_{c,k}$ of the class- $c$ samples to its private set. A lower $\eta$ value indicates greater heterogeneity. (2) Pathological partition $|\mathbb{C}_{J}^{(k)}|=N$ : each client has data from a fixed number of classes $N$ , with an equal number of samples for each class distributed among the assigned clients.

Baseline: We benchmark FedCC against nine parameter-based methods (FedAvg [1], FedProx [13], FedNova [12], FedRS [19], FedSAM [18], FedLC [20], FedMR [15], FedConcat [22], FedVLS [21]) and six distillation-based methods (FedMD [31], FedDF [11], LSR [30], FedET [32], FedHKT [25], FedCT [26]). For fairness, we exclude methods requiring a labeled public dataset or those based on generating labeled data. For details, please refer to the corresponding paper.

Experiment settings: Inspired by [25], parameter-based FL uses ResNet20 on all nodes; in distillation-based FL, clients split evenly between ResNet14 and ResNet20. The setup includes $10$ clients, $15$ global epochs, and $20$ local epochs per step in client model training. We adopt Adam optimizer with batch size $64$ , learning rate $0.001$ . Temperature $\tau$ is set to $1$ , tradeoff $\lambda$ in Eq. 4 to $0.2$ , $\delta$ in $\tilde{\mathbf{p}}^{(k)}$ (Eq. 9) to $10^{-3}$ . We categorize classes by client $k$ ’s bias $\mathbf{p}^{(k)}$ (Eq. 9): a class $c$ is a majority if $p^{(k)}_{c}>2\%$ , minority if $0<p^{(k)}_{c}\leq 2\%$ , and missing if $p^{(k)}_{c}=0$ . We report results over three random seeds, with model accuracy shown without percent signs.

V-B An overall comparison

Experiment results, as summarized in Table I, show that FedCC outperforms current FL methods across almost all tested scenarios, with its advantage widening as label skew intensifies. On heavy skew ( $p\sim\text{Dir}(0.05)$ or pathological partition), it is ahead by at least 3% points and often by more than 10%. In the harshest scenario where each client has access to only one class of CIFAR-10, baselines collapse to near-random guessing ( $\leq$ 12.7%), yet FedCC still delivers a robust 67.3%. This robustness stems from permitting clients to only predict familiar data and to label uncertain samples as ‘unknown’, thereby abstaining from unreliable predictions and reducing misleading information.

Comparison of local model performance: We evaluate the performance of local models on both local and global test data, as shown in Table II. The models are generated using FedAvg [1], FedLC [20], FedMR [15], standard SSL [27], and our FedCC. For the global test data, if the true label does not belong to the majority classes, classifying it as the ‘unknown’ class is considered valid. As expected, FedCC shows significant improvement on the global test data. On the local test set, FedCC leverages uncertainty to supply richer signals for minority classes and increases their likelihood of being correctly learned, consistently outperforming all baselines.

V-C Performance visualization

((a))

p\sim\text{Dir}(0.05)

((b))

|\mathbb{C}_{J}^{(k)}|=2

Fig. 2: Distribution of decision margins for local model on CIFAR-10.

Decision margin: The decision margin measures the gap between the model’s score for the correct class and its highest incorrect guess. Evaluating under two challenging data partitions, Figure 2 shows margin histograms with a logarithmic y-axis. FedCC markedly shifts the distribution rightward to achieve a positive mean, whereas the best competing method remains negative. This rightward shift confirms that FedCC effectively ensures clear class separation and suppresses client-side misclassification, even under severe label skew.

T-SNE analysis of learned representations: Figure 3 visualizes the learned feature representations where each client holds data from a single class ( $|\mathbb{C}_{J}^{(k)}|=1$ ) on CIFAR-10, where each color denotes a distinct class. The embeddings are obtained by applying PCA whitening followed by t-SNE in a unified pipeline. Baseline: each client trains a local model using standard cross-entropy on its data, with the embedding of a representative client shown in Fig.3(a), and that of the FedAvg global model in Fig.3(b). Due to the lack of inter-class variation, local representations collapse along class-specific axes, and averaging these misaligned feature spaces yields weak global performance. FedCC: we next train local models with FedCC’s objective and aggregate them by ensembling their soft predictions. The corresponding client-side embedding (Fig.3(c)) cleanly isolates the client’s own class from the “unknown” region, and the aggregated embedding (Fig.3(d)) recovers distinct clusters for all classes. FedCC thus curbs error propagation by explicitly separating learned versus unseen classes at the client level, allowing the server to combine information without blurring minority categories.

VI Conclusion

We present FedCC, a novel distillation-based FL algorithm that mitigates client misclassification by incorporating an auxiliary ‘unknown’ class. By enabling clients to defer ambiguous predictions to this class, FedCC acts as an implicit regularizer, preventing overconfidence on majority classes and suppressing noisy updates. Extensive empirical evaluation confirms that it consistently outperforms state-of-the-art baselines in both local and global performance. Crucially, FedCC’s lightweight reliance on logit exchanges and robust performance make it well suited for deployment in real-world communication networks, which involves large numbers of devices with heterogeneous compute power, bandwidth, and data distributions.

References

[1] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas (2017) Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, Cited by: §I, §V-A, §V-B.
[2] P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, et al. (2021) Advances and open problems in federated learning. Foundations and trends® in machine learning. Cited by: §I, §I, §III.
[3] W. Ye, C. Qian, X. An, X. Yan, and G. Carle (2023) Advancing federated learning in 6g: a trusted architecture with graph-based analysis. In GLOBECOM 2023-2023 IEEE Global Communications Conference, pp. 56–61. Cited by: §I.
[4] B. Liu, W. Y. Poe, R. Trivisonno, and G. Caire (2026) Foundation models for generalizable semantic and goal-oriented communication. In ICC 2026 - IEEE International Conference on Communications, Vol. , pp. 1–6. External Links: Document Cited by: §I.
[5] J. Bi, Y. Wang, H. Chen, X. Xiao, A. Hecker, V. Tresp, and Y. Ma (2025) LLaVA steering: visual instruction tuning with 500x fewer parameters through modality linear representation-steering. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Cited by: §I.
[6] G. Hinton (2015) Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. Cited by: §I.
[7] W. Ye, Y. Zhang, X. An, G. Carle, and Y. Ma (2026) Select to think: unlocking SLM potential with local sufficiency. In Forty-third International Conference on Machine Learning, Cited by: §I.
[8] Z. Li, X. Wang, D. Hu, N. M. Robertson, D. A. Clifton, C. Meinel, and H. Yang (2022) Not all knowledge is created equal: mutual distillation of confident knowledge. In Workshop on trustworthy and socially responsible machine learning, NeurIPS 2022, Cited by: §I, §II.
[9] J. Bi, Y. Wang, D. Yan, X. Xiao, A. Hecker, V. Tresp, and Y. Ma (2025) PRISM: self-pruning intrinsic selection method for training-free multimodal data selection. ArXiv abs/2502.12119. Cited by: §I.
[10] R. Dai, Y. Zhang, A. Li, T. Liu, X. Yang, and B. Han (2024) Enhancing one-shot federated learning through data and ensemble co-boosting. arXiv preprint arXiv:2402.15070. Cited by: §I, §II.
[11] T. Lin, L. Kong, S. U. Stich, and M. Jaggi (2020) Ensemble distillation for robust model fusion in federated learning. Advances in neural information processing systems. Cited by: Fig. 1, Fig. 1, §I, §I, §V-A.
[12] J. Wang, Q. Liu, H. Liang, G. Joshi, and H. V. Poor (2020) Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems. Cited by: §I, §V-A.
[13] T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith (2020) Federated optimization in heterogeneous networks. Proceedings of machine learning and systems. Cited by: §I, §II, §V-A.
[14] X. Gong, A. Sharma, S. Karanam, Z. Wu, T. Chen, D. Doermann, and A. Innanje (2021) Ensemble attention distillation for privacy-preserving federated learning. In Proceedings of the IEEE/CVF international conference on computer vision, Cited by: §I.
[15] Z. Fan, J. Yao, R. Zhang, L. Lyu, Y. Zhang, and Y. Wang (2024) Federated learning under partially class-disjoint data via manifold reshaping. arXiv preprint arXiv:2405.18983. Cited by: §I, §II, §V-A, §V-B.
[16] Y. Diao, Q. Li, and B. He (2023) Towards addressing label skews in one-shot federated learning. In International conference on learning representations, Cited by: §I, §II, §IV-C.
[17] W. J. Scheirer, A. de Rezende Rocha, A. Sapkota, and T. E. Boult (2012) Toward open set recognition. IEEE transactions on pattern analysis and machine intelligence. Cited by: §I.
[18] Z. Qu, X. Li, R. Duan, Y. Liu, B. Tang, and Z. Lu (2022) Generalized federated learning via sharpness aware minimization. In International conference on machine learning, Cited by: §II, §V-A.
[19] X. Li and D. Zhan (2021) Fedrs: federated learning with restricted softmax for label distribution non-iid data. In Proceedings of the ACM SIGKDD conference on knowledge discovery & data mining, Cited by: §II, §V-A.
[20] J. Zhang, Z. Li, B. Li, J. Xu, S. Wu, S. Ding, and C. Wu (2022) Federated learning with label distribution skew via logits calibration. In International Conference on Machine Learning, Cited by: §II, §IV-A, §V-A, §V-A, §V-B.
[21] K. Guo, Y. Ding, J. Liang, Z. Wang, R. He, and T. Tan (2025) Exploring vacant classes in label-skewed federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Cited by: §II, §V-A, §V-A.
[22] Y. Diao, Q. Li, and B. He (2024) Exploiting label skews in federated learning with model concatenation. In Proceedings of the AAAI Conference on Artificial Intelligence, Cited by: §II, §V-A.
[23] J. Lu, S. Li, K. Bao, P. Wang, Z. Qian, and S. Ge (2023) Federated learning with label-masking distillation. In Proceedings of the 31st ACM International Conference on Multimedia, Cited by: §II.
[24] D. Campos, M. Zhang, B. Yang, T. Kieu, C. Guo, and C. S. Jensen (2023) LightTS: lightweight time series classification with adaptive ensemble distillation. Proceedings of the ACM on management of data. Cited by: §II.
[25] Y. Deng, J. Ren, C. Tang, F. Lyu, Y. Liu, and Y. Zhang (2023) A hierarchical knowledge transfer framework for heterogeneous federated learning. In IEEE conference on computer communications, Cited by: §II, §V-A, §V-A.
[26] A. Abourayya, J. Kleesiek, K. Rao, E. Ayday, B. Rao, G. I. Webb, and M. Kamp (2025) Little is enough: boosting privacy by sharing only hard labels in federated semi-supervised learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Cited by: §II, §V-A.
[27] D. Yarowsky (1995) Unsupervised word sense disambiguation rivaling supervised methods. In Annual meeting of the association for computational linguistics, Cited by: §IV-A, §V-B.
[28] Y. Zou, Z. Yu, B. Kumar, and J. Wang (2018) Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Proceedings of the European conference on computer vision, Cited by: §IV-A.
[29] H. Lee and H. Kim (2024) CDMAD: class-distribution-mismatch-aware debiasing for class-imbalanced semi-supervised learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Cited by: §IV-B.
[30] L. Yuan, F. E. Tay, G. Li, T. Wang, and J. Feng (2020) Revisiting knowledge distillation via label smoothing regularization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Cited by: §IV-B, §V-A.
[31] D. Li and J. Wang (2019) Fedmd: heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581. Cited by: §IV-C, §V-A.
[32] Y. J. Cho, A. Manoel, G. Joshi, R. Sim, and D. Dimitriadis (2022) Heterogeneous ensemble knowledge transfer for training large models in federated learning. International joint conferences on artificial intelligence. Cited by: §V-A.