Does every client need to participate in every round?

No. A round can use a selected subset, with the participation and aggregation rules documented.

Are larger clients always weighted more?

Sample-weighted FedAvg does that, but other schemes may use different objectives and should not be silently treated as equivalent.

Does FedAvg provide differential privacy?

No. Differential privacy requires an additional mechanism and analysis; averaging alone does not establish that guarantee.

AI Glossary / Retrieval and knowledge systems / Intermediate

Federated averaging

Definition

Federated averaging, often shortened to FedAvg, is a federated-learning algorithm that combines locally trained model parameters into a shared model. Participating clients start from a common model, train on their own data and send model information for aggregation. A common aggregation weights each client's contribution by its number of training examples.

What it means

The method allows training to use data distributed across devices or organizations without requiring those raw records to be collected in one central training dataset. The coordinating server exchanges model parameters or updates rather than simply asking clients to upload their examples.

Averaging here concerns corresponding model parameters, not averaging final predictions from unrelated models. Clients need compatible models and a coordinated training procedure. Keeping raw data local can reduce some exposure, but model updates can still reveal information; FedAvg alone is not a complete privacy or security guarantee.

How it works

At the start of a round, the server selects participating clients and distributes the current global model. Each client performs local optimization, such as several minibatch gradient steps, using its own examples. The server then combines the returned parameters or equivalent updates and distributes the new model in a later round.

For a sample-weighted average over participating clients, each client's weight is its example count divided by the total count represented in that aggregation. Other weighting and participation conventions must be stated explicitly.

More local work can reduce communication frequency, but clients with different data distributions may move in different directions. Local learning rates, the number of local steps, client selection and aggregation behavior all affect the training result.

Three practical examples

1. A keyboard-model experiment trains on participating devices while leaving their text examples on those devices, with separate controls for update privacy.

2. Several manufacturing sites test a shared equipment model using local sensor histories and a coordinated model-aggregation process.

3. A distributed recommendation project compares sample-weighted FedAvg with other training approaches when participants have very different amounts and types of interaction data.

Think of it this way

Think of several teams starting from the same draft, improving it using their own local evidence and sending compatible revisions to a coordinator. The coordinator combines corresponding changes according to a stated weighting rule. If the teams work on different document structures or use incompatible definitions, averaging their revisions would not produce a coherent result.

Worked example

Suppose an invented round has three participating clients with 20, 30 and 50 training examples. After local training, one corresponding model parameter has value 1.2 on client A, 0.8 on client B and 1.0 on client C.

The total represented example count is 100, so the sample weights are 0.2, 0.3 and 0.5. The aggregated parameter is 0.2 times 1.2 plus 0.3 times 0.8 plus 0.5 times 1.0 = 0.24 + 0.24 + 0.50 = 0.98.

An unweighted client average would be (1.2 + 0.8 + 1.0)/3 = 1.0, which is different. Neither arithmetic choice is interchangeable with the other: the weighting determines whose training objective receives more influence.

If all clients began this round with parameter 0.9, their updates would be 0.3, minus 0.1 and 0.1. The weighted update is 0.06 minus 0.03 plus 0.05 = 0.08. Adding it to 0.9 again gives 0.98. This equivalence assumes the same starting parameter and weights; it does not show that the aggregate model is more accurate.

Illustrative case study

Fictional case: Cedar Devices pilots a shared prediction model across several sites. Each site trains the same model locally, but their equipment and observation counts differ. The coordinator uses documented sample weights and evaluates the global model on separate local test sets.

The overall average improves while one small site performs worse. The team investigates the mismatch rather than declaring success from the pooled score alone. It tests local-training settings and examines participation patterns, while a separate review assesses update protection and access. The pilot's results remain conditional on the participating sites and evaluation period.

Common misconceptions and mistakes

FedAvg does not average arbitrary independently trained neural networks without alignment. Corresponding parameters must refer to a compatible model, normally developed from a shared starting state within each round.

It is not equivalent in general to centralized gradient descent over all raw data, especially when clients take multiple local steps on different distributions. More local training is not always better. Sample weighting also does not guarantee fair performance for every client, and the absence of raw-data uploads does not make privacy leakage or malicious updates impossible.

Business use

Consider FedAvg when decentralized data access and communication constraints are central to a shared training task. Establish compatible feature definitions, model versions, local optimization settings and aggregation rules. Track which clients participate and how their contributions are weighted.

Evaluate both aggregate and client-level performance, along with communication volume, completion time and robustness to missing participants. Decide separately how to protect updates and validate participants. Compare the full operational cost with simpler alternatives, since federated coordination can add complexity even when the underlying averaging calculation is straightforward.

Check your understanding

Two clients return parameters 2 and 4 after training on 10 and 30 examples. What is their sample-weighted average?

Answer: (10 times 2 + 30 times 4)/40 = 140/40 = 3.5. The unweighted value of three would assign equal influence to clients rather than examples.

Related terms

Federated learning
Stochastic gradient descent
local training
model aggregation
non-IID data
secure aggregation
differential privacy and client sampling

Learn next

reproduce one aggregation round with unequal client sizes, then compare equal-client and equal-example weighting. Examine how multiple local steps affect clients with different data, and evaluate outcomes separately for each participating group.

Sources

Primary sources: McMahan, Moore, Ramage, Hampson and Aguera y Arcas, Communication-Efficient Learning of Deep Networks from Decentralized Data: https://proceedings.mlr.press/v54/mcmahan17a/mcmahan17a.pdf

The authors' publication record and paper versions: https://arxiv.org/abs/1602.05629

Previous: Feature vector Next: Federated gradient descent