Is a federated learning system always decentralized in every sense?

No. Raw data can remain distributed while a central service coordinates training.

Does every participant need to join every round?

Not necessarily. Participation policies vary, and the algorithm and evaluation must account for the chosen policy.

Are model updates safe to share by definition?

No. Their information content and handling require assessment; the word federated does not establish a privacy guarantee.

AI Glossary / Mathematics and optimization / Intermediate

Federated learning system

Definition

A federated learning system is the complete arrangement of participants, local data, computing resources, communication and software used to train models collaboratively without routinely pooling the raw training records. Participants carry out local computation and exchange information prescribed by a learning algorithm, such as parameter updates. The system includes the practical mechanisms that make these activities work together, not just the mathematical update rule.

What it means

An algorithm describes how model values change. A network describes relationships or communication among participants. A system brings those elements into an operating environment with identities, compatible software, scheduling, monitoring and failure handling. Some systems use a coordinator and one shared model; others support different local models and neighbor exchanges. Keeping data at their sources can address an important operational constraint, but it does not remove the need to examine what messages reveal or whether the learned models work for each participant.

How it works

A typical training cycle begins by identifying eligible participants and the model version they should use. Selected participants prepare local data and perform the agreed computation. Their messages are validated and combined or exchanged according to the algorithm. The system records the resulting version and decides whether to continue, evaluate or stop. Real implementations also need to handle late messages, unavailable devices and incompatible updates. Training and deployment are separate stages: a model obtained through federation still needs evaluation and an appropriate delivery process before it is used for decisions.

Three practical examples

A mobile application can investigate collaborative model training while leaving user records on participating devices and sending only the prescribed training messages.
Several factories can operate a joint learning service with local data preparation and agreed feature definitions, while retaining responsibility for their own measurements.
A research testbed can deliberately disconnect participants to evaluate how a training protocol handles incomplete rounds and whether results remain reproducible.

Think of it this way

Think of a coordinated workshop held across several locations. Participants work with local materials, follow a shared procedure and exchange agreed results. The organizer must also handle attendance, versioned instructions and missing submissions. The analogy captures coordination, but a federated learning system has numerical compatibility and information-leakage concerns that a workshop metaphor cannot resolve.

Worked example

Suppose a fictional coordinator invites four sites to a training round. Each receives a vector containing 100,000 parameters stored as four-byte values. The raw parameter payload is 400,000 bytes per recipient, or 1,600,000 bytes across the four downloads. If all four return an equally sized update, uploads add another 1,600,000 bytes. Total model payload is therefore 3,200,000 bytes, excluding encryption overhead, headers, retries and other messages. Now one site misses the deadline. A policy might continue with three valid updates, but the aggregation denominator must reflect the stated rule. Suppose those sites represent 100, 200 and 300 local examples, and one scalar component of their updated models is 1, 2 and 4. Example-count weighting gives (100 × 1 + 200 × 2 + 300 × 4)/600 = 17/6, approximately 2.833. Equal weighting gives 7/3, approximately 2.333. Neither calculation can be silently substituted for the other. The absent site's examples must not be included in the denominator unless a separately specified procedure supplies its contribution. This arithmetic illustrates one coordinated model-averaging round, not every federated architecture. It also shows why message handling and participation records are part of the system's correctness.

Illustrative case study

In a fictional pilot, Cedar Manufacturing connects four factories to a training coordinator. The team discovers that one site normalizes measurements differently and another returns updates from an old model version. Engineers add schema and version checks before evaluating the training method itself. They also test the policy for a missed deadline and keep a record of which sites contributed to each round. The case illustrates operational preparation; it does not claim improved defect detection or certify that the exchanged updates are private.

Common misconceptions and mistakes

Federated learning is not simply several independent models with no collaboration. It is also not the same as moving raw data into a central warehouse and distributing ordinary training jobs afterward. A working protocol does not guarantee representative participation: frequently unavailable sites can be systematically underrepresented. Authentication can establish a sender's identity without establishing that its update is correct. Keeping raw records local does not prevent every inference about them. Finally, a lower aggregate loss can conceal poor performance at a particular site, so system evaluation should not stop at one average.

Business use

Specify who participates, what is exchanged, how updates are weighted and how model versions are tracked. Establish local data checks and compatible feature meanings before launching training. Measure communication, completion time, device resource use and participant coverage. Decide how to handle invalid, stale or missing contributions and test those decisions deliberately. Evaluate the model across relevant participant groups and data conditions. Match information protections to the actual messages and threat model. Maintain enough operational records to explain a round without collecting unnecessary sensitive content from local datasets.

Check your understanding

Why is including the absent site's example count in the worked aggregation denominator a problem?

Answer: It would scale the received contributions using weight from a site that supplied no valid contribution, changing the specified weighted average unless an explicit missing-update rule justifies that treatment.

Related terms

Federated learning network
Federated averaging
client selection
model versioning
distributed system
secure aggregation

Learn next

draw one full training round, showing the local data boundary and every message crossing it. Add a late participant and an outdated model version, then specify what the system should do in each case. Compare equal-site weighting with example-count weighting on unequal datasets.

Sources

Sources: McMahan and colleagues, Communication-Efficient Learning of Deep Networks from Decentralized Data, https://proceedings.mlr.press/v54/mcmahan17a.html ; Alexander Jung, Federated Learning: From Theory to Practice, https://arxiv.org/pdf/2505.19183 ; Jung and colleagues, The Aalto Dictionary of Machine Learning, https://aaltodictionaryofml.github.io/ADictML_Main.pdf . Payload calculations, weighting arithmetic and the fictional manufacturing pilot are original illustrations of stated system choices.

Previous: Federated learning network Next: Federated stochastic gradient descent