Is federated averaging the definition of horizontal federation?

No. It is one training approach within that setting.

Must participants share their raw training rows?

Typically the design avoids central collection of those rows, but useful model information still moves between participants or a coordinator.

Does the same feature space mean identical data?

No. Compatible variables can have very different distributions across sites.

AI Glossary / Mathematics and optimization / Intermediate

Horizontal federated learning

Definition

Horizontal federated learning is collaborative model training across participants that hold different examples described using a compatible feature space and, for supervised learning, compatible label definitions. Training records stay with their holders while participants exchange information such as model updates. Horizontal describes how the data is partitioned, not a particular aggregation algorithm or a guarantee of privacy.

What it means

Imagine several warehouses recording the same kinds of measurements for different machines. Their tables have compatible columns but contain different rows. They could potentially train a common predictor without first collecting all those rows in one central table.
The compatibility requirement is substantive. A column called temperature must have compatible units and meaning, and a failure label must refer to the same event and prediction horizon. Shared column names alone are insufficient. Participants can also have very different data distributions, equipment types, sample counts, and operating conditions despite using the same schema.

How it works

In a common server-coordinated approach, participants receive a shared model, train it locally for specified steps, and return updates. The coordinator aggregates contributions and distributes a revised model for another round. Federated averaging is one influential instance; horizontal federated learning is the broader data-partition setting.
The training design must specify sampling, local work, contribution weights, and how unavailable participants are handled. Security measures depend on the threat model. Secure aggregation can conceal individual contributions within an aggregate under its assumptions, while differential privacy can bound specified disclosures using a defined mechanism. Neither property follows merely from keeping raw records local.

Three practical examples

Several factories with compatible sensor measurements can investigate a shared equipment model while retaining their individual machine histories locally.
A keyboard application can train from separate devices that use a common prediction task and model interface.
Regional service teams can collaborate on a common ticket-classification task when their category definitions and input representation are aligned.

Think of it this way

Think of several teachers improving a shared lesson plan using different classrooms, then reporting suggested changes rather than collecting all student notebooks centrally. The analogy stops at confidentiality: suggestions can still reveal information, and conflicting classroom experiences may make an average unsuitable for some groups.

Worked example

Consider an original one-parameter training example. Site A holds 20 observations and has local average loss F_A(w)=0.5(w−2)^2. Site B holds 80 observations and has F_B(w)=0.5(w−6)^2. These simple quadratic losses stand in for two compatible tasks with different local data.
Start both sites at w=0 and take one full-gradient step with learning rate 0.1. The gradients are −2 at A and −6 at B, so the returned parameters are 0.2 and 0.6. Weighting by sample counts gives a new parameter (20/100)(0.2)+(80/100)(0.6)=0.52.
This matches one gradient step on the sample-weighted objective F=0.2F_A+0.8F_B: its gradient at zero is −5.2, and subtracting 0.1 times that gradient gives 0.52. An equal-site average instead gives 0.4, corresponding to a different weighting policy. Neither weighting should be described as universally fair; they answer different aggregation questions.
The sample-weighted objective is minimized at w=5.2, obtained by solving 0.2(w−2)+0.8(w−6)=0. This common model is not each site's separate optimum, which is 2 at A and 6 at B. A global objective can therefore conceal local trade-offs.
The one-step equivalence uses a common starting parameter, the same learning rate, full local gradients, and the stated weighting. Multiple local steps on general losses need not reproduce centralized training. No privacy mechanism is included in this toy arithmetic; the example only explains aggregation.

Illustrative case study

In a fictional example, Harbor Maintenance has three sites using similar pumps. Teams initially assume their records are ready for horizontal federated learning. A schema review discovers that one site measures operating time in minutes while another uses hours, and their breakdown labels use different time windows.
They align these definitions before running a pilot, then evaluate the shared model separately at every site. One site benefits less because its operating conditions differ. The team considers personalization rather than relying solely on an overall average. This invented case illustrates compatibility and evaluation issues, not measured performance gains.

Common misconceptions and mistakes

Horizontal does not mean that every participant has identical distributions or equal numbers of examples. Non-identically distributed data is a central practical challenge, not an exclusion from the setting.
Do not equate local storage with complete confidentiality. Model updates, participation patterns, and the final model can reveal information unless appropriate protections address the relevant threats. Likewise, cryptographic aggregation does not by itself ensure that malicious contributions are harmless.
Do not confuse horizontal and vertical federation. Horizontal participants contribute different examples with compatible features; vertical collaboration combines different feature views associated with overlapping entities. Real systems may require more complex hybrid arrangements.

Business use

Horizontal federated learning is worth considering when organizations have compatible tasks but cannot conveniently centralize their training records. The business case must include coordination, communication, local computing, monitoring, and governance costs, alongside any modeling benefit.
A pilot should compare local-only and collaborative baselines using consistent evaluation rules. Record performance by participant, not just across the pooled evaluation population. Define responsibilities for model updates and operational failures, and assess the actual privacy design separately. Federation is an architecture choice; it does not automatically establish superior accuracy, lower cost, or permission to use data.

Check your understanding

Two sites record compatible variables for different machines, but one uses a different failure definition. Are they ready for the supervised example described here?

Answer: No. The label meanings need alignment or an explicit model design that handles the difference before their training contributions can be treated as the same task.

Related terms

Federated learning
Federated averaging
Vertical federated learning
Differential privacy
non-IID data
secure aggregation
personalization
distributed optimization

Learn next

Repeat the worked calculation with equal participant weights and explain the changed objective. Then inspect a proposed federation's feature units, label timing, and participant-level evaluation plan before selecting an aggregation method.

Sources

Primary sources: Yang, Liu, Chen, and Tong, Federated Machine Learning: Concept and Applications, section 2.3.1: https://arxiv.org/pdf/1902.04885 . McMahan and colleagues, Communication-Efficient Learning of Deep Networks from Decentralized Data: https://proceedings.mlr.press/v54/mcmahan17a.html . The quadratic aggregation example is original.

Previous: Horizon effect Next: Horn clause