Does a shard have to live on a separate machine?

No. It is a logical portion. Several shards can share one machine, and systems may move or replicate them across machines.

Can one missing model shard prevent loading?

Yes. If it contains required parameters or state without an available replacement, the complete model cannot be reconstructed correctly.

Is dataset sharding itself a learning algorithm?

No. It organizes access and execution. Its configuration can nevertheless affect which examples the learning algorithm sees and how updates are formed.

AI Glossary / MLOps and deployment / Beginner

Shard

Definition

A shard is a logical portion of a larger dataset, model or data structure that can be stored or processed separately. In machine learning, sharding commonly divides training examples or model parameters among workers so a workload can use more than one machine or device.

What it means

A large job may exceed one device's memory or take too long to process sequentially. Splitting its contents into manageable portions creates opportunities for parallel work. The shard is the portion; sharding is the process of creating or assigning those portions.

A shard is not necessarily one file, one batch or one physical server. Its meaning depends on the system. A dataset may have many file shards assigned dynamically to a few workers, while a model shard may hold only selected parameter tensors. Clarifying what is being divided prevents confusion about memory, coverage and coordination.

How it works

Choose a partitioning rule, such as record ranges, a hash of an identifier, or a division of model tensors. Assign the resulting portions to storage locations or workers, and maintain enough metadata to locate them.

For data-parallel training, workers often process different examples while maintaining copies of the model and combining updates. For model sharding, devices hold different portions of model state and communicate the information needed to perform the overall computation. Practical systems can combine these approaches.

Work must be coordinated: examples should not be accidentally omitted or duplicated, and model operations need the appropriate dependencies. Failures, changing worker counts and uneven workloads complicate assignment. A shard plan therefore includes both a partition rule and a way to track what was actually processed.

Three practical examples

1. Training-data loading: a corpus is divided into files that workers read in parallel. Shuffling and worker assignment determine which examples appear in each batch and epoch.

2. Large-model execution: parameter tensors or other model state are distributed across accelerators when one device cannot hold the required working state. Communication becomes part of the computation cost.

3. Database growth: records are distributed across database shards using a key. Queries that identify the relevant shard can avoid contacting the entire cluster, while broad queries may need results from several shards.

Think of it this way

Think of distributing a large reference collection across several reading rooms. Each room holds a portion, and a catalog tells readers where to find it. More rooms create capacity, but readers needing material from several rooms must coordinate their visits.

The analogy is limited: compute workers can exchange data rapidly, and a model shard may be unusable alone. Additional rooms also do not automatically balance demand if almost everyone needs the same shelf.

Worked example

Suppose a training dataset contains 1,200,000 examples and is stored in twelve equal shards of 100,000 examples each. Four workers receive three shards apiece, so each is assigned 300,000 examples.

With a local batch size of 250 and no dropped or duplicated examples, each worker processes 300,000/250=1,200 local batches per epoch. In synchronized data-parallel training, one step can combine four local batches, giving a global batch of 1,000 examples and 1,200 steps for the full epoch.

Now suppose one worker's assigned examples are long documents and take twice as much processing time per example. Equal example counts no longer mean equal work, and synchronized steps may wait for that worker. Repartitioning by approximate token volume or dynamically assigning smaller shards could improve balance, but the loader must still preserve intended coverage.

These calculations describe a hypothetical schedule. Real loaders may pad, repeat, drop incomplete batches or reshuffle between epochs. Those choices change coverage and step counts, so they must be checked rather than inferred from the word “shard.”

Illustrative case study

Fictional case: CorpusBridge moves a training job from two workers to four. Each worker successfully opens its files, but a configuration mistake gives every worker the complete shard list.

The job appears busy and throughput counters increase, yet the same examples are processed repeatedly. The team adds stable example identifiers to its input audit and compares assigned, read and unique counts for an epoch.

After correcting worker assignment, it verifies coverage again and reviews whether the effective batch size changed. This fictional case illustrates that dividing files on disk does not by itself guarantee distributed training processes distinct data correctly.

Common misconceptions and mistakes

“Sharding and replication are the same.” Sharding divides responsibility for different portions. Replication keeps additional copies of the same portion; systems can use both.

“Equal bytes guarantee equal compute.” Examples can differ in sequence length or processing complexity. Balanced storage does not necessarily produce balanced execution.

“More shards always make a job faster.” Very small shards can increase file-open, metadata and scheduling overhead. Cross-shard communication can also dominate.

“A shard is a train - test split.” Sharding is an execution or storage arrangement. Evaluation separation is a statistical design decision that should remain explicit and protected independently.

Business use

For an AI project, identify the bottleneck before choosing a shard strategy: storage bandwidth, worker memory, model capacity or computation. The right partition for one bottleneck may worsen another. Record the intended unit of balance, such as examples, tokens, bytes or expected work.

Monitor per-worker throughput and memory alongside overall progress. Keep manifests and checkpoint information sufficient to resume after a failure without silently changing coverage. For stored models, confirm that all required shards and their configuration are present before loading. For databases, include routing, rebalancing and availability requirements rather than treating the extra machines as independent copies.

Check your understanding

Four workers each read the same three dataset shards. Has the job necessarily covered twelve distinct shards?

Answer: No. Worker count does not establish unique coverage. The assignment may have processed the same three shards four times.

Related terms

Learn next

Learn the difference between data and model parallelism, then trace one epoch's example assignments and batch formation. Inspect manifests and recovery behavior before changing worker counts, and measure imbalance using the workload unit that actually drives processing time.

Model parallelism

Sources

Primary definitions: Google's Machine Learning Glossary, “shard”: https://developers.google.com/machine-learning/glossary#shard . Database context: MongoDB's sharding documentation: https://www.mongodb.com/docs/manual/sharding/ . The training schedule and fictional input-loader case are original illustrations, not framework-specific guarantees.

Previous: Shapiro-Wilk test Next: Shared memory