What is a mini-batch?

The small slice of data used for each weight update, between the extremes of one example and the whole dataset.

Why not use the whole dataset?

Each update would be slow and memory-bound; mini-batches give frequent, stable-enough updates that fit hardware.

Does batch size affect final quality?

Yes, through gradient noise and its interaction with the learning rate; it is a real dial, not just a speed setting.

What is gradient accumulation?

Summing gradients over several small batches before one update, simulating a larger batch within memory limits.

AI Glossary / AI and machine learning / Intermediate

Mini-batch

Definition

A mini-batch is the small slice of training data, commonly 32 to a few thousand examples, used for each weight update instead of the whole dataset or a single example.

What it means

Training on one example at a time is noisy; training on the whole dataset at once is slow and memory-hungry. The mini-batch is the practical middle: enough examples for a stable gradient, few enough for fast, frequent updates.

The batch size is one of training's real dials. Small batches bring noise that can help escape poor minima but slow each pass; large batches bring stable gradients and hardware efficiency but can settle in sharper, less general minima.

Mini-batches are also a memory story. Hardware runs on fixed-size chunks, so batch size decides whether a model fits on the available hardware at all, which is why the number often reflects the machine more than the theory.

Shuffling between epochs matters: batches should be fresh mixes each pass, or the model learns the order of the data instead of the content of the data.

For a business, the mini-batch explains training behavior in reports: unstable loss curves, slow epochs and out-of-memory errors are all batch-size conversations in disguise.

The word batch alone sometimes means the whole dataset in older texts; the mini prefix is the reminder that modern practice slices.

How it works

Shuffle the training data, then cut it into batches of the chosen size for the first pass, or epoch.

For each batch, compute the gradient of the loss over just those examples and take one optimizer step.

Repeat through the dataset, reshuffling each epoch so batches never repeat their composition.

Tune the size against three pressures: gradient stability, hardware memory and total training time.

Pair size changes with learning-rate changes, since the two dials interact: larger batches usually tolerate larger rates.

Three practical examples

A vision model training in batches of 128 images, chosen because that size fills the hardware without spilling.
A fine-tuning run forced down to batches of 4 because the model barely fits in memory, with gradients accumulated over several batches to recover stability.
A tabular model whose loss curve calmed noticeably when the batch size rose from 32 to 256 on noisy data.

Think of it this way

Think of tasting a stew: one spoonful is a jumpy verdict, the whole pot is impractical, but a ladleful gives a fair reading you can act on, several times while the pot cooks.

Worked example

the memory wall

Suppose a team trains a document classifier and the run crashes with out-of-memory errors at batch size 64. The figures are illustrative.

Halving to 32 fits but doubles training time and adds noise to the loss curve, so they accumulate gradients over two batches, recovering the effective size of 64 in stable steps.

Training completes at nearly the original speed with the memory footprint of the smaller batch, a standard compromise between the hardware's limits and the optimizer's preferences.

The lesson travels: when a report says batch size, it is often describing a negotiation with the hardware, not a theoretical choice.

Illustrative case study

A fictional retailer's data team copied a batch size of 8,192 from a research paper without the learning-rate schedule the paper used with it.

Training converged fast to a sharp minimum, and the model underperformed its predecessor on new-season data despite better training scores.

The invented lesson: batch size is one half of a pair. Change it without the matching learning-rate adjustment and you have changed the training, not just the throughput.

Common misconceptions and mistakes

Bigger batches always train faster. They process more per step but often need more epochs to generalize as well, and they demand the learning rate be retuned.
Batch size is a purely theoretical choice. It is mostly a hardware negotiation: memory limits and device efficiency usually set the range.
Batch order does not matter. Unshuffled or repeated batches teach the model the data's ordering, a subtle and real failure.

Business use

Treat batch size as a paired dial with the learning rate; change one and re-check the other.

When training reports look unstable, ask about batch size before blaming the model or the data.

Use gradient accumulation when memory forces small batches, recovering stability without new hardware.

Record batch size and shuffle settings in the run log, because reproducing a model requires reproducing its batches.

When comparing two training runs, compare batch sizes first: different sizes make the runs different experiments, not repeats.

Check your understanding

Why is batch size as much a hardware decision as a modeling one?

Answer: Because the batch must fit in device memory and runs fastest at sizes the hardware prefers, so the theoretical preference for a gradient's stability is negotiated against the machine's limits on every training run.

Related terms

Learn next

Prompt Like a Pro explains how to read training settings in vendor claims and team reports: https://aiforbusinessandlife.com/academy/prompt-like-a-pro-clear-framework. [C3]

Training

Sources

[S2] Google, Machine Learning Glossary; [S12] Aalto University, Dictionary of Machine Learning. Full URLs are in the source ledger. Verified 11 October 2026.

Previous: Min-max optimization Next: Mini Batch K-Means