Does a spot instance always stop after a fixed duration?

No. Its lifetime depends on capacity and service rules rather than a universal fixed schedule.

Can spot resources train large models?

They can support suitable training designs, but coordination and checkpoint recovery become especially important across many workers.

Does a lower rate make waiting free?

Even if waiting has no computing charge, delayed results can have operational costs.

AI Glossary / Mathematics and optimization / Intermediate

Spot instance

Definition

A spot instance is a cloud computing instance offered from spare capacity under terms that allow the provider to reclaim it. Its lower price is exchanged for less predictable availability, making interruption handling part of the workload design.

What it means

In Amazon EC2 terminology, a Spot Instance uses available spare computing capacity. Similar purchasing arrangements exist elsewhere, with different names and rules. The precise contract, interruption behavior and billing conditions belong to the provider and service.

For AI work, the attraction is straightforward: experiments, preprocessing and some training jobs can run when capacity is available. The challenge is that a machine disappearing can interrupt an unfinished calculation. A lower hourly rate alone does not establish a lower cost per completed result.

Spot describes a purchasing and availability arrangement, not a different mathematical training method. A compatible instance can perform the same calculations as other instances of its type. The workflow must still handle interrupted execution, lost local state and unavailable replacement capacity.

How it works

A scheduler requests suitable resources and starts work when capacity is allocated. Tasks can be designed as independent units that are safe to retry, or longer processes can save checkpoints to durable storage.

A training checkpoint usually needs more than model weights if the goal is faithful continuation. Optimizer state, training position and relevant randomness state may matter. Saving a checkpoint on a machine that is about to disappear is insufficient unless the saved data survives independently.

When an interruption occurs, the scheduler detects failure, finds usable capacity and resumes or retries according to policy. Some providers emit advance signals, but software should also tolerate abrupt failures. Exact warning and interruption rules must be checked for the selected service.

Capacity diversification can reduce dependence on one resource pool. It does not guarantee uninterrupted service, and performance differences between selected machines should be considered when comparing run costs.

Three practical examples

A research team runs independent image-preprocessing tasks that can be retried from their source files.
A model-training experiment periodically saves recoverable state and resumes after its worker is reclaimed.
A company runs optional overnight evaluation jobs when spare capacity is available, keeping deadline-critical capacity under a separate operational plan.

Think of it this way

Think of borrowing an unused workshop bench at a reduced fee, with the understanding that its owner may need it back. Portable tools and labeled work in progress make moving easier.

The analogy explains preparedness, not cloud guarantees. Actual interruptions, storage durability and charges follow technical behavior and provider terms, rather than an informal agreement.

Worked example

Comparing cost per finished experiment

Use invented cost units, not current cloud prices. An uninterrupted baseline run needs eight computing hours at 10 units per hour, costing 80 units.

Assume a spot run has the same processing speed and costs 4 units per billed computing hour. Checkpointing and restarts add half an hour of computing. Two interruptions each discard twenty minutes of work, adding another two-thirds of an hour. Total billed computing is therefore 8 + 0.5 + 2/3 = 9 1/6 hours.

The assumed computing charge is 4 × 9 1/6 = 36 2/3 units. If durable storage and transfer add 5 units, total modeled cost is 41 2/3 units. Compared with 80, the saving is 38 1/3 units, about 47.9%.

Now suppose replacement machines take three hours to obtain. Under the assumption that this waiting incurs no computing charge, the direct estimate is unchanged, but completion takes at least 12 1/6 hours. The same design may be economical for an experiment and unsuitable for an eleven-hour deadline. Real billing, throughput and failure patterns need measurement.

Illustrative case study

A fictional language-technology team moves a batch of training experiments to reclaimable capacity. Its first restart test restores model weights but omits the optimizer state, so the resumed trajectory differs from the intended continuation.

The team expands its checkpoint contents, stores them outside the worker and verifies recovery using a deliberately interrupted test run. It also makes output publication idempotent so that a retry does not register the same experiment twice.

The project report compares completed experiments, elapsed time and total charges. It retains a separate record of failed attempts. The lesson is that affordable compute becomes useful only when the surrounding workflow produces reliable, traceable results.

Common misconceptions and mistakes

Spot guarantees the cheapest completed job. Repeated work, overhead and delays can change the outcome.
Checkpointing means saving weights alone. The state needed depends on the computation and desired continuation.
An interruption notice replaces fault tolerance. Systems must also cope with failures and notice-handling problems.
Every AI workload is suitable. Tight deadlines or difficult recovery may make interruption costly.
All providers use identical rules. Capacity, termination behavior and charges are service-specific.

Business use

Evaluate spot capacity at the workload level. Useful candidates have flexible timing, recoverable state, independent tasks or acceptable restart costs. Ask how long a job can wait, how much work an interruption loses and where persistent results are stored.

Measure cost per successful run alongside completion-time percentiles. Include storage, transfers, orchestration and operational effort where material. Before wider adoption, demonstrate recovery and check that resumed results meet the experiment's reproducibility requirements. Keep current provider terms separate from an educational example so that a historical discount does not become a budgeting promise.

Check your understanding

Why is the example's percentage saving smaller than the 60% reduction in hourly rate?

Answer: The spot workflow consumes additional computing time and incurs added storage and transfer costs. The comparison concerns finished work, not only the price of an hour.

Related terms

Checkpoint
Fault tolerance
Cloud instance
idempotency
distributed training
workload scheduling

Learn next

Read checkpointing and distributed training. Write a recovery plan for a long-running experiment, including what state survives a worker failure, who restarts it and how duplicate outputs are prevented. Test that plan before relying on its cost estimate.

Sources

Amazon EC2 User Guide, Spot Instances: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html. Provider definition and capacity-dependent availability.

Aalto Dictionary of Machine Learning, spot instance: https://aaltodictionaryofml.github.io/ADictML_Main.pdf. Machine-learning systems context. Rates, timings and the fictional experiment are original illustrations, not current pricing or measured customer outcomes.

Previous: Spoken dialog system Next: Spreading activation