An ordered line of items waiting to be processed, usually first in, first out.

Why do AI systems use queues?

To absorb bursts of requests, decouple arrival from processing and keep work orderly when capacity is limited.

What is a priority queue?

A queue where urgent items are processed before older but less important ones.

What should be measured?

Queue length, the age of the oldest item and the gap between arrival rate and completion rate.

The set of items still waiting in a queue; a growing backlog means arrivals are outpacing the capacity to process them.

AI Glossary / AI and machine learning / Beginner

Queue

Definition

A queue is an ordered line of items waiting to be processed, where the item that arrived first is normally handled first, so work is taken in the order it came.

What it means

Queues are one of the most ordinary ideas in computing and one of the most useful in AI systems. Requests, documents, messages and jobs arrive faster than they can be handled, and a queue holds them in order until a worker is free.

The rule is first in, first out. Variants add priority, so urgent items jump ahead, and limits, so a full queue refuses or delays new arrivals instead of collapsing.

Queues decouple the pace of arrival from the pace of work. A burst of five hundred requests does not crash the system; it waits in line while workers handle them, and the line shrinks as capacity catches up.

For a business, queues also appear as human review lines: the flagged items waiting for a person. Their length and wait time are a direct measure of whether an automated system is keeping up.

How it works

Items arrive and are added to the back of the queue, each with a timestamp and any priority marking.

Workers, whether machines or people, take the next item from the front when they are free.

If processing fails, the item is retried a set number of times and then moved to a separate failed list for inspection.

Limits and alerts watch queue length and age, raising a flag when work waits longer than agreed.

Capacity is adjusted by adding workers when the queue grows or slowing intake when it cannot be cleared.

Three practical examples

A document service queues uploaded invoices and extracts fields at a steady rate, so a month-end surge is absorbed without errors.
A language model service queues prompts and serves them in order, with paying customers placed in a faster priority lane.
A fraud team's review queue holds flagged transactions, and a dashboard shows the oldest unreviewed item in hours.

Think of it this way

Think of a supermarket checkout. Customers join a line and are served in turn. A longer line signals that more cashiers are needed, and an express lane is a priority queue.

Worked example

the queue that became wallpaper

Suppose a fraud detector sends flagged transactions to a review queue staffed by four analysts. The figures are illustrative.

A new rule triples the number of flags, the queue grows to four thousand items, and the oldest wait nine days, by which time the flagged payments have already settled.

The fix is queue discipline: a limit on the alert rate, a priority score so the riskiest items surface first, and an alarm when the oldest item exceeds a day.

The general lesson: a queue without a watched age and a staffing plan is just a place where decisions go to expire.

Illustrative case study

An online retailer introduced an AI tool that flagged suspicious orders for manual review. Within weeks the review queue held thousands of items and genuine fraud was slipping through simply because nobody reached it in time. The operations lead made the queue a first-class metric: median wait time, oldest item age and daily arrivals versus completions all appeared on a shared dashboard. The team tuned the model to send fewer, better-ranked flags, added two part-time reviewers at peak and set an automatic release rule for low-risk orders waiting past a deadline. Wait times fell from days to hours, and fraud losses fell with them.

Common misconceptions and mistakes

Queues are only a technical detail. They determine how long work waits, which is often what customers and regulators feel.
A queue guarantees every item gets processed. Items can fail, expire or be dropped unless retries and failure lists are designed.
Adding workers always clears a queue. If the arrival rate stays above total capacity, the queue only grows.

Business use

Put queue length and oldest-item age on the dashboard for every automated flow that feeds humans.

Set an agreed service time for each queue and an alert when it is breached.

Design retries and failed-item lists, so nothing disappears silently.

Match alert volume to reviewer capacity, and rank items so the most important are reviewed first.

Review queue statistics weekly with the team that owns the automation, since they show strain before customers complain.

Check your understanding

What is a queue, and what two numbers tell you whether it is healthy?

Answer: A queue is an ordered line of items waiting for processing, normally first in, first out. Two numbers show its health: the age of the oldest waiting item, which tells you whether work is expiring, and the gap between arrival rate and completion rate, which tells you whether the line will grow or clear. Both belong on the dashboard of any automated flow.

Related terms

Pipeline
Priority queue
Latency
throughput
backlog
human review

Learn next

AI Agents and Automation for Small Business covers review queues, alert limits and error handling for automations: https://aiforbusinessandlife.com/academy/ai-agents-automation-small-business [C4]

Sources

[S2] Google, Machine Learning Glossary; [S24] scikit-learn documentation, Glossary of Common Terms and API Elements. Full URLs are in the source ledger. Verified 11 October 2026.

Previous: Questionnaire error Next: Quickprop