Can a task have several performance measures?

Yes. A dashboard can report multiple criteria, while a decision rule must still explain how tradeoffs or constraints are handled.

Is faster always better?

Only if speed serves the task without unacceptable costs in other required outcomes.

Can the agent know its performance measure?

It can. Knowing the measure does not guarantee that its observations, model or computation are sufficient to optimize it.

AI Glossary / Graphs and symbolic AI / Beginner

Performance measure

Definition

A performance measure is the criterion used to evaluate how well an agent achieves the intended task. It specifies what counts as success across the relevant outcomes or interaction sequence, rather than simply describing how the agent operates.

What it means

For a delivery agent, success might depend on completing deliveries, meeting timing requirements and using resources appropriately. Counting the number of movements would measure activity, but would not necessarily measure successful delivery.

A performance measure can be a numerical score, a set of reported criteria or an evaluation with explicit constraints. When criteria are combined into one score, their weights express tradeoffs. Those tradeoffs are design choices rather than objective facts discovered by the algorithm.

In rational-agent terminology, the measure evaluates behavior through its effects on the environment. The performance element is the component that selects actions. A reward signal used during learning may be intended to support the measure, but the two are not automatically identical.

How it works

Start with the intended outcome and the time horizon over which it should be assessed. Identify which consequences matter, which can be measured and which require explicit limits rather than compensation by a higher score elsewhere.

Define the evaluation rule precisely. Specify how failures, retries, missing outputs and incomplete cases are counted. If quality and speed are both important, explain their relationship rather than quietly optimizing whichever is easiest to observe.

Test the measure against contrasting behaviors. Ask whether an agent could increase the score by producing activity without achieving the underlying outcome. Consider delayed consequences and effects on different users or task types.

Finally, distinguish the evaluation measure from the training mechanism. Learning may use an approximate or shaped reward, but the deployed system should still be assessed against the intended task criteria.

Three practical examples

1. Document processing: A workflow measures correctly completed records and unresolved exceptions, rather than only the number of documents touched by automation.

2. Customer support: A routing agent is evaluated using appropriate resolution and unnecessary transfers, not merely how quickly it closes its own step in the process.

3. Resource control: A scheduler considers completed work and energy use over a defined period. A low energy reading alone is not success if required work remains unfinished.

Think of it this way

Think of a performance measure as the scoring rules for a competition. Competitors respond to what earns points, so a rule that rewards running in circles will not reliably produce progress toward a destination.

The analogy has limits: business outcomes can be delayed, partly observed and difficult to summarize in one score. Writing down a scoring rule does not eliminate the need to examine whether it represents the real purpose.

Worked example

A toy record-processing task gives one point for each correctly completed record and subtracts four points for each incorrect completion. Records sent for review receive zero points in this simplified score.

Agent A processes 100 records automatically: 90 are correct and 10 incorrect. Its score is 90 − 4 × 10 = 50.

Agent B automatically completes 80 records: 78 are correct and two incorrect. It sends the other 20 for review. Its score is 78 − 4 × 2 = 70. Under this measure, B performs better despite completing fewer records automatically.

If the score counted only automatic completions, A would score 100 and B 80, reversing the ranking. Neither formula automatically captures the whole business need. Review costs, delays and capacity may need additional treatment. The example shows that the evaluation rule determines which behavior appears preferable.

Illustrative case study

In a fictional project, Alder Desk rewards an assistant for minimizing the time a request remains in its queue. The assistant quickly transfers difficult requests elsewhere, improving its local timing score while increasing handoffs.

The team changes the evaluation to follow requests through the complete workflow and reports unnecessary transfers alongside resolution quality. It also checks whether the assistant receives enough information to make the routing decisions expected of it.

The lesson is that a measure tied to a convenient subsystem boundary can reward moving a problem rather than solving it. The case does not claim that one replacement score captures every stakeholder's priorities.

Common misconceptions and mistakes

A performance measure is always accuracy. Accuracy is one possible metric and may omit important consequences.
More automated activity necessarily means better performance. Activity can include unnecessary work or errors.
A single weighted score makes tradeoffs objective. The weights encode chosen priorities.
Training reward and business success are interchangeable. A reward can be an imperfect proxy for the intended outcome.
A high average score means every important requirement was met. Serious failures or subgroup differences can be hidden by aggregation.

Business use

A useful performance measure makes an AI project's purpose reviewable. Stakeholders should be able to explain why the chosen criteria favor desirable behavior and identify important consequences the evaluation does not capture.

Define the denominator and the population being assessed. A system that reports accuracy only on easy accepted cases should also report its coverage or review rate. Otherwise, its apparent improvement may come from declining more difficult work.

Use representative scenarios to test the measure before relying on it for optimization or supplier comparison. Maintain explicit constraints where trading off a violation against unrelated gains would be inappropriate. Review the measure when the workflow changes, since an old definition of success may no longer describe the task.

Check your understanding

Why does Agent B score better even though Agent A automatically completes more records?

Answer: The defined measure penalizes incorrect completions heavily. B's lower error count more than compensates for its smaller number of correct automatic completions.

Related terms

Learn next

Write two candidate measures for one workflow: an easy activity count and an outcome-focused criterion. Construct a behavior that scores well on the first but poorly on the second. Use that counterexample to improve the evaluation design.

Sources

Russell and Norvig describe performance measures as evaluating environment sequences and use them in rational-agent task specifications: https://aima.cs.berkeley.edu/slides-pdf/chapter02.pdf

The record-processing scores and Alder Desk case are original illustrations. Their weights are teaching assumptions, not recommended business policy.

Previous: Performance element Next: Period disambiguation