Must all agents take a physical action?

No. A wait or no-operation component can represent an inactive participant in a synchronous model.

Can a joint action have different rewards for different agents?

Yes. Shared reward is a cooperative special case; other models provide separate payoffs.

Is a central controller required?

No. A joint action can result from decentralized choices. Whether those choices are coordinated depends on the policies and available information.

AI Glossary / Graphs and symbolic AI / Intermediate

Joint action

Definition

A joint action is a combination of actions, one for each participating agent, considered together at a decision step. In a multi-agent model, the resulting state change or reward can depend on the entire combination rather than on any agent's action in isolation.

What it means

If two agents each choose between move and wait, their joint action records both choices in an ordered pair. Move-and-wait is different from wait-and-move because a different agent moves.

The term does not imply that the agents cooperate, communicate or share an objective. Competitive players also produce joint actions. Nor does it necessarily mean that one central controller selected every component; separate policies can jointly generate the combination.

A joint action is one step in the model. A joint plan or policy describes behavior across multiple steps or possible observations. The distinction matters when evaluating whether a locally acceptable combination supports a complete coordinated task.

How it works

Define each agent's available actions in the current state. Their Cartesian product contains the possible joint combinations before any additional compatibility restrictions. If agent one has three choices and agent two has four, there are twelve combinations.

The environment's transition rule takes the current state and the joint action and determines the next state or a probability distribution over next states. Reward rules can assign one shared value or different values to different agents.

Interactions must be modeled explicitly. Two individually available moves might conflict, or an action might succeed only when another agent performs a supporting action at the same step. A wait or no-operation choice can represent an agent remaining inactive. The definition of a step, including timing and simultaneity assumptions, should be clear before interpreting the combined result.

Three practical examples

Two simulated robots choose movements whose combination determines whether they compete for the same location.
A cooperative game model gives a reward only when two agents choose complementary roles at the same step.
A multi-agent learning experiment records every participant's selected action to explain a transition that cannot be understood from one participant alone.

Think of it this way

Think of two people deciding whether to pass through a narrow doorway. One person's choice to enter has a different outcome depending on whether the other waits or also enters.

The analogy highlights interaction between choices. A formal model needs explicit rules for timing, conflicts and outcomes; the mere phrase joint action does not supply a real-world coordination protocol.

Worked example

Enumerating combinations and checking expected reward

Consider two agents, A and B, each choosing Enter or Wait in a fictional single-step doorway game. There are 2 × 2 = 4 joint actions: (Enter, Enter), (Enter, Wait), (Wait, Enter) and (Wait, Wait).

Assign a shared reward of 3 when exactly one agent enters, 0 when both wait and −5 when both enter. These are invented teaching values. The reward for A's Enter choice cannot be determined without B's choice: the joint reward is 3 if B waits and −5 if B enters.

If both agents independently choose Enter with probability one-half, each joint action has probability one-quarter. The expected shared reward is (−5 + 3 + 3 + 0)/4 = 0.25.

Now suppose they use a valid shared convention selecting (Enter, Wait) half the time and (Wait, Enter) half the time. Their individual Enter frequencies are still one-half, but the joint distribution differs. The expected reward is now 0.5 × 3 + 0.5 × 3 = 3.

Thus identical individual action frequencies do not determine the joint outcome distribution. The calculation assumes the convention is reliably shared and followed; it does not explain how agents establish it. A deployment model would also need an explicit timing and conflict-handling mechanism rather than relying on this simplified payoff table.

Illustrative case study

The fictional Maple Agents team tests two simulated workers sharing a resource. Each worker's separate report shows that it requests access on half the decision steps, yet the combined simulation contains frequent conflicts.

Reviewers inspect the joint action log and discover that the individual summaries hide whether requests occur together. They evaluate an explicit coordination rule and test what happens when the agents receive inconsistent information about whose turn it is.

Maple keeps the payoff calculation separate from implementation requirements such as timing and acknowledgment. This fictional case shows why evaluation needs joint combinations and interaction assumptions alongside per-agent statistics.

Common misconceptions and mistakes

A joint action always means teamwork. The same representation applies to competing or mixed-interest agents.
Individually reasonable actions necessarily combine well. Shared resources and conditional effects can create conflicts.
Individual action probabilities determine joint probabilities. That requires an independence assumption or another specified dependence model.
One joint action is a complete joint plan. It represents one modeled decision step, not the whole future course of behavior.

Business use

The concept helps evaluate AI systems that share tools, resources or an environment. A useful review asks which combinations are permitted, how simultaneous effects are resolved and what each agent knows when choosing its component.

Action spaces can grow quickly. With n agents each having k options, the unrestricted joint space has k^n combinations. This count motivates methods that exploit limited interactions or other structure, but those methods need evidence that the simplification fits the task. Do not assume that testing each agent separately covers failures that occur only when their actions combine.

Check your understanding

Why do the two worked strategies have different expected rewards despite the same individual Enter frequencies?

Answer: Their joint distributions differ. Independent choices sometimes produce two simultaneous entries, whereas the shared convention allows exactly one entry each time.

Related terms

Action space
Cartesian Product
Joint plan
Stochastic games
Multi-agent system
joint policy
coordination
stochastic game
shared reward

Learn next

Add a third agent with Enter and Wait choices. List the eight joint actions and mark those with exactly one entry. Explain why knowing that each agent enters one-third of the time still does not specify the probability of a conflict.

Sources

Bowling and Veloso define joint action spaces and payoffs in their stochastic-game framework: https://www.cs.cmu.edu/~mhb/papers/02aij.pdf

Pynadath and Tambe model team actions, information and communication: https://www.cs.cmu.edu/afs/cs/project/jair/pub/volume16/pynadath02a-html/index.html

Research on concurrent conditional effects illustrates interactions between simultaneous actions: https://arxiv.org/abs/1906.08157

The doorway payoffs, probability calculations and Maple Agents case are original illustrations, not reported experimental findings.

Previous: Join tree Next: Joint Approximation Diagonalization of Eigen-matrices