Is left-of an absolute physical property?

No. It normally depends on a reference frame or viewpoint.

Do intersecting boxes prove that objects touch?

No. The boxes may include background or represent projections of objects at different depths.

Are contains and covers always the same?

No. Formal geometry predicates distinguish some boundary cases, so choose the definition that matches the task.

AI Glossary / Computer vision and multimodal AI / Beginner

Spatial relation

Definition

A spatial relation describes how the positions or extents of two or more things relate in space. Examples include left of, inside, touching, overlapping, and near. In computer vision, these relations connect detected objects into a description of a scene. Their interpretation depends on the coordinate system, object representation, and definition of the relation.

What it means

Recognizing a cup and a table is different from determining whether the cup is on the table, under it, or beside it. Spatial relations express that additional structure.

Some relations concern direction, such as above or to the left. Others concern distance, such as within two meters. Topological relations concern connections and overlap: whether shapes intersect, contain one another, or touch at their boundaries.

A relation can be qualitative, such as nearby, or quantitative, such as a measured separation. Qualitative language still needs an operational meaning when software must act on it. Near for objects on a desktop does not necessarily mean the same distance as near for buildings on a map.

How it works

First identify the entities and represent their locations or shapes. A point, bounding box, segmentation mask, or three-dimensional surface gives different evidence. Then choose the coordinate frame: image coordinates, a calibrated world frame, an observer's viewpoint, or an object's own orientation.

Apply a defined rule or estimate the relation with a model. A simple image-plane left-of rule might compare horizontal centers; a stricter one might require one object's entire box to lie left of the other. Both can be useful, but they answer different questions.

Geometric libraries define precise predicates for relationships such as contains, covers, and touches. Their boundary conventions may differ from casual language. A vision-language system can also predict relations, but a fluent description should still be checked against the available geometry and uncertainty. Store the direction of the relation: contains(A,B) is not generally interchangeable with contains(B,A).

Three practical examples

1. Visual assistance: an application describes a bag as beside a chair so the user can locate it among several detected objects.

2. Robotic manipulation: a system checks whether a target lies inside a permitted workspace, using calibrated geometry and a stated treatment of its boundary.

3. Document understanding: software relates a caption to the image below it or associates a label with a neighboring form field, using layout as well as recognized text.

Think of it this way

Think of arranging furniture on a floor plan. A list of item names tells you what is present; relations such as beside, within, and facing describe how the arrangement works.

The analogy has limits. A two-dimensional plan suppresses height, and a photograph adds perspective and occlusion. A relation inferred in one representation may require additional evidence before it describes the physical scene accurately.

Worked example

Let rectangle A span x = 0 to 4 and y = 0 to 3. Let rectangle B span x = 3 to 6 and y = 1 to 2, in a common continuous coordinate system.

Their intersection spans x = 3 to 4 and y = 1 to 2, so its area is one. A has area twelve and B has area three. Their union area is 12 + 3 − 1 = 14, giving intersection-over-union 1/14, about 0.0714.

Their centers are (2,1.5) and (4.5,1.5). A center-based rule says A is left of B. A stricter entirely-left rule would fail because A's right edge at four extends beyond B's left edge at three. The rectangles can therefore overlap while still satisfying a center-based left-of relation.

Now consider point P = (4,1.5), which lies on A's right boundary. Under Shapely's standard definitions, A covers P but does not contain P, because the point is on the boundary rather than in the interior. An application that intends boundary inclusion should select the appropriate predicate explicitly.

These calculations describe the supplied rectangles and point. If the rectangles are loose object boxes in a photograph, they do not prove that the physical objects intersect or touch in three-dimensional space.

Illustrative case study

Fictional case: a warehouse assistant is asked to find the carton left of a red container. Two camera views show different apparent left-right orderings because the camera positions differ.

The team defines whether requests use the active camera view or a fixed warehouse coordinate frame. It adds clarification in the interface when several cartons satisfy the relation and checks bounding-box uncertainty near the decision boundary.

The design treats viewpoint as part of the instruction's meaning. A spatial relation alone does not necessarily identify the correct object or authorize its manipulation.

Common misconceptions and mistakes

Do not infer physical contact from overlapping image boxes. Occlusion and perspective can create overlap between objects separated in depth.

Do not assume natural-language relations have one universal threshold. Near, beside, and above may depend on scale, reference frame, or the intended task.

Avoid treating every relation as symmetric or transitive. Touching is symmetric, while left-of and containment are directional. Nearness is not generally transitive: A may be near B and B near C without A being near C. Validate any logical deductions against the exact relation definition.

Business use

Spatial relations help businesses turn object detection into useful scene descriptions, layout interpretation, search, and action planning. They provide a bridge between geometry and instructions such as select the item inside this area.

Define the representation, frame, tolerance, and uncertainty handling before connecting a relation to an operational decision. For automated image review, retain the evidence supporting each relation. This makes it easier to distinguish a missed object, an inaccurate boundary, and a mistaken relational rule when a result is disputed.

Check your understanding

A has a center left of B's center, but their boxes overlap. Is the statement A is left of B necessarily incorrect?

Answer: No. It can be correct under a center-based definition and incorrect under an entirely-left definition. State the rule before judging the result.

Related terms

Spatial reasoning
Bounding box
Intersection over union
Scene graph
coordinate frame
topological relation
containment
depth estimation

Learn next

Compare center-based directional rules with full-shape predicates on a few drawn objects. Then repeat the exercise after changing the viewpoint, noting which relations reflect image layout and which require three-dimensional evidence.

Sources

Shapely, Predicates: https://shapely.readthedocs.io/en/stable/predicates.html Shapely, contains: https://shapely.readthedocs.io/en/stable/reference/shapely.contains.html Official documentation was opened for formal geometric relationships and boundary conventions. The broader vision explanation, rectangle calculations, and warehouse case are original teaching examples.

Previous: Spatial Regression Next: Spatial statistics