Are cross-brackets the same as crossing dependency arcs?

No. This entry concerns constituent spans compared between parses; dependency nonprojectivity is a different structural notion.

Do labels matter to the basic crossing test?

The geometric test concerns boundaries. Labeled precision and recall assess category labels separately.

Is fewer always better?

Fewer crossings means fewer measured boundary conflicts, but overall quality also depends on coverage and other errors.

AI Glossary / Natural language and speech / Intermediate

Cross-brackets

Definition

Cross-brackets, or crossing brackets, are a constituency-parser evaluation measure of incompatible phrase boundaries between a predicted parse and a reference parse. A crossing occurs when two spans overlap without either containing the other. The measure captures a particular structural disagreement, not every possible parsing error.

What it means

A constituency tree groups consecutive words into phrases. Within an ordinary projective tree, these groups are nested or separate. When two different trees are compared, a phrase from one can partially overlap a phrase from the other, exposing incompatible grouping decisions.

For example, a reference may group the first two words of a three-word sequence, while a prediction groups the last two. Both trees can be internally well formed, yet their selected phrases cross when placed over the same word positions.

Crossing-bracket statistics are often reported alongside constituent precision, recall and F1. Zero crossings do not establish that the predicted tree is fully correct. It can omit reference constituents or assign wrong category labels without producing a crossing. The measure is therefore informative but incomplete.

How it works

Represent each constituent by its start and end positions using one consistent indexing convention. For half-open spans [a,b) and [c,d), a crossing occurs when a < c < b < d, or when c < a < d < b. Strict inequalities exclude nesting, identical spans and spans that merely touch at a boundary.

An evaluator compares predicted constituents with the reference constituents. In a common convention, it counts each predicted constituent that crosses at least one reference constituent. Counting every crossing pair instead can produce a different number, so reports must identify the convention and implementation.

Token alignment and evaluation settings matter too. If the two trees use different tokenization, their boundary positions may not be comparable. Treatment of punctuation, root nodes and other excluded categories can change which constituents are scored. A meaningful comparison uses the same evaluator configuration and compatible input representation.

Three practical examples

A parsing researcher inspects sentences with crossing spans to understand where the model chose an incompatible phrase grouping.
A regression check compares a parser's structural errors before and after a model change using unchanged reference trees and scoring settings.
A team evaluating grammar-analysis software combines crossing-bracket counts with labeled constituent accuracy to avoid rewarding an incomplete flat analysis.

Think of it this way

Think of two proposed ways to draw boxes around items arranged in a row. One box encloses items one and two; the other encloses items two and three. The boxes overlap but neither fits inside the other.

This captures the boundary conflict. It does not mean either drawing contains physically crossed lines in its own tree, or that every difference between the drawings has been detected. Missing boxes and wrong labels are separate problems.

Worked example

Checking crossing and noncrossing spans

Take three tokens, A, B and C, with boundaries numbered zero through three. The reference groups A B, giving span [0,2). The prediction groups B C, giving span [1,3). They cross because 0 < 1 < 2 < 3.

Under the stated convention counting predicted constituents with at least one crossing, that predicted B C constituent contributes one. The full-sentence span [0,3) contributes none because it contains both smaller spans. A predicted [0,1) span also does not cross [0,2); it is nested inside it.

Now imagine a prediction with only the full-sentence grouping and individual terminals, under a policy excluding terminals from constituent scoring. It can have zero crossing constituents while missing the reference's A B phrase. That is why a low crossing count must be interpreted together with recall and other measures. This is a hand-checked illustration, not an executed evaluator report.

Illustrative case study

A fictional writing-tool team celebrates a drop in crossing-bracket errors after changing its parser. A closer review shows that the new output contains fewer intermediate phrases, so there are fewer opportunities to cross the reference boundaries.

The team checks labeled precision and recall alongside crossing counts and inspects representative trees. It discovers that an apparently cleaner structural score can coexist with missing useful phrase information. It then documents the evaluator settings used for all comparisons.

The lesson is that a metric can improve because the output becomes less informative. Crossing brackets measure one kind of conflict; they do not independently establish that a parser supplies the correct or complete analysis.

Common misconceptions and mistakes

Any overlapping spans count as crossing. Nested spans overlap but do not cross under the standard boundary test.
Zero crossings means a perfect parse. Missing constituents and label errors can remain.
The count always means the number of crossing pairs. Some implementations count affected predicted constituents instead.
A score can be compared without tokenization details. Misaligned tokens can invalidate boundary comparisons.

Business use

Cross-bracket measures are most relevant when choosing or diagnosing a constituency parser, including parsers used inside document analysis or educational software. They are less directly meaningful as a user-facing quality promise than a clear account of the errors affecting the application.

Require reports to state the reference data, tokenization, excluded categories and counting convention. Compare parser versions on the same setup, and inspect whether score changes reflect better structure or simply fewer predicted phrases. If the business task depends on extracting particular noun or verb phrases, evaluate those outputs directly in addition to a general parsing metric.

Check your understanding

Do spans [0,4) and [1,3) cross?

Answer: No. The second is fully contained in the first. A crossing requires partial overlap with interleaving boundaries, such as [0,3) and [2,4).

Related terms

Learn next

Draw spans above a short token sequence and distinguish disjoint, nested and crossing pairs. Then compare a sparse prediction with a reference to see why zero crossings can coexist with poor recall.

Sources

Jurafsky and Martin, Speech and Language Processing, Chapter 19, Evaluating Parsers: https://web.stanford.edu/~jurafsky/slp3/19.pdf. PARSEVAL and crossing brackets.

University of Washington LING 571 parser-evaluation slides: https://teaching.shane.st/571/aut24/slides/5_PCKY-Eval.pdf. EVALB counting discussion. Boundary examples are original and use explicit half-open indexing; no software benchmark was run.

Previous: Cross-attention Next: Cross-correlation matrix