Must the source be XML?

The principle comes from structured retrieval, but the idea can apply whenever meaningful document parts and their relationships are available.

Can the whole document be the right result?

Yes. A broad query may require it, or no smaller eligible part may answer the need adequately.

Does this principle determine a generated answer's wording?

No. It concerns what evidence or document part to retrieve, not how a generator should compose its response.

AI Glossary / Retrieval and knowledge systems / Intermediate

Structured document retrieval principle

Definition

The structured document retrieval principle says that a search system should return the most specific part of a document that answers the query. It aims to avoid both unnecessarily large results and fragments too small to contain the information the user needs.

What it means

A structured document has meaningful parts such as chapters, sections, paragraphs, tables and titles. Searching that document raises a choice: should the result be the whole document or one of those parts?

For a question about one procedure, a relevant section may be more useful than a complete manual. For a request about the manual itself, its title or a tiny matching paragraph may be inadequate. The appropriate unit depends on the information need, not simply on where a keyword appears.

Most specific does not mean shortest possible text. The selected part must actually answer the query and retain the context needed to interpret it. The principle is a retrieval design objective rather than a single algorithm that always identifies the right boundary.

How it works

A system represents the document hierarchy and chooses units to index or consider as results. It may score whole documents, selected elements or many levels of the structure. It then needs to select a useful result unit and control overlap.

Indexing every element can produce redundant hits: the same relevant paragraph may appear directly and again inside its section, chapter and document. Indexing only large units can hide a highly relevant small passage inside otherwise unrelated material.

Possible designs retrieve large units and refine them, start with small units and expand them, or score eligible elements directly. Each can fail if the relevance of one level is a poor guide to another. Showing a focused result with access to surrounding context can help, but presentation and retrieval-unit selection remain separate decisions.

Three practical examples

A maintenance search returns the complete reset procedure rather than an entire service manual or just the sentence containing reset.
An archive search returns a specific correspondence section when that section answers the query.
A policy search returns the eligibility rule together with its qualifying exception when both are required to understand the answer.

Think of it this way

Think of asking a librarian for the passage that answers a question. Receiving the whole shelf is excessive, but receiving one sentence cut off before its qualification is insufficient.

The analogy does not assume there is always one obvious passage. Some questions require several sections or documents, and a retrieval system must represent that need rather than force every answer into one tiny fragment.

Worked example

Choosing a unit that preserves the condition

An invented manual contains a chapter of 2,000 words. Within it is a 300-word section on filter replacement. A 40-word paragraph lists the steps, while the following 30-word paragraph states that one model requires power isolation before the first step.

The query asks, “How do I replace the filter on that model?” Returning only the 40-word steps paragraph omits a relevant condition. Returning the entire chapter includes much material unrelated to the request. If the section is the smallest eligible document element containing both the steps and the condition, the section is the appropriate unit under this principle.

For a different query asking only for the filter part number, a smaller specification element might suffice. These stipulated lengths make the granularity choice visible; they are not scoring weights or an evaluation of a real manual.

Illustrative case study

The fictional WillowPolicies team initially indexes every XML element in its handbook. Searches return a relevant paragraph, its parent section and the entire handbook as separate hits. Users see repeated content and struggle to tell which result to open.

The team defines eligible result units and groups overlapping hits. Reviewers check whether selected sections preserve exceptions and source context. For broad queries about the handbook, the system can still return the whole document rather than insisting on paragraph-level answers.

The case illustrates query-dependent granularity and redundancy control. It reports no measured user benefit or claim that one element size works for every query.

Common misconceptions and mistakes

The principle always prefers the smallest matching element. The element must answer the query, not merely contain its words.
More indexed elements automatically mean better results. Indexing nested elements can create redundant or incoherent hits.
One fixed chunk size implements the principle completely. Suitable boundaries depend on document structure, context and the information need.

Business use

This principle is useful for searching manuals, policies, reports and other long structured documents. It can also inform passage selection for retrieval-assisted answer systems, though those systems introduce additional generation and evidence requirements.

Define which units are meaningful to users and how nested results are displayed. Evaluate whether results include necessary qualifications, provenance and links to surrounding text. Avoid assuming that a whole-document score identifies its best passage, or that a highly matching sentence establishes the relevance of its entire parent section. A useful retrieval unit should support the task the user is trying to complete.

Check your understanding

A paragraph contains the query terms, but its next paragraph contains an exception needed to answer correctly. Should the first paragraph automatically be preferred because it is smaller?

Answer: No. The selected unit must include the information needed to answer the query, including the relevant exception.

Related terms

Structured retrieval
XML element
passage retrieval
document granularity
nested results
indexing unit
context preservation
relevance assessment

Learn next

Read structured retrieval and indexing units. For three questions about one document, identify the smallest meaningful part that answers each question and explain why a smaller fragment would be insufficient.

Sources

Christopher Manning, Prabhakar Raghavan and Hinrich Schutze, Introduction to Information Retrieval, Challenges in XML retrieval: https://nlp.stanford.edu/IR-book/html/htmledition/challenges-in-xml-retrieval-1.html

The authors' retrieval principle and discussion of indexing units, nesting and context were read. The manual example and policy-search case are original. The maintenance scenario explains retrieval completeness only; it does not provide instructions for servicing actual equipment.

Previous: Structured data Next: Structured entity relationship model