Must the documents be translated in advance?

No. Query translation and shared multilingual representations provide other approaches, each with its own costs and limitations.

Does a multilingual retrieval benchmark necessarily test CLIR?

No. Some benchmarks test same-language query - document retrieval in several languages. The task definition must be checked rather than inferred from the benchmark name.

AI Glossary / Natural language and speech / Intermediate

Cross-language information retrieval

Definition

Definition: Cross-language information retrieval, or CLIR, searches for relevant information written in a language different from the query language. It helps a person express an information need in one language and find useful documents or passages in another.

What it means

What it means: A search engine that supports many languages does not necessarily perform cross-language retrieval. It might simply match English queries to English documents and Spanish queries to Spanish documents. CLIR requires the relevance match to cross that language boundary.
The goal is retrieval, not necessarily producing a complete translation. A system can find a relevant foreign-language passage before deciding how to help the user read it. Conversely, an accurate translation of an irrelevant document does not make the search successful. Evaluation must consider whether the returned material answers the information need, rather than whether the words look like plausible translations.

How it works

How it works: One approach translates the query into the document language and then searches. Another translates documents into the query language. Systems can also map queries and documents into shared representations that support relevance comparison without displaying an intermediate translation.
After candidate retrieval, a ranking stage orders the results. A mixed-language collection introduces additional questions, including whether equivalent evidence in different languages should be grouped. Short queries, ambiguous words, names and specialist terminology can make language transfer difficult. The returned interface may provide translated snippets, but those snippets need to remain connected to the original source so users can inspect the evidence.

Three practical examples

Three practical examples:
1. An engineering team searches a multilingual technical archive using its preferred working language. Relevant material may come from manuals written by another regional team.
2. A researcher looks for descriptions of an event in foreign-language news collections. The retrieval process finds candidates; source evaluation and interpretation remain separate tasks.
3. A support portal lets users ask questions in one language while searching approved help articles in another. It tests whether the retrieved passage actually addresses the issue before generating an answer from it.

Think of it this way

Think of it: CLIR resembles asking a multilingual librarian to search shelves whose books use different languages. The librarian must connect the question to relevant content, not simply replace each query word with its dictionary counterpart. The analogy is limited because automatic matching can miss context or rank fluent-looking but irrelevant material highly.

Worked example

Worked example: Suppose an English query asks about replacing a bicycle tire, and the candidate collection contains six Spanish documents. Bilingual reviewers judge D2 and D5 relevant to the complete request. A toy retrieval system returns D2, D4 and D5 as its top three results.
Precision at three is 2/3 because two of the three returned documents are relevant. Recall at three is 2/2 = 100% because both relevant documents in this deliberately complete toy judgment set were retrieved. If another system returns D2, D4 and D6, its precision at three is 1/3 and its recall at three is 1/2.
The example does not award relevance merely for mentioning bicycles or tires. D4 might discuss tire manufacturing without explaining replacement. Nor does 100% recall at three imply that a real web search found every relevant page: that calculation is valid here only because the toy collection and its relevance judgments are explicitly complete.

Illustrative case study

Illustrative case study: Fictional Alder Knowledge offers search across English and French equipment guides. A prototype trained on translated general text retrieves passages about a similarly named consumer product instead of the specialist component requested by technicians.
The team builds representative bilingual relevance judgments and includes model identifiers and domain terms in its tests. It also displays the source language and links translated snippets to original passages. This invented case illustrates domain-specific evaluation and result presentation. It does not claim that one translation or embedding method is universally best, or that a multilingual model performs equally well in every language direction.

Common misconceptions and mistakes

Common misconceptions and mistakes: Multilingual support is not proof of cross-language matching. Training on several languages does not by itself establish good retrieval across them. Translating every query word independently can choose the wrong sense or lose a multiword expression. Performance in one direction, such as English to French, need not predict performance in the reverse direction. Finally, duplicate translations of one document should not automatically be counted as independent evidence supporting a claim.

Business use

Business use: Specify the actual query languages, document languages and domains before choosing a retrieval system. Evaluate the intended language directions separately and include realistic short queries, names and technical terms. Measure whether users can identify and use the result, not just whether a ranker assigns a high score. Preserve source-language access and clarify which displayed text is translated. If retrieved passages feed an answer generator, inspect retrieval failures separately from translation and answer-generation errors.

Check your understanding

Check your understanding: A search service handles German and English but always searches documents in the same language as the query. Does that alone demonstrate cross-language retrieval?

Answer: No. It demonstrates multilingual coverage. CLIR requires a relevance match where query and retrieved content use different languages.

Related terms

multilingual information retrieval
query translation
document translation
cross-lingual embeddings
relevance ranking
bilingual lexicon
cross-language question answering
retrieval evaluation

Learn next

create a small bilingual document set and ask speakers of both languages to judge relevance to several queries. Compare exact-word matching, translated queries and a shared-representation method. Separate failures caused by meaning transfer from failures caused by irrelevant candidate documents.

Sources

Sources: Sun and Duh, CLIRMatrix: A Massively Large Collection of Bilingual and Multilingual Datasets for Cross-Lingual Information Retrieval, 2020: https://aclanthology.org/2020.emnlp-main.340.pdf . Galuščáková, Oard and Nair, Cross-language Information Retrieval: https://user.eng.umd.edu/~oard/pdf/clirsurvey22.pdf . MIRACL project task description: https://project-miracl.github.io/ . The six-document metrics and Alder Knowledge case are original illustrations, not benchmark results.

Previous: Cross / joint bilateral filter Next: Cross-sectional data