Can shape identify a person's name?

It can provide a clue, but the same pattern can describe ordinary words or organizations. Context and other evidence remain necessary.

Does a shortened shape preserve word length?

Generally not when repeated character classes are collapsed. Keep a separate length feature if that information matters.

AI Glossary / Natural language and speech / Intermediate

Word shape

Definition

Definition: Word shape is a representation of a token's character pattern, such as uppercase letters, lowercase letters, digits and punctuation. In natural language processing, it provides a feature describing how a word is written while abstracting away some or all of its exact letters.

What it means

What it means: Under one common convention, uppercase letters become X, lowercase letters become x and digits become d, while punctuation stays unchanged. The identifier QZ-42 therefore becomes XX-dd. This representation captures its broad format without retaining Q, Z, 4 or 2.
Word shape can help a model recognize patterns in unfamiliar tokens. A capitalized name, an abbreviation and a product code may have different shapes. However, shape does not establish meaning or category. A word capitalized at the start of a sentence is not necessarily a name, and many unrelated identifiers share the same format. The convention must also specify how it handles scripts, symbols and character categories beyond simple English examples.

How it works

How it works: A feature extractor processes each token according to a defined character mapping. It may keep the full mapped sequence or shorten it by collapsing consecutive repetitions of a character class. These two versions retain different information: the full sequence preserves class-run lengths, while the shortened version emphasizes transitions.
The resulting feature can be combined with the token itself, nearby words, prefixes, suffixes or learned representations in a tagger or entity recognizer. Its value comes from complementary evidence, not from replacing context. Implementers should document tokenization, Unicode character handling and whether punctuation is preserved or generalized, because these choices change the feature even when users perceive the same visible text.

Three practical examples

Three practical examples:
1. A named-entity recognizer uses initial capitals as one clue when identifying a previously unseen organization name. Surrounding words help distinguish the name from an ordinary sentence-initial noun.
2. A support-ticket classifier notices that a token mixes capitals, a hyphen and digits, making it a candidate equipment identifier. A later lookup determines whether the identifier is valid and which equipment it names.
3. A document-cleaning workflow compares expected reference-code formats with observed shapes. An unexpected lowercase letter or punctuation mark can trigger review, although an unusual shape alone does not prove the text is wrong.

Think of it this way

Think of it: Word shape resembles describing clothing by its pattern rather than identifying the person wearing it. Stripes or colors can be useful clues, but several people can share the same pattern. The analogy stops at that distinction: character features are formally specified transformations, and their predictive value must be tested for the actual language task.

Worked example

Worked example: Define an ASCII-only teaching rule: A - Z maps to X, a - z maps to x, 0-9 maps to d, and a hyphen remains a hyphen. For the token AbC-12, the full shape is XxX-dd. For QRS-98, it is XXX-dd. For Elm, it is Xxx, and for elm it is xxx.
Now collapse consecutive repeated class symbols. AbC-12 becomes XxX-d; QRS-98 becomes X-d; Elm becomes Xx; and elm becomes x. The full QRS-98 shape has six symbols, including the hyphen; the shortened shape has three. The shortened version has discarded the fact that there were three uppercase letters and two digits.
Finally, ZYX-71 has the same full and shortened shapes as QRS-98. That collision is intentional abstraction, not evidence that the two tokens identify the same object. A system needing the exact identifier must retain the original text alongside its shape feature.

Illustrative case study

Illustrative case study: Fictional Maple Repairs receives support requests containing new model numbers. Its existing classifier relies heavily on previously seen tokens, so the team adds character-pattern features as another source of evidence.
Reviewers then discover that order numbers and equipment model numbers often share the same shape. They include nearby labels such as “order” or “model” and use a product catalog for final identity checks. The team evaluates unfamiliar codes separately from familiar ones. This invented case illustrates why a format feature can be useful yet insufficient; it does not claim a measured accuracy gain or that every language model needs a separate hand-built shape feature.

Common misconceptions and mistakes

Common misconceptions and mistakes: Word shape does not mean the visual outline of printed letters in this NLP context. It is not stemming, which concerns word forms and roots, and it is not a semantic embedding. Different software may use different shape conventions. Collapsing repeated classes loses length information, while retaining only shape loses lexical identity. Capitalization rules also vary across languages and writing styles. Applying an English-oriented feature without checking its behavior on other scripts can make the feature uninformative or misleading.

Business use

Business use: Word-shape features can support tagging, identifier detection and quality checks when format carries useful information. Specify the mapping and retain original tokens for audit and lookup. Evaluate separately on sentence beginnings, all-capital messages, unfamiliar names, mixed scripts and malformed input. Compare performance with and without the feature rather than assuming it helps. If a workflow needs exact validation, follow shape detection with the relevant catalog, checksum or application-specific rule; a matching pattern alone is only a candidate signal.

Check your understanding

Check your understanding: Under the stated mapping, two product codes both become XX-dd. Must they refer to the same product?

Answer: No. Their formatting matches, but the original letters and digits may differ. Shape intentionally discards information needed for exact identity.

Related terms

Feature extraction
Named entity recognition
Part-of-speech tagging
Tokenization
character class
orthography
prefix feature
unknown word

Learn next

map a small collection of names, ordinary words and identifiers into full and shortened shapes. Find collisions and identify which distinctions were lost. Then decide whether context, exact spelling or a separate length feature would resolve the ambiguity relevant to your application.

Sources

Sources: Jurafsky and Martin, Speech and Language Processing, Chapter 18, word-shape and shortened word-shape features for sequence labelling: https://web.stanford.edu/~jurafsky/slp3/18.pdf . The ASCII mapping exercise and Maple Repairs case are original illustrations. The explicitly limited teaching convention is not presented as a universal Unicode implementation.

Previous: Word-sense induction Next: Word tokenization