AI Glossary / Retrieval and knowledge systems / Beginner
Embedding vector
Definition
An embedding vector is an ordered list of numerical coordinates representing an item for a machine-learning task. It may represent a token, sentence, image, product or another input. The vector lets a model process or compare that item using relationships encoded in a compatible embedding space.
What it means
The vector is one item's representation, rather than the entire model or the full collection of representations. Its number of coordinates is its dimension. In many systems the values are learned or produced by a trained encoder, so they reflect patterns useful to that system's objective.
The coordinates usually do not have simple names that a person can read directly. A value in the third position might contribute to many relationships rather than stand for one clearly defined concept. Understanding the representation therefore requires examining how it is produced and how it behaves on the intended task.
How it works
A system obtains the vector through a lookup or an encoder calculation. A token lookup selects a row from an embedding matrix. A sentence or image encoder processes its input and outputs a representation, potentially combining many intermediate features.
The application can then compare vectors, feed them to another model or store them for later retrieval. Coordinate order, dimension and model compatibility must remain consistent. Normalization may rescale the vector to unit length when that matches the intended comparison rule.
A contextual token vector can vary with surrounding text, even for repeated occurrences of the same word. In contrast, a basic embedding-table lookup returns the same stored row for the same identifier until its parameters change.
Three practical examples
1. A search system stores one embedding vector per document passage and compares those vectors with a query representation.
2. A recommendation model uses a product vector as an input to a scoring calculation that also includes a user's representation.
3. An image classifier uses an intermediate feature vector to support category prediction, while a separate evaluation checks whether that representation is suitable for image similarity search.
Think of it this way
Think of a compact numerical profile created for a particular job. The profile allows consistent calculations, but its entries are not a readable biography of the item. Two profiles can resemble each other because of the patterns the model learned, while differing in important details that the profile does not preserve well.
Worked example
Suppose an invented item has embedding vector v = (3, 4). Its Euclidean length is square root of (3 squared plus 4 squared), or square root of 25 = 5. Dividing each coordinate by five produces the unit vector (0.6, 0.8).
A second invented item has unit vector u = (0, 1). The cosine similarity between v and u is their dot product, 4, divided by the product of their lengths, 5 times 1. The result is 0.8. Using the normalized version of v gives the same cosine value through a dot product: 0.6 times 0 plus 0.8 times 1 = 0.8.
The raw dot product is four, while the normalized dot product is 0.8. They should not be confused even though both calculations use the same underlying direction.
These coordinates are teaching values, not output from a named model. A cosine similarity of 0.8 describes this geometric relationship; it does not mean an 80% probability of relevance or factual agreement. A zero vector would require special handling because its cosine denominator would be zero.
Illustrative case study
Fictional case: Acacia Search finds that a small fraction of indexed records have vectors with the wrong dimension after a data import. Engineers trace the issue to a process that mixed outputs from two model configurations.
They validate dimensions, reject malformed records and regenerate affected vectors using the correct configuration. Reviewers then inspect nearest neighbors for a sample of repaired records. The team keeps model identifiers and source references beside each vector so that successful numerical validation can be followed by a check of actual retrieval usefulness.
Common misconceptions and mistakes
An embedding vector is not normally a list of probabilities, and its coordinates need not be positive or add to one. It is also not automatically anonymous or safe to distribute merely because the original text is not readable in the vector.
A longer vector does not guarantee a better representation. Changing coordinate order destroys compatibility unless the same transformation is applied appropriately throughout the system. Rounding or quantizing values can change comparisons, so storage optimizations should be evaluated rather than assumed harmless. Similar vectors can still correspond to contradictory statements.
Business use
Teams encounter embedding vectors when building search indexes, recommendations, clustering tools and classification pipelines. Document the vector dimension, generating model, preprocessing and any normalization or quantization. Keep source identifiers and useful metadata attached to stored representations.
Check for malformed values, unexpected dimensions and incompatible model versions before indexing. Then evaluate the actual task: numerical validity alone does not establish meaningful neighbors. When assessing storage, include index structures and metadata as well as raw coordinates, and measure the effect of compression on the results users receive.
People also ask
Is a vector the same as an embedding matrix?
No. A vector represents one item, while a matrix can store many such vectors as rows.
Can the same word have different vectors?
Yes. Contextual models can produce different representations for different occurrences, and different models can also represent the word differently.
Can a vector be read as plain-language meaning?
Usually not directly. Its usefulness comes from learned relationships and downstream computations, not a simple label attached to every coordinate.
Check your understanding
A vector has coordinates (6, 8). What is its unit-length version?
Answer: Its length is square root of (36 + 64) = 10, so the unit vector is (0.6, 0.8). The direction is preserved while the length becomes one.
Related terms
- Embedding
- Embedding matrix
- Embedding space
- Normalization
- Cosine similarity
- vector norm
- dot product and vector quantisation
Learn next
calculate vector lengths and similarities for a few small examples. Compare raw and normalized dot products, then test actual model outputs on examples with subtle factual differences. Keep geometric similarity separate from the judgment of whether a result is useful.
More in Retrieval and knowledge systems
- Document clustering - Document clustering is the process of grouping documents according to patterns of similarity in their representations.
- Document partitioning - Document partitioning divides a search collection into subsets of documents and maintains a local index for each subset.
- Double moving average - A double moving average is a moving average applied to a series that has already been smoothed by a moving average.
- Extended Boolean model - The extended Boolean model is an information-retrieval approach that combines structured Boolean-style queries with weighted matching and ranked results.
- Faceted search - Faceted search is a search approach that lets users narrow results through several organized attributes, called facets.
- Federated averaging - Federated averaging, often shortened to FedAvg, is a federated-learning algorithm that combines locally trained model parameters into a shared model.
Sources
Primary sources: Google, Machine Learning Glossary, embedding vector: https://developers.google.com/machine-learning/glossary#embedding-vector
Google, Measuring similarity from embeddings: https://developers.google.com/machine-learning/clustering/dnn-clustering/supervised-similarity
TensorFlow, Word embeddings, including lookup vectors and learned values: https://www.tensorflow.org/text/guide/word_embeddings
Previous: Embedding space Next: Embodied agent