Does it use training labels?

Standard random projection does not use labels to choose its matrix.

Can it work with sparse data?

Suitable sparse projection constructions can help, but check the sparsity and memory use of the resulting representation.

Is the random seed enough to reproduce it forever?

Saving the actual transformation and its preprocessing specification is stronger, because implementation and random-generation details can change.

AI Glossary / Mathematics and optimization / Intermediate

Random projection

Definition

Random projection is a dimensionality-reduction method that maps a feature vector into fewer coordinates using a suitably generated random matrix. With appropriate scaling and enough output dimensions, it can approximately preserve pairwise Euclidean distances in a finite dataset with high probability.

What it means

A record described by thousands of numerical features can be costly to store and compare. Random projection replaces those features with fewer weighted combinations. The weights are sampled according to a chosen construction rather than learned to maximize variance or predict labels.

The result is a new representation. Its coordinates usually mix many original features, so they may be harder to interpret individually. Reducing the number of coordinates also loses information in general; the useful promise concerns specified geometric relationships, not exact recovery of every original record.

In this context, projection need not mean an orthogonal projection represented by a square idempotent matrix. The mapping commonly uses a rectangular matrix from a larger feature space into a smaller one.

How it works

For a column vector x with d coordinates, choose a matrix A with k rows and d columns, where k<d, and compute z=Ax. Use the same A for all records that will be compared, including later inputs.

One standard construction samples independent Gaussian entries with mean zero and variance 1/k. Sparse constructions use many zeros with an appropriate distribution and scaling. The scaling controls how output distances relate to input distances.

For suitable constructions, concentration results support a Johnson - Lindenstrauss-style guarantee on a fixed finite set: squared distances remain within factors 1−ε and 1+ε with a specified high probability when k is sufficiently large. Typical dimension bounds grow logarithmically with the number of points and inversely with the square of the distortion tolerance, with additional constants and probability details.

A guarantee for a fixed finite dataset does not preserve every direction in the entire original space. Every dimension-reducing linear map has a nontrivial null space. Practical teams therefore measure distortion and downstream performance on relevant data, rather than treating a random draw as automatically satisfactory.

Three practical examples

A document-search prototype reduces large numerical feature vectors before benchmarking approximate similarity calculations.
An analyst compares clustering behavior before and after a random projection to see whether important neighborhood relationships remain useful.
A research team tests sparse projection matrices when the dense transformation itself would consume too much memory or computation.

Think of it this way

Think of describing an object through a smaller set of mixed measurements. The measurements may preserve useful differences between the objects you care about without recording every original detail.

The analogy does not imply that any measurements work. Their construction, scaling and number matter, and two different objects can become indistinguishable if their difference falls in a discarded direction.

Worked example

A small map with both success and failure

Consider a possible signed projection matrix A=(1/√2) times the matrix with rows (1,1,1) and (1,−1,1). This illustrates a three-to-two coordinate mapping; it is not a claim that two dimensions satisfy a particular large-dataset guarantee.

For x=(1,0,0), the output is (1/√2,1/√2). For y=(0,1,0), it is (1/√2,−1/√2). The original squared distance is 2, and the projected squared distance is also 2.

Now take z=(0,0,1). Its output is again (1/√2,1/√2), exactly the same as x's output. The original squared distance between x and z is 2, while the projected squared distance is zero.

The difference x−z=(1,0,−1) lies in A's null space. Preserving one pair exactly did not establish preservation of every pair. A realistic assessment must examine the relevant set, the selected output dimension and the probability assumptions of the construction.

Illustrative case study

A fictional customer-support research team experiments with shorter text-feature vectors. Its first demonstration shows several similar tickets remaining close after projection, and the team considers the method promising.

Before integrating it, the analysts check distances across a broader sample, including rare ticket types. They also evaluate whether useful search results remain near the top of the ranking. A coordinate transformation can preserve many distances while still changing a closely contested ranking.

The team records the generated matrix and preprocessing settings with the experiment. Regenerating a different matrix for incoming tickets would place those tickets in incompatible coordinates, even if the output length stayed the same.

Common misconceptions and mistakes

It selects the best original features. Usually it creates mixtures rather than selecting a supervised subset.
Every random matrix preserves distances. Distribution, scaling, dimension and probability conditions matter.
Fewer coordinates guarantee faster end-to-end processing. Transformation costs and dense outputs can offset savings.
It is equivalent to principal component analysis. PCA learns directions from data; ordinary random projection samples them.
The original data can always be recovered. A lower-dimensional linear map generally loses information.

Business use

Random projection can be a useful baseline when a team needs a compact numerical representation without fitting a complex feature-learning model. Evaluate the complete workflow: projection time, storage, search or model performance, and reproducibility.

Preserving Euclidean distances is useful only if those distances are meaningful for the task. Feature scaling and encoding choices should therefore be reviewed first. A compact representation also does not, by itself, provide a formal privacy guarantee.

Check your understanding

A projection preserves the distance between two test records exactly. Does this prove that it preserves all possible input distances?

Answer: No. The worked example preserves one pair but collapses another. A dimension-reducing linear map necessarily discards some directions.

Related terms

Learn next

Read null space and dimensionality reduction. Apply the example matrix to (1,0,−1), then design a comparison that measures relative squared-distance error across many relevant record pairs.

Dimensionality reduction

Sources

Scikit-learn, Random Projection: https://scikit-learn.org/stable/modules/random_projection.html. Official documentation for Gaussian and sparse constructions and dimensionality considerations.

Aalto Dictionary of Machine Learning: https://aaltodictionaryofml.github.io/ADictML_Main.pdf. Finite-set geometric scope. The matrix, null-space demonstration and fictional scenario here are original illustrations, not an empirical benchmark.

Previous: Random (probability) sample Next: Random-restart hill climbing