What is a tensor mode?

It is an axis of the multidimensional array.

Does MPCA need class labels?

Standard MPCA learns from variation without class labels.

Is the output always a single vector?

The usual tensor-to-tensor formulation produces a smaller tensor, which can later be vectorized if required.

AI Glossary / AI and machine learning / Advanced

Multilinear principal component analysis

Definition

Multilinear principal component analysis is a dimensionality-reduction method that learns compact representations of tensor-shaped data through projections along multiple axes. It seeks to retain substantial variation without first treating every observation as one long, unstructured vector.

What it means

Images, videos and multichannel measurements have a natural arrangement. An image has rows and columns; a video adds time. Flattening such an observation into a vector is possible, but it discards the explicit organization of those axes from the representation.

Multilinear principal component analysis, or MPCA, learns projection matrices for the different tensor modes. Here, mode means an axis of the array, not a peak in a probability distribution. The resulting representation is typically a smaller tensor.

MPCA is an unsupervised feature-extraction approach. Retaining variation is its central aim, so the most variable features are not necessarily the ones most useful for a later prediction task.

How it works

Training observations are represented as tensors with consistent dimensions and centered using an appropriate mean. The method chooses reduced sizes and estimates projections along each mode to retain variation in the projected data.

Because the projections interact, estimation commonly alternates between them, updating one while holding the others fixed. New observations are transformed using the learned projections.

The compact tensor can feed a classifier, clustering method or reconstruction process. The learned projection structure is more constrained than an arbitrary projection of a flattened vector, which can reduce parameters but also limit the relationships represented. Validation determines whether that trade-off is useful for the task.

Three practical examples

A computer-vision experiment reduces image arrays while maintaining separate row and column projections.
A video-analysis project compresses spatial and temporal dimensions before applying a classifier.
A sensor study extracts compact features from consistently arranged channel-by-time measurements and compares them with ordinary PCA features.

Think of it this way

Think of summarizing a large table by learning useful combinations of rows and useful combinations of columns, rather than first pouring every cell into a single list.

The analogy captures the structured projection. It does not mean the method merely resizes an image or averages adjacent cells. The combinations are learned from variation across training observations, and the resulting features need not resemble recognizable rows or columns.

Worked example

Following the dimensions through a projection

Suppose each observation is a 20-by-30 numerical image, containing 600 values. A demonstration MPCA model reduces the first mode to 5 and the second to 6, producing a 5-by-6 representation containing 30 values.

For a matrix-shaped observation X, this can be expressed as U-transpose X V, where U has shape 20-by-5 and V has shape 30-by-6. The output has the expected shape 5-by-6.

The two projection matrices contain 100 + 180 = 280 entries. A completely unrestricted linear map from 600 inputs to 30 outputs would contain 18,000 entries before considering constraints or additional terms.

This arithmetic illustrates structural economy, not an accuracy guarantee or a full accounting of model storage. The smaller representation may lose important information, and the projection must be learned from training data without using the held-out test set.

Illustrative case study

Fictional case: compact inspection features

Maple Components, an invented workshop, compares MPCA features with flattened PCA features for an image-classification research exercise. All images are aligned and resized consistently before training.

The MPCA representation is compact, but a validation review finds that one small defect is poorly represented because it contributes little overall variation. The team evaluates different reduced dimensions and a task-specific alternative rather than selecting a method solely for its compression ratio.

The case illustrates the difference between preserving variance and preserving decision-relevant details. It does not report a real inspection system or certify any industrial application.

Common misconceptions and mistakes

MPCA is not simply ordinary PCA applied independently to arbitrary pieces of data. Its mode projections interact through the tensor representation. It is also different from uncorrelated MPCA variants, which impose additional feature-correlation conditions.

Preserving array structure does not guarantee better accuracy than vector methods. An iterative optimization procedure should not be described as automatically finding the globally best solution in every setting.

Business use

Businesses may consider MPCA when observations naturally form arrays and a compact representation could reduce downstream modeling costs. Its suitability depends on sample size, alignment quality and the importance of relationships across modes.

Compare it with straightforward baselines using identical data splits. Measure downstream accuracy, computation and sensitivity to reduced dimensions. Fit all preprocessing and projections using training data only. Inspect whether unusual but important patterns survive the reduction. A compact feature representation is useful only when the resulting system still performs adequately on the cases that matter operationally.

Check your understanding

MPCA retains most overall variation but removes a subtle signal needed for classification. Is high retained variance sufficient justification to keep it?

Answer: No. The downstream task must be evaluated directly. Important predictive information can occupy a low-variance direction.

Related terms

Learn next

Learn next how a tensor-mode product changes one array dimension. Then compare the parameter shapes of a structured multilinear projection and an unrestricted vector projection, while keeping their final feature counts the same.

Sources

Primary sources: Lu, Plataniotis and Venetsanopoulos, MPCA, https://www.comm.toronto.edu/~kostas/Publications2008/pub/102.pdf ; related analysis, https://arxiv.org/abs/1104.5281 . Primary MPCA sections II-B and III on mode projection, centring, scatter maximisation and alternating updates were consulted; the related analysis record was checked. Dimension calculation and fictional case are original.

Previous: Multilevel Monte Carlo method Next: Multilinear subspace learning