Can a space have more than one basis?

Yes. A finite-dimensional space generally has many choices, but every basis has the same number of vectors.

Can the zero vector belong to a basis?

No. Its presence makes the set linearly dependent.

Does using fewer basis directions always preserve the original vector?

Only if the vector lies in the span of the retained directions. Otherwise, a lower-dimensional representation is an approximation or loses information.

AI Glossary / Mathematics and optimization / Intermediate

Basis

Definition

A basis is a set of linearly independent vectors whose linear combinations produce every vector in a specified vector space. In a finite-dimensional space, it provides exactly enough independent building blocks to represent any vector uniquely. The number of vectors in any basis of that space is its dimension.

What it means

Two requirements work together. Spanning means the building blocks can reach every vector in the space. Linear independence means none of the blocks can be formed from the others, so the representation has no redundancy.

A basis depends on the space being described. Two independent vectors may form a basis for a plane inside three-dimensional space without forming a basis for the whole three-dimensional space. This entry concerns the linear-algebra meaning used throughout machine learning, rather than the everyday meaning of a justification or foundation.

How it works

Choose candidate vectors and check both independence and spanning. If their columns form a matrix B, representing a vector v means solving Bc=v for its coordinate vector c.

When the columns form a basis for the target space, every vector in that space has exactly one solution c. If the target space is all of R^n and there are n basis vectors, B is square and invertible. For a lower-dimensional subspace, B can be rectangular while still providing unique coordinates for vectors in its span.

Changing basis changes the coordinates, not the underlying vector. Orthonormal bases add useful geometric properties, but a basis need not be perpendicular or normalized. Numerical conditioning matters when basis vectors are nearly dependent.

Three practical examples

1. Data representation: express observations in a new coordinate system, such as a complete set of principal-component directions, while distinguishing exact coordinate changes from dimensionality reduction.

2. Signal modeling: represent a signal using coefficients of a chosen finite basis, or approximate it using a limited subset of components.

3. Linear-system analysis: choose a basis for a column space or nullspace to describe all possible outputs or homogeneous solutions compactly.

Think of it this way

Think of directions on a map. East and north let you describe every displacement in a flat plane uniquely. Two other nonparallel directions can also work, but the numerical instructions will change.

The analogy assumes unrestricted positive and negative amounts, as linear combinations allow. A basis is not a recipe restricted to positive quantities. Nor does changing directions move the destination; it changes how that destination is described.

Worked example

In R^2, choose b1=(1,1) and b2=(1,-1). To represent v=(5,1), solve c1b1+c2b2=v. The coordinate equations are c1+c2=5 and c1-c2=1, giving c1=3 and c2=2.

Thus v has standard coordinates (5,1) but coordinates (3,2) relative to this ordered basis. Substitution checks the result: 3(1,1)+2(1,-1)=(5,1).

The vectors are independent because c1b1+c2b2=(0,0) implies c1=c2=0. They span the plane because for any (x,y), the required coefficients are (x+y)/2 and (x-y)/2.

Now replace b2 with (2,2). Every combination lies on the line y=x, so (5,1) cannot be represented. The set also has redundancy because (2,2)=2b1. It fails both requirements for a basis of R^2. These are exact symbolic calculations, not numerical software output.

Illustrative case study

Fictional case: an analytics team stores two correlated measurements for each item. It introduces a rotated coordinate system to separate common movement from contrast between the measurements.

Using both new basis directions preserves the original information, provided the transformation is invertible and recorded. The team can recover the original measurements from the new coordinates.

Later, it considers dropping one coordinate for compression. Reviewers recognize that this is a projection onto a smaller subspace, not merely another full change of basis. They measure reconstruction error before deciding whether the lost information is acceptable. The scenario illustrates the distinction without claiming a real compression result.

Common misconceptions and mistakes

A spanning set is not necessarily a basis: it may contain redundant vectors. An independent set is not necessarily a basis for the desired space: it may be too small to span it.

Basis vectors do not need unit length or right angles. Those properties define special convenient bases, not the general concept.

Coordinates are not intrinsic to a vector without a stated basis and ordering. Reordering the basis reorders the coordinates needed to reconstruct the same vector.

Near dependence is different from exact dependence. A matrix may be invertible in exact arithmetic yet lead to unstable coordinate calculations when its columns are almost parallel.

Business use

Basis concepts help practitioners understand feature transformations, compression, regression, and numerical stability. They clarify when a representation preserves information and when it deliberately restricts the signals or patterns that can be represented.

Record the transformation and its ordering alongside transformed data. When a basis is learned from data, fit it within the appropriate training process to avoid evaluation leakage. Check conditioning and reconstruction behavior rather than assuming that any mathematically valid basis is equally convenient for computation or interpretation.

Check your understanding

The vector (5,1) has coordinates (3,2) in the worked basis. Has the vector itself changed?

Answer: No. Only its description has changed. Combining the stated basis vectors with coefficients 3 and 2 reconstructs the original vector exactly.

Related terms

Span
Rank
Nullspace
Condition number
Vector space
linear independence
dimension
coordinate vector
change of basis
orthonormal basis

Learn next

Solve coordinate equations in two different bases for the same vector and verify reconstruction. Then examine a plane embedded in R^3 to distinguish a subspace basis from a basis for the whole space. Study orthonormal bases and conditioning next to understand their computational advantages.

Sources

Primary glossary definition read: The Aalto Dictionary of Machine Learning, basis: https://aaltodictionaryofml.github.io/ADictML_Main.pdf . Primary lecture transcript consulted: Gilbert Strang, MIT Linear Algebra, Independence, Basis, and Dimension: https://ocw.mit.edu/courses/18-06-linear-algebra-spring-2010/0fab20a14c050d85ef29563cd6c52d37_MIT18_06S10_L09.pdf . The two-dimensional coordinate example is original and hand-checked.

Previous: Basic emotions Next: Basis function regression