Can a kernel value be negative?

Yes. A linear kernel between oppositely directed vectors can be negative while remaining valid.

Must the feature map be written out?

No. The kernel can provide the required inner products implicitly.

Is the kernel function the kernel trick?

No. The function defines the comparison; the trick uses those comparisons to replace explicit feature-space inner products in a compatible algorithm.

AI Glossary / Graphs and symbolic AI / Intermediate

Kernel function

Definition

In kernel-based machine learning, a kernel function computes an inner product between feature representations of two inputs. It allows an algorithm to compare those representations without necessarily constructing them explicitly. This meaning is distinct from an operating-system kernel, a convolution filter or a density-estimation kernel.

What it means

Write the function as K(x, z). For a valid real-valued positive-semidefinite kernel, there is a feature representation φ such that K(x, z) equals the inner product of φ(x) and φ(z). That feature space may be much larger than the original input space, or even infinite-dimensional.

People often call the result a similarity, but that wording needs care. A kernel value is not automatically a probability, a percentage or a distance. Its scale and interpretation depend on the chosen function and input representation.

The kernel specifies a comparison structure. A learning method still needs data, an objective, fitted parameters and an evaluation procedure. Choosing a kernel alone does not train a classifier or establish that its comparisons are useful for a particular task.

How it works

Choose a kernel suited to the represented inputs. Common examples include a linear inner product, a polynomial of that inner product, and a Gaussian radial-basis function based on squared distance.

For training inputs, evaluate the function on pairs to obtain a Gram matrix. A valid real kernel is symmetric and produces a positive-semidefinite Gram matrix for every finite choice of inputs. This means every quadratic form c-transpose K c is nonnegative; it does not mean every individual entry must be positive.

Kernel-based algorithms use these pairwise values when fitting or predicting. Hyperparameters alter the comparison: for a Gaussian kernel exp(−γ times squared distance), increasing positive γ makes similarity decay more quickly with separation. Preprocessing and parameter selection therefore belong to the model design, not to an unrelated cosmetic step.

Three practical examples

A classification experiment compares a linear kernel with a nonlinear kernel on the same held-out evaluation task.
A regression model uses kernel comparisons to represent relationships that are not linear in the original measured variables.
A structured-data study designs a valid kernel for inputs such as strings, while checking that the comparison reflects relevant structure.

Think of it this way

Think of comparing products through a hidden checklist of features. The kernel returns the checklist's inner-product score without necessarily printing every item on the checklist.

The score is only as useful as the checklist's meaning. A large value does not automatically mean two products are interchangeable, and the analogy does not turn a mathematical comparison into a calibrated probability.

Worked example

A small kernel matrix that can be checked by hand

Use one-dimensional inputs −1, 0 and 1 and the kernel K(x, z) = (1 + xz) squared. One explicit feature representation is φ(x) = (1, square root of 2 times x, x squared).

For x = −1, this gives (1, −√2, 1). For x = 0, it gives (1, 0, 0). For x = 1, it gives (1, √2, 1). Taking inner products yields the same values as the compact kernel formula.

For example, K(−1, 1) = (1 − 1) squared = 0. The feature-space calculation gives 1 − 2 + 1 = 0. Meanwhile K(1, 1) = 4, and K(−1, 0) = 1.

In the input order −1, 0, 1, the Gram matrix has rows (4, 1, 0), (1, 1, 1) and (0, 1, 4). It is symmetric. More strongly, for any real coefficients c1, c2 and c3, its quadratic form equals the squared length of c1φ(−1) + c2φ(0) + c3φ(1). A squared length cannot be negative, establishing positive semidefiniteness for this matrix.

The feature construction proves the same property for any set of scalar inputs. Notice that the self-comparison value 4 exceeds 1: this kernel is not a probability scale. These calculations verify the comparison function, not the accuracy of a trained prediction model.

Illustrative case study

The fictional Alder Models team builds a demonstration with the three scalar inputs. A draft interface labels kernel values as percentages, so the self-comparison value 4 becomes a nonsensical 400 percent match.

A reviewer replaces that label with kernel score and adds the formula and preprocessing assumptions. The team uses the explicit three-feature representation to check the implementation, including zero-valued comparisons and the matrix symmetry.

For a later prediction exercise, Alder evaluates the fitted model separately. The fictional case shows why a mathematically valid comparison and a useful predictive system are distinct achievements, both requiring their own evidence.

Common misconceptions and mistakes

Every similarity formula is a valid kernel. Symmetry alone does not establish positive semidefiniteness.
Positive semidefinite means all entries are positive. It concerns quadratic forms or eigenvalues; valid kernel values can be zero or negative.
Kernel scores are probabilities. Their scale generally has no such interpretation.
A valid kernel guarantees good predictions. Validity supports the mathematics, while predictive usefulness depends on the task and data.

Business use

Kernel functions can make a useful middle ground between simple linear comparisons and explicitly constructing a large feature expansion. They are particularly valuable when a meaningful comparison can be computed directly.

A project should document the function, parameters, feature units and preprocessing. Compare performance against simpler baselines using an appropriate evaluation split. Also estimate the cost of pairwise computations: a fully stored Gram matrix for n examples has n squared entries. Mathematical elegance does not remove sample-size costs, weak data or an unsuitable definition of similarity.

Check your understanding

Why does the example's K(1, 1) = 4 not violate kernel validity?

Answer: A kernel is an inner-product function, not necessarily a normalized probability. The mapped vector (1, √2, 1) has squared length 4.

Related terms

Learn next

Compute the same kernel on inputs 0, 1 and 2 using both its formula and explicit feature map. Then compare it with a linear kernel. Describe how the numerical comparisons change without calling either set of values probabilities.

Feature map

Sources

Cornell's kernel notes explain validity and feature-space comparisons: https://www.cs.cornell.edu/courses/cs4780/2018fa/lectures/lecturenote14.html

Scikit-learn documents pairwise kernels and their distinctions: https://scikit-learn.org/stable/modules/metrics.html

Its polynomial and radial-basis definitions provide concrete parameterizations: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.polynomial_kernel.html ; https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.rbf_kernel.html

The three-input calculations and Alder Models case are original examples.

Previous: Kernel embedding of distributions Next: Kernel (kernel method)