AI Glossary / AI and machine learning / Intermediate

Support vector machine

Definition

A support vector machine, or SVM, is a learning method that can build a decision boundary between classes while balancing separation against errors. Related support vector methods also handle regression and novelty detection.

What it means

Imagine describing each customer enquiry using numerical features. A classifier needs a rule that separates one category from another. An SVM seeks a boundary with a useful margin: a buffer between the separating boundary and nearby training examples.

The examples that determine the fitted boundary are called support vectors. Their role is more specific than simply being typical examples of each category. They constrain the solution, often because they sit near a boundary or are difficult to separate.

A linear SVM uses a straight boundary in the feature space. Kernel methods can represent more complex separation without explicitly constructing every transformed feature. This flexibility creates opportunities but also introduces choices that require careful validation.

For business readers, an SVM is a model family to compare with alternatives. It is not automatically the best option because its name sounds mathematical or because it performs well on one demonstration dataset.

How it works

Training represents each example numerically and uses labeled cases to fit a decision function. Feature scaling is often important because large numerical ranges can dominate the geometry used by the model.

A soft-margin formulation permits some violations of the ideal separation. A setting commonly called C controls the trade-off between penalizing training violations and allowing a wider, more regularized boundary. Its useful value depends on the data and implementation.

Kernel choices introduce additional settings. These are selected using development data or cross-validation, with all preprocessing fitted within the training portion of each evaluation split.

The final model produces decision scores or predictions. A score is not automatically a calibrated probability. If probabilities are required for a business decision, the team should examine how they are obtained and how well they match observed frequencies.

Three practical examples

• Message classification: a linear SVM provides a baseline for routing text represented by word or phrase features.
• Product inspection: a team compares an SVM using engineered image measurements with other classifiers for a small labeled collection.
• Sensor analysis: support vector methods estimate a numerical response or identify unusual observations, using formulations appropriate to those tasks.

Think of it this way

Think of placing a divider between two collections of objects while leaving as much room around it as practical. Some awkward objects may remain on the wrong side if forcing a perfect separation would make the divider unhelpfully complicated.

The analogy is limited to geometry. Real models may operate in thousands of dimensions, and a kernel's boundary cannot always be pictured as a simple line on a desk. The business meaning comes from the features and evaluation.

Worked example

Understanding a decision score

Suppose a teaching example has a fitted linear decision rule f(x) = 2x1 - x2 - 1. The model predicts class A when the score is positive and class B when it is negative.

For a record with x1 = 2 and x2 = 1, the score is 4 - 1 - 1 = 2, so the model selects A. For another with x1 = 1 and x2 = 3, the score is 2 - 3 - 1 = -2, so it selects B.

These numbers illustrate prediction from an already fitted rule; they do not demonstrate how the optimization found its coefficients. A score of two also does not mean a 200% probability or a probability twice as large as a score of one.

The team still needs an independent evaluation and a policy for uncertain or consequential cases.

Illustrative case study

A fictional document-service company compares classifiers for routing scanned forms. Its first SVM experiment mixes page count, character counts and several small-valued quality indicators without consistent scaling.

The analysts build a preprocessing pipeline, evaluate settings within training folds and retain a separate test collection from later submissions. They also review errors on unfamiliar form layouts rather than reporting only an overall score.

The lesson is that model preparation and evaluation determine whether a classifier is useful. Selecting an SVM does not remove the need to handle changing document formats, uncertain labels or operational review queues.

Common misconceptions and mistakes

• Support vectors are always the most representative examples. They are examples that influence the boundary under the fitted optimization problem.
• An SVM score is a probability. Probability estimates need an additional, evaluated procedure where required.
• A nonlinear kernel is always better. Additional flexibility can overfit and increase computation; a linear model may be sufficient.

Business use

SVMs are useful candidates when a team has suitable numerical representations and wants a strong classification baseline. Practical suitability depends on dataset size, feature representation, latency and training cost.

Ask for comparison with simpler methods using the same evaluation split. Include preprocessing in the deployed pipeline and document how decision thresholds relate to review capacity. For very large datasets, investigate computational requirements before committing to a kernel-based approach.

People also ask

Can an SVM predict more than two categories?

Yes. Implementations commonly combine binary problems using multiclass strategies.

Does it work without feature scaling?

Sometimes, but scaling often matters and should be evaluated as part of the pipeline.

Is support vector regression the same task?

It uses related ideas to predict numerical values rather than class labels, with a different objective.

Check your understanding

An SVM returns a score of 1.8. Can a dashboard display it as an 180% chance that the item belongs to a class?

Answer: No. A decision score is not a percentage probability. Any probability display requires a suitable conversion whose calibration has been assessed.

Related terms

• Classification
• Margin
• Kernel
• Regularization
• support vector
• feature scaling

Learn next

Read classification and regularization. Compare a linear SVM with a simple baseline using identical splits and preprocessing, then examine the errors that create the most operational work.

• Classification
• Regularization

More in AI and machine learning

• Superintelligence ban - A superintelligence ban is a proposed prohibition on developing, deploying, or otherwise enabling AI systems that exceed a defined superintelligence threshold.
• Supervised Clustering - Supervised clustering groups objects using guidance from known labels.
• Supervised learning - Supervised learning trains a model using examples that pair inputs with known target outputs.
• SUPS - SUPS stands for synaptic updates per second, a throughput measure used in neural simulation and some neural computing contexts.
• Surrogate model - A surrogate model is a simpler or cheaper model used to approximate the behavior of another system, model, experiment, or computational process.
• Switching Hidden Markov Model - A switching hidden Markov model describes a sequence by switching among the outputs of several hidden Markov models.

Sources

scikit-learn, Support Vector Machines: https://scikit-learn.org/stable/modules/svm.html. Primary implementation documentation opened for terminology, probability and practical considerations. All numerical examples and organisations are fictional.

Previous: Supervised principal component analysis Next: Supporting hyperplane