A support vector machine (SVM) decides where to draw the line between two groups using only a subset of the training examples, the ones that sit closest to that line or on the wrong side of it, which are the hardest to tell apart. You could delete all the other examples and the model wouldn’t change.
I know SVMs well because they were my first tool for language work. My first natural language processing project, in 2000, was an SVM that sorted customer support emails into categories, and for most of the next decade SVMs were a default choice for sorting text, images, and lab results into groups. They’re less fashionable now, but the idea behind them is one of the cleanest in machine learning, and they still beat bigger models on some problems.
The short version: A support vector machine is a classification model that separates two groups by finding the boundary with the widest possible gap between them. Only the examples closest to that boundary or on the wrong side of it, called support vectors, decide where it goes. With a technique called the kernel trick, an SVM can also draw curved boundaries when a straight line won’t work.
How an SVM draws the boundary
Picture a scatter plot of customer emails. Each dot is one email, placed by two measurements: how many times it mentions an order number and how many exclamation points it has. Billing questions cluster in one corner, angry complaints in another. You could draw many straight lines that separate the two groups, and most of them would pass very close to a few dots on one side or the other.
Out of all the lines that separate the two groups, an SVM picks the one with the widest empty strip on both sides. That strip is called the margin. A wide margin leaves room for error: a new email that lands a little off from the training examples still ends up on the right side.
The examples sitting right at the edge of the margin are the support vectors, and they’re the only ones that decide where the boundary goes. Move or delete any other example and the boundary stays where it is, as long as that example stays outside the margin. “Vector” is the math word for the list of numbers that describes each example. Once trained, the model only needs to keep the support vectors, often a small fraction of the original data, which made SVMs efficient for their time.
Real data has far more than two measurements, so the “line” is usually a flat surface in many dimensions, called a hyperplane. The idea stays the same in 2 dimensions or 20,000.
When the groups overlap
Real data is messy. Some billing questions are written by angry customers, and no straight line will separate every example perfectly. SVMs handle this with a soft margin: the model is allowed to misplace a few training examples in exchange for a wider, more sensible boundary. With a soft margin, any example inside the margin or on the wrong side also counts as a support vector.
A setting usually called C controls that trade. A high C tells the model to get as many training examples right as possible, even if the margin gets narrow and fussy. A low C tells it to accept a few mistakes and keep the margin wide. Set it too high and the model memorizes its training data, a problem called overfitting. Set it too low and it misses real patterns. C is the SVM’s regularization setting, and you find the right value by testing on data the model hasn’t seen, usually with cross-validation.
The kernel trick: curved boundaries
Sometimes no straight line works at all. Imagine fraud cases clustered in the middle of a chart, with normal transactions in a ring around them. Any line you draw cuts through both groups.
The fix is to add a measurement. If you add each point’s distance from the center as a third dimension, the fraud cases sit low and the normal transactions sit high, and now a flat sheet can slide between them. Back on the original chart, that flat sheet looks like a circle.
The kernel trick lets an SVM make this kind of jump, even into spaces with thousands of dimensions, or infinitely many in the case of the RBF kernel below, without ever calculating the new coordinates. It only needs a measure of how similar each pair of examples is, which is much cheaper to compute. Common choices include the polynomial kernel and the radial basis function (RBF) kernel, which draws boundaries around clusters of similar examples. RBF is the default in scikit-learn, one of the most widely used machine learning libraries.
How SVMs rose and fell
Vladimir Vapnik developed the core ideas over decades of work on statistical learning theory. In 1992, Bernhard Boser, Isabelle Guyon, and Vapnik combined wide-margin classifiers with kernels, and in 1995 Corinna Cortes and Vapnik added the soft margin in a paper called “Support-Vector Networks”.
SVMs spread quickly. They came with solid math, worked well on small datasets, and training one always finds the same best boundary for the same data, while the neural networks of that era could land somewhere different on every run. Through the 2000s, SVMs were used for spam filtering, text classification, handwriting recognition, and classifying tumors from gene expression data.
SVMs have two big limits. First, training time grows quickly with the number of examples. The scikit-learn documentation notes that fit time grows at least with the square of the number of examples, so beyond tens of thousands of examples, kernel SVMs get slow. Linear SVMs scale much further, but they can only draw straight boundaries. Second, an SVM works with the features you give it. Someone has to decide which measurements describe an email or an image well, and for images and language that turned out to be very hard. Neural networks learn their own features from raw data, and once GPUs and large datasets arrived, that advantage won.
I watched the switch happen. In late 2012, a colleague forwarded me the AlexNet image recognition results with a one-line note: “Looks like the neural net people were right.” Within a couple of years, most new projects on my team started with a neural network instead of an SVM. My history of language models covers what came next.
When an SVM is still a good choice
SVMs still make sense for small to medium datasets, from a few hundred to tens of thousands of examples. They do especially well when there are many features and few examples, such as text represented as word counts, or genetic data with thousands of measurements per patient. They also fit projects that need repeatable results and a model that trains in minutes on a laptop.
Kernel SVMs are a poor fit for very large datasets, where training time gets out of hand. For large spreadsheet-style datasets, gradient boosted trees like XGBoost usually win. For images, audio, and language, neural networks do.
Support vector machine FAQ
Is an SVM supervised or unsupervised?
Supervised. It learns from labeled examples, like emails already tagged as billing or complaint. See supervised vs. unsupervised learning for the difference. A variant called a one-class SVM is used to spot anomalies in unlabeled data.
Can an SVM handle more than two classes?
Yes. Libraries combine several two-class SVMs behind the scenes, either one class against all the others or every pair of classes against each other.
Can SVMs do regression?
Yes. Support vector regression fits a band around the data instead of a gap between groups, and ignores errors that fall inside the band. Here’s more on classification vs. regression.
What’s the difference between an SVM and logistic regression?
Both draw boundaries between groups. Logistic regression uses every example and gives a probability. An SVM focuses on the examples near the boundary and gives a yes-or-no answer, and needs an extra step to produce probabilities. With a straight-line boundary, the two often perform about the same.
Neural networks are the natural next step, since they took over most of the work SVMs used to do. What Is a Neural Network? explains how they learn their own features. And not every problem needs a model at all. If you’re deciding what kind of tool fits your problem, start with Do You Need Rules, ML, an LLM, or an AI Agent? or take the AI-ML Fit Quiz.
