支持向量机

1995 年，Corinna Cortes 与 Vladimir Vapnik 在《Machine Learning》杂志上发表了一篇论文，题目朴素，内容却为之后的十年定下了基调：《Support-Vector Networks》。支持向量机——SVM——从此进入机器学习的主流工具箱。这篇论文不是 SVM 的第一次亮相，Vapnik 和他的团队此前已在 AT&T 做过相关工作，但 1995 年这篇是系统而完整的总结。

SVM 要解决的老问题是分类：给定一堆带标签的点，画一条线把它们分开。难的是怎么画才最好。Vapnik 的回答带着统计学家的底气：不是随便哪条能把点分开的线都好，要选间隔最大的那一条——离两类样本都最远的线。为什么？因为间隔越大，对新数据的泛化越有保障。这条线只由最靠近它的少数样本决定，这些样本被称作支持向量，整个算法因此得名。

线性不可分怎么办？SVM 用了核技巧：一个核函数把数据隐式映射到更高维的空间，在那边做线性划分，回来后在原空间看起来就是一个弯曲的边界。计算量没有真的爆炸，因为算法里只出现样本两两之间的内积，而核函数直接给出高维内积的结果。这个设计优雅又实用，让 SVM 能处理文本分类、图像识别、生物信息学里大量高维稀疏数据。

1990 年代末到 2000 年代，SVM 几乎成为机器学习的默认答案。文本分类、手写识别、蛋白质分类、癌症基因表达分析——SVM 出现在各种竞赛排行榜和应用论文里。它稳定、有理论、有开源工具（如 LIBSVM），在神经网络再次崛起之前，它是「聪明算法」的代名词。相比当时训练困难、没有理论保证的神经网络，SVM 提供了工程界最需要的可预期性。

转折发生在 2012 年。AlexNet 在 ImageNet 上以碾压姿态夺冠，深度学习随之接管了视觉、语音、文本的绝大多数任务。SVM 没有被彻底淘汰——它仍是许多小数据任务和基线比较的可靠选择——但聚光灯已经转移。这种交替本身是机器学习史最自然的一幕：每一种方法都有一段属于自己的统治期，而被取代不等于没有贡献。

回看 SVM，它的遗产有两层。方法层面，它把泛化理论、凸优化和实用算法缝在了一起，教会了整整一代研究者「间隔」「边界」「支持向量」这些概念。历史层面，它承接了神经网络第一次衰落留下的空白，守住了机器学习的地盘，直到深度学习的浪潮再次涌来。今天的教科书里，SVM 与核方法依然是必修章节——它用一种近乎数学上的干净，证明了一个朴素信念：好的理论，最终会变成好用的工具。

In 1995, Corinna Cortes and Vladimir Vapnik published a paper in the journal Machine Learning with a modest title that set the tone for the following decade: "Support-Vector Networks." Support vector machines—SVM—entered the mainstream toolbox. This paper was not SVM's debut; Vapnik's group had been working on related ideas at AT&T. But the 1995 paper was the systematic, complete statement.

SVM attacked the old problem of classification: given labeled points, draw a line that separates them. The question was how best to draw it. Vapnik's answer carried a statistician's confidence: not any separating line is good—choose the one with the maximum margin, the line farthest from both classes. Why? The larger the margin, the better the generalization to new data. This line is determined by only the few closest samples, called support vectors—hence the algorithm's name.

What about data that is not linearly separable? SVM used the kernel trick: a kernel function implicitly maps the data into a higher-dimensional space, draws a linear boundary there, and back in the original space that boundary appears curved. Computation does not actually explode, because the algorithm only ever sees inner products between pairs of samples, and the kernel directly yields the inner product in the high-dimensional space. Elegant and practical, it let SVM handle the high-dimensional sparse data of text classification, image recognition, and bioinformatics.

From the late 1990s through the 2000s, SVM was almost the default answer in machine learning. Text classification, handwriting recognition, protein classification, cancer gene-expression analysis—SVM showed up in leaderboards and application papers everywhere. It was stable, had theory, and had open-source tools like LIBSVM; in the years before neural networks rose again, it was synonymous with "smart algorithm." Compared to neural networks that were hard to train and lacked theoretical guarantees, SVM gave engineers the predictability they needed most.

The turning point came in 2012. AlexNet crushed ImageNet, and deep learning took over vision, speech, and text tasks almost wholesale. SVM was not eliminated—it remained a reliable choice for small-data tasks and baselines—but the spotlight had moved. That succession is one of the most natural scenes in machine-learning history: every method has its reign, and being replaced is not the same as having no contribution.

Looking back, SVM's legacy has two layers. Methodologically, it stitched generalization theory, convex optimization, and practical algorithms together, teaching a generation concepts like margins, boundaries, and support vectors. Historically, it held the fort during the valley between neural-network waves, keeping machine learning alive as a discipline until the deep-learning tide returned. SVM and kernel methods remain required reading in textbooks today—evidence, in near-mathematical cleanliness, of a plain belief: good theory eventually becomes a good tool.