Seq2Seq 统一可变长度序列转换

图 1 里，源句的 ABC 是倒着写的。

2014 年前后，机器翻译的工业与学术主流仍是统计机器翻译：分词、短语表、对齐模型、调序模型与语言模型各有训练目标，再在解码时拼成一条假设。神经网络已经能做分类与固定标签任务，但“整句进、整句出”的端到端学习仍不常见。Sutskever、Vinyals 与 Le 在 Sequence to Sequence Learning with Neural Networks 里给出的结构看起来粗暴而干净：一个多层 LSTM 读完整句源文，最终隐状态交给另一个多层 LSTM，逐词生成目标句，直到输出结束符。

编码器把变长序列压缩成固定维向量；解码器以该向量为条件，每一步在目标词表上做分类。训练使用教师强制——解码器在真实前缀上学习下一词分布，整句负对数似然作为损失。推理时用束搜索在候选前缀上扩展。论文采用四层 LSTM，隐层宽度 1000，在 WMT’14 英译法上报告有竞争力的 BLEU；解码侧词表约 8 万量级。对神经网络来说，这意味着第一次把“源序列到目标序列”收成单一可微目标，而不是先对齐短语再拼装。

最容易被记住的实验细节，正是那反向的 ABC。作者发现把源句词序反转后送入编码器，训练更稳定，长句 BLEU 明显改善。工作直觉是：反转后，源句开头与目标句开头在时间上更近，反向传播要跨越的有效距离变短，梯度更容易建立对应。这不是漂亮的语言学理论，而是带着实验运气的工程观察，却被写进主文与插图，成为这篇论文的识别符号之一。深度与宽度、梯度裁剪、大型词表上的训练技巧，则共同支撑了“深层 LSTM 也能训起来”的另一半故事。

固定向量瓶颈没有消失。很长的句子仍要把全部信息挤进编码器末状态；信息瓶颈与长程依赖会在后续工作中被注意力机制松开——Bahdanau 等人让解码器每一步回看源句各位置，2017 年的 Transformer 则连循环本身也去掉。那些步骤不属于 2014 年这篇论文的正文。本文留下的是可迁移的任务形状：翻译、摘要、对话、语音识别中的序列映射，都可以先叙述成“读入一条序列，写出另一条序列”。流水线组件仍可争论，框架已经统一。

论文还报告了模型对长句的敏感性分析：在未反转源句时，性能随源句长度增加而下降更陡；反转后曲线明显变平。束搜索宽度、词表截断与罕见词处理等解码细节，也决定了 BLEU 能否接近短语系统。端到端并不意味着没有工程旋钮，只是旋钮从“短语表特征权重”换成了“网络深度、隐层宽度与搜索宽度”。与统计短语系统的对比表明，神经模型在短句上已可竞争，在长句与稀有词上仍需技巧支撑。序列到序列并没有消灭特征工程，它把特征工程从短语表迁移到网络结构与训练规程。

把源句倒过来读，听起来不像重大发明。它提醒读者，端到端神经翻译最初的突破里，既有编码器—解码器的结构选择，也有对优化路径的细小改写。可变长映射被一个损失函数接住；长程依赖则靠一次反转，暂时被拉近到 LSTM 能够稳定学习的距离之内。统计短语系统不会在一夜之间消失，但研究方向已经开始偏向：用同一网络直接学习整句对应，而不是永远维护一张巨大的短语表。

In Figure 1, the source ABC is written backwards.

Around 2014, production machine translation still chained tokenization, phrase tables, alignment models, reordering, and language models, each with its own objective. Neural nets could already classify and emit fixed labels, but whole-sentence in, whole-sentence out end-to-end learning was still uncommon. In Sequence to Sequence Learning with Neural Networks, Sutskever, Vinyals, and Le did something coarser and cleaner: one multilayer LSTM reads an entire source sentence; another multilayer LSTM receives the final hidden state and emits the target one token at a time until an end symbol appears.

The encoder compresses a sequence of any length into a fixed-dimensional vector. The decoder is conditioned on that vector and, at each step, predicts the next word. Training uses teacher forcing: the decoder learns next-token distributions given true prefixes, minimizing the negative log-likelihood of the full target. Inference expands candidates with beam search. The paper used four-layer LSTMs with 1,000-unit hidden states and reported competitive BLEU on WMT’14 English-to-French, with a decoder vocabulary on the order of 80,000 words. For a neural model, that meant optimizing whole-sentence mapping under a single differentiable loss rather than assembling phrase alignments after the fact.

The detail easiest to remember sits in that reversed ABC. The authors found that reversing the source word order—feeding the encoder the source sentence backwards—made training easier and improved long-sentence performance. A working intuition is that early source words then sit closer in time to early target words, shortening the effective path gradients must travel. It is not an elegant linguistic theory so much as an engineering observation that made it into both the main text and the diagram. Depth, width, gradient clipping, and large-vocabulary training tricks carried the other half of the story: multilayer LSTMs could actually be trained.

Seq2Seq did not remove the fixed-vector bottleneck; very long sentences still had to squeeze all information into the encoder’s final state. Attention mechanisms that followed, notably Bahdanau et al., let the decoder re-read source positions at each step; the 2017 Transformer later removed recurrence itself. Those steps are not this paper’s body text. What remains is a transferable task shape: translation, summarization, dialogue, and many later text-to-text problems can first be described as reading one sequence and writing another. Components of the pipeline could still be debated; the frame had unified.

Length analyses in the paper show performance falling more steeply with source length before reversal and flattening afterward. Beam width, vocabulary truncation, and rare-word handling still decide whether BLEU approaches phrase-based systems. End-to-end does not mean knob-free; knobs move from phrase-table feature weights to depth, width, and search. Against statistical phrase systems, neural models could already compete on short sentences and still needed craft on long ones and rare words. Sequence-to-sequence did not abolish feature engineering; it moved feature engineering from phrase tables into architecture and training protocol.

Reading the source backwards does not sound like a grand invention. It is a reminder that early neural translation advances mixed architectural bets with small rewrites of the optimization path. Variable-length mapping was caught by one loss. Long-range dependence was, for a while, shortened by a reversal into a distance LSTMs could learn more stably. Phrase-based systems would not vanish overnight, but research direction had begun to lean: learn whole-sentence correspondence with one network, rather than forever maintain a giant phrase table.