OpenAI 开源 Whisper 语音识别

2022 年 9 月 21 日，OpenAI 放出 Whisper：代码、模型卡，以及从 tiny 到 large 的一系列权重。标题若写成“又一个语音识别论文”，会漏掉那天真正改变工作流的东西——文件可以下载，推理可以离线跑，不必先把会议录音交给某一家云端 ASR。对独立开发者与小团队，这往往比又高一个百分点的干净朗读基准更重要：权限、成本与延迟突然变成自己机器上的参数。

训练故事写在博客与随后的论文里：大约 68 万小时从网络收集的多语、多任务监督数据，本质是大规模弱监督——音频与字幕对齐，噪声、口音、领域混在一起，而不是只在 LibriSpeech 式干净语料上精修。模型是编码器–解码器 Transformer：音频进编码器，解码器自回归吐出文本 token，并在同一套多任务格式里覆盖转写、翻译成英语、语言识别等。官方强调：因为没有为单一基准微调，Whisper 在 LibriSpeech 这类竞赛指标上未必压过专项模型；但在多样数据集上的零样本表现更稳健，错误约比那些模型少 50%。主语是 OpenAI 的测量与报告，读作方向性证据，而不是第三方盲测判决书。把“更稳健”误读成“永不听错”，会在法庭录音或医疗口述里付出代价。

对开发者，数字不如接口重要。克隆仓库、装依赖、指向一段 wav，就能得到带时间戳的文本。播客字幕、课堂录音、采访整理、视频平台的多语轨道，突然多了一条不经过采购流程的路径。敏感内容可以留在本机；二次开发者把 Whisper 嵌进桌面工具与字幕流水线。社区很快长出 whisper.cpp 等端口，把同一套检查点塞进更瘦的运行时——官方给基线，社区付部署税。不同尺寸的检查点让人在精度与速度之间滑动：tiny 适合预览，large 适合终稿，中间档位填满笔记本电脑与服务器的空隙。

它没有结束商业 ASR。云服务仍在延迟、说话人分离、行业词表、流式协议与合规审计上竞争；Whisper 的价值是把“够用的通用转写”的地板抬高，并公开一套可复现的弱监督叙事。训练数据来自网络带来的偏见、幻觉式转写与隐私问题同样公开存在：模型会一本正经地听错专有名词，也会在噪声里编造流畅但错误的句子。负责任的用法是本地预转写加人工校对，或在领域数据上再适配，而不是把首版输出直接当档案正文。弱监督的另一面是标注伦理：网络字幕可能含错误、仇恨或隐私片段；开源让审计成为可能，却不自动完成审计。

时间线要分开写。9 月是权重与代码；更完整的论文 Robust Speech Recognition via Large-Scale Weak Supervision 在 12 月挂上 arXiv。后来的 large-v2、v3 与各类蒸馏、加速版属于版本史，不应回填成首日全部能力。首日留下的结构很清楚：用脏数据的规模换零样本稳健，用开源分发换生态速度。ChatGPT 几个月后会让“对话”成为大众动词；Whisper 则在更安静的一层，把“听写”变成许多软件里默认可以调用的库。2022 年 9 月的记忆，仍是一个可哈希的模型文件，和一段终于能在笔记本上跑完的转写——不完美，可下载，够你开工。

On 21 September 2022 OpenAI released Whisper: code, a model card, and a ladder of weights from tiny to large. Call it “another speech-recognition paper” and you miss what actually changed that day—files you could download, inference you could run offline, no need to hand a meeting recording to a cloud ASR first. For independent developers and small teams that often mattered more than another point on a clean read-speech leaderboard: permission, cost, and latency became parameters on their own machines.

The training story sits in the blog and the later paper: roughly 680,000 hours of multilingual, multi-task supervision scraped from the web—large-scale weak supervision of audio aligned to subtitles, with noise, accents, and domains mixed in, not a clean LibriSpeech-only polish. The model is an encoder–decoder Transformer: audio into the encoder, autoregressive text tokens from the decoder, covering transcription, translation into English, language identification, and related tasks in one multi-task format. OpenAI stressed that because the model was not fine-tuned for a single benchmark, it need not beat specialized systems on LibriSpeech-style contest metrics; on more diverse sets its zero-shot behavior was more robust, with roughly 50% fewer errors than those models in the company’s measurements. Read that as directional evidence under a named subject, not a third-party blind verdict. Mistaking “more robust” for “never mishears” has costs in courtroom audio and medical dictation.

For developers the numbers matter less than the interface. Clone the repo, install dependencies, point at a wav, get timestamped text. Podcast captions, classroom recordings, interview cleanup, multilingual tracks for video platforms suddenly had a path that skipped procurement. Sensitive audio could stay on-device; third parties embedded Whisper in desktop tools and caption pipelines. Ports such as whisper.cpp soon pushed the same checkpoints into leaner runtimes—baseline from the lab, deployment tax paid by the community. Size tiers let people slide between accuracy and speed: tiny for preview, large for final draft, intermediate checkpoints filling the gap between laptops and servers.

Commercial ASR did not end. Cloud services still compete on latency, speaker diarization, industry lexicons, streaming protocols, and compliance audit. Whisper’s value was raising the floor of “good enough general transcription” and publishing a reproducible weak-supervision narrative. Bias, hallucinated transcripts, and privacy risks from web data are equally public: the model can mishear proper names with confidence and invent fluent wrong sentences in noise. Responsible use is local pre-transcription plus human correction, or domain adaptation—not treating first-pass output as archival text. Weak supervision also carries labeling ethics: web captions may contain errors, hate, or private fragments; open release makes audit possible without completing it.

Keep the timeline split. September was weights and code; the fuller paper Robust Speech Recognition via Large-Scale Weak Supervision hit arXiv in December. Later large-v2, v3, and the distillation and acceleration ecosystem are version history, not day-one capability. Day one left a clear structure: dirty data at scale for zero-shot robustness, open distribution for ecosystem speed. ChatGPT would soon make “chat” a mass verb; Whisper, more quietly, made “dictation” a library many products could call. The September memory is still a hashable model file and a transcription that finally finishes on a laptop—imperfect, downloadable, enough to start.