DeepSeek-V3 发布

2024 年 12 月 26 日，DeepSeek 发布 V3 权重与技术报告。报告中的费用表很快被摘出一个醒目的数字：最终预训练阶段使用约 278.8 万 H800 GPU 小时，按文中每 GPU 小时 2 美元估算，约为 557.6 万美元。这个数字让前沿模型的训练成本第一次显得像一张任何人都能读懂的账单，也因此最容易被读错。

表格计算的是最终预训练 run 的算力，不是整个项目的发票。数据获取与清洗、人员、失败实验、前期架构探索、基础设施建设和上线后的推理服务，都不在这一个数字里。把“557.6 万美元”改写成“做出一个前沿模型只花了 557.6 万美元”，等于把一项会计科目冒充全部成本。报告真正改变公共讨论的地方，不是宣称研发变得廉价，而是明确公开了一个可比较的范围。

V3 的架构解释了这笔算力为何值得与传统稠密模型分开看。模型约有 6710 亿总参数，但每个 token 只激活约 370 亿。混合专家路由让巨大的参数容量不必在每一步全部参与计算；Multi-Head Latent Attention 则试图压低注意力中的键值缓存与带宽压力。总参数决定权重存储和装载的规模，激活参数更接近单次前向计算的负担。两个数字并列，才能看见所谓效率究竟发生在哪里。

报告还记录了路由平衡、数值稳定和训练系统的工程处理。对如此规模的稀疏模型而言，“理论上少激活参数”远远不够：专家负载失衡会浪费设备，训练不稳定会让已花掉的 GPU 小时无法转化为可用权重。V3 的成本故事因此不是事后把总时长乘以单价，而是算法、网络通信和系统调度共同让最终训练能够完成。

发布方给出了代码、数学和通用知识等基准结果，并开放权重供外部部署与研究。基准仍是厂商评测，许可证也应以仓库文本为准；开放权重不自动等于最宽松的开源许可。对开发者而言，更直接的变化是高能力模型有了私有部署与二次研究的入口。对行业而言，它迫使每一家声称“训练成本巨大”或“效率领先”的公司，更具体地回答自己到底统计了什么。

阅读 DeepSeek-V3，最好让架构图和费用表停留在同一页。6710 亿总参数与 370 亿激活参数说明计算被怎样裁剪；278.8 万 GPU 小时说明报告愿意公开哪一段资源消耗；“最终预训练”则说明还有哪些支出没有进入镜头。V3 没有给出完整发票，却把成本神秘化撕开了一道足够宽的口子——前提是读者愿意把脚注也读完。

DeepSeek released V3 and its technical report on 26 December 2024. One line from the cost table quickly escaped into headlines: the final pretraining run used about 2.788 million H800 GPU-hours, which the report valued at roughly $5.576 million using an assumed rate of two dollars per GPU-hour. The figure made frontier-model training look, for once, like a bill that anyone could read. That also made it unusually easy to misread.

The table accounted for compute in the final pretraining run, not the invoice for the entire project. Data acquisition and cleaning, staff, failed experiments, earlier architectural work, infrastructure, and post-launch inference were not contained in that number. Turning “$5.576 million for final pretraining compute” into “a frontier model cost $5.576 million to create” substitutes one accounting category for all the others. The report’s important act was more disciplined: it published a defined slice of cost that outsiders could compare.

V3’s architecture explains why that slice should not be read like the bill for an ordinary dense model. The system had about 671 billion total parameters but activated roughly 37 billion for each token. Mixture-of-experts routing allowed a large stored capacity without involving every parameter in every step. Multi-Head Latent Attention aimed to reduce key–value cache and bandwidth pressure. Total parameters governed the size of storage and loading; active parameters tracked more closely with the work of a forward pass. Efficiency became visible only when both numbers stayed on the page.

The report also described engineering for routing balance, numerical stability, and the training system itself. At this scale, “fewer active parameters in theory” is not sufficient. Uneven expert load can strand hardware, and instability can turn already-spent GPU-hours into no usable checkpoint at all. V3’s cost story was therefore not an after-the-fact multiplication of time by price. Algorithms, network communication, and systems scheduling had to make the final run finish.

DeepSeek supplied publisher evaluations across code, mathematics, and general knowledge, and released weights for outside deployment and research. The benchmarks remained vendor results, and the repository license remained the authority; downloadable weights do not automatically imply the most permissive open-source terms. For developers, the practical change was access to private deployment and further study of a high-capability model. For the industry, the report made every claim about “enormous training cost” or “leading efficiency” answer a more specific question: what exactly was counted?

The most useful reading of DeepSeek-V3 keeps the architecture diagram and the cost table open together. The 671-billion total and 37-billion active parameters show how computation was selected. The 2.788 million GPU-hours show which resource interval the authors chose to expose. The words final pretraining show what remained outside the frame. V3 did not publish a complete invoice. It cut a wide opening into the cost mystique—provided the reader was willing to finish the footnote.