OpenAI Five 击败 Dota 2 世界冠军

2019 年 4 月 13 日，OpenAI Five 在旧金山面对 OG——上一届 Dota 2 国际邀请赛冠军。赛制为三局两胜，Five 连赢前两局，比赛提前结束。屏幕上看见的是五名英雄像一支队伍那样推进、支援和团战；屏幕后并没有一位教练逐条写下战术，五个智能体各自控制一名英雄，通过同一个团队胜负信号练习协作。

Dota 2 比棋盘游戏更像一场持续失控的工程实验。它没有整齐轮次，战争迷雾遮住部分地图，一局可持续数十分钟，早期一次买装备或走位的影响可能很久之后才显现。每个智能体都要在不完整信息下行动，又要猜测队友与对手接下来会做什么。胜利属于整个团队，这使单个动作应得多少功劳变得难以分配。

OpenAI 的办法是把算力变成经验。大规模分布式强化学习让系统不断与自己的不同版本对战，公开材料将训练量折算为每天约 180 年的人类游戏时间。算法使用以 PPO 为核心的策略优化，观测主要来自游戏 API 提供的结构化状态，而不是直接从视频像素理解整个画面。奖励设计、训练基础设施和持续更新的对手池，都是能力的一部分；这不是一条只凭“自我博弈”四个字就会自动出现的曲线。

公开胜利也有清楚的规则围栏。比赛使用约 17 名英雄的池子，并限制部分物品和机制，远小于完整 Dota 的全部组合。即便经过裁剪，环境仍包含实时控制、长程规划和五人协作，因此击败 OG 是实质性结果；但它不能被改写成“AI 已经征服完整 Dota”。补丁改变数值、英雄或地图后，重新适应所需的计算，正是机器与职业选手不在同一条成本曲线上的地方。

比赛结束后，人类玩家还曾通过公开 Arena 与 Five 对局，进一步暴露策略在新打法和分布变化下的强弱。这样的后续比一场胜利更接近研究本身：固定环境里练到极致的策略，能把多少能力带到略有不同的对局？Five 给出的答案并不完整。它展示了规模化自我对弈可以在嘈杂的多人环境中产生复杂配合，也展示了这种配合依赖数量惊人的模拟经验与精心维护的训练世界。

OG 被击败的两局值得记住，但英雄池列表和“每天 180 年”也应与比分放在一起。前者说明系统究竟赢了什么，后者说明胜利是怎样被制造出来的。OpenAI Five 把强化学习从棋盘推入实时团队战场，同时把一张更难回避的账单留在场外：当经验可以自动生成，学习能力会迅速扩张；当现实无法被高速重置，样本成本又会重新成为边界。

On 13 April 2019, OpenAI Five met OG in San Francisco. OG had won the previous year’s Dota 2 International; the exhibition was best of three. Five won the first two games, ending the series early. On screen, five heroes pushed lanes, reinforced one another, and entered team fights as a unit. Behind them was no coach scripting every maneuver. Each agent controlled one hero and learned through a shared signal: whether the team eventually won.

Dota 2 is less orderly than a board game and closer to an experiment that keeps trying to escape its apparatus. There are no clean turns. Fog of war hides much of the map, matches last tens of minutes, and the consequence of an early purchase or movement may not become clear until much later. Every agent acts with partial information while anticipating teammates and opponents. Because victory belongs to the group, deciding how much credit any single action deserves becomes part of the learning problem.

OpenAI’s answer was to turn compute into experience. Large-scale distributed reinforcement learning kept the system playing against changing versions of itself. Public materials translated the volume into roughly 180 years of human play each day. The training system used policy optimization centered on PPO, and observations largely arrived as structured state from the game API rather than as an attempt to understand the entire match from raw video. Reward design, infrastructure, and a maintained population of opponents were part of the capability. The curve did not emerge from the words self-play alone.

The public victory also stood inside a visible rule fence. The match used a pool of about seventeen heroes and restricted some items and mechanics, far short of every combination in full Dota. The clipped environment still demanded real-time control, long planning horizons, and five-player coordination, so beating OG was a substantive result. It was not evidence that AI had conquered unrestricted Dota 2. When patches alter heroes, numbers, or maps, the compute required to adapt reveals a cost curve very different from that of professional players.

After the exhibition, public Arena matches gave human teams further opportunities to probe Five, exposing strengths and failures under unfamiliar tactics and distribution shifts. Those encounters were closer to the research question than a trophy alone: how much does a policy trained to an extreme in one environment carry into a slightly different one? Five did not settle the question. It showed that scaled self-play could produce intricate cooperation in a noisy multiplayer world, and that the cooperation depended on an extraordinary quantity of simulated experience inside a carefully maintained training universe.

The two wins over OG belong in the record, but so do the hero list and the “180 years per day” conversion. One says what the system defeated; the other says how the victory was manufactured. OpenAI Five pushed reinforcement learning from boards into a real-time team battlefield while leaving a difficult bill outside the arena: when experience can be generated and reset at speed, learning scales dramatically; when reality cannot, sample cost returns as the boundary.