斯坦福 Alpaca 低成本复刻

2023 年 3 月，开源社区还沉浸在对 LLaMA 的兴奋里——Meta 刚开源了这款基础模型，但「微调大模型」这件事，在很多人印象里依然是大公司才玩得起的游戏：几百张卡、几百万预算、几周时间。斯坦福的一个小团队决定打破这个印象。

他们的做法出奇地简单。既然没有高质量的开源指令数据，那就向 GPT-3.5「借」：让 GPT-3.5 生成 52K 条指令和回复，用这些数据去微调 LLaMA-7B。结果训练一次只要约 3 小时，花费不到 100 美元；算上整个流程，成本大约 600 美元。一个行为上接近 GPT-3.5 的助手，就这样「白菜价」地诞生了。

Alpaca 发布后迅速引爆社区。它的意义不在于模型本身有多强——简单任务上它表现不错，复杂任务就露怯了——而在于它撕开了一个可能性：把一个像样的指令模型造出来，不需要大公司级别的资源。全球的研究者、创业团队、个人开发者，都能在这个基础上复现、改进、做实验。

紧随其后的是一系列争议。用闭源模型的输出来训练开源模型，版权上、伦理上都站不住脚的地方很多。OpenAI 后来修改了服务条款来限制这类「蒸馏」，但 Alpaca 打开的那扇门已经关不上了——低成本微调、数据蒸馏成为开源生态最常用的玩法。

回看 Alpaca，它像是开源 AI 的一次「点火试验」。六百美元的训练成本，把「人人可造模型」从口号变成了现实；关于数据版权的争论，也从此成为开源大模型绕不开的议题。它没有造出最强的模型，却改写了整个行业对「造模型需要什么」的认知——这件事本身，就足够在历史上留下名字。

In March 2023, the open-source community was still buzzing about LLaMA—Meta had just open-sourced the base model, but "fine-tuning a large model" still felt, to most people, like a game only big companies could play: hundreds of GPUs, millions in budget, weeks of time. A small Stanford team decided to break that impression.

Their approach was strikingly simple. Since no high-quality open instruction data existed, why not "borrow" from GPT-3.5: have GPT-3.5 generate 52K instructions and replies, and fine-tune LLaMA-7B on that data. The result: one training run took about three hours and cost under $100; the whole process totaled around $600. An assistant with behavior approaching GPT-3.5 had been born at a bargain price.

Alpaca exploded across the community immediately after release. Its significance lay not in how strong the model was—it handled simple tasks well but exposed its limits on complex ones—but in the possibility it tore open: building a decent instruction model did not require big-company resources. Researchers, startups, and individual developers worldwide could reproduce, improve, and experiment on top of it.

A wave of controversy followed. Training an open model on a closed model's outputs was shaky on copyright and ethics grounds. OpenAI later revised its terms of service to restrict such "distillation," but the door Alpaca opened could not be closed—low-cost fine-tuning and data distillation became the most common play in the open ecosystem.

Looking back, Alpaca reads like an ignition test for open AI. A six-hundred-dollar training cost turned "anyone can build a model" from slogan into reality, and the debate over data copyright has been an unavoidable topic for open models ever since. It did not build the strongest model, but it rewrote the industry's understanding of what building a model requires—and that alone earns it a place in history.