蒸馏在AI训练中的现状及争议

Where I currently stand on how distillation is used and what performance uplift it uses. For context, Anthropic HAS said that DeepSeek and others are using their models in an RL shaped data pipeline, but that does not mean it has substantial effect. These were very small numbers of samples, and likely to initialize an internal model or do a small training experiment. The place where distillation is used as a key step is in SFT and/or midtraining (for seeding reasoning behaviors, thinking of sft and midtraining as separate is not helpful). This is why the Chinese models say they're claude -- when training with next-token prediction, the models learn "features" that are clusters of tokens, which will regularly appear in model outputs. This SFT data is generated by skirting the intended behavior of the API and getting reasoning tokens out. Without getting the reasoning tokens directly from the model, the data would be very hard to train on (and likely not include the "I'm claude" behaviors). SFT datasets have been O(1 million prompts) with high quality completions. Today, getting the prompts is often the hardest part -- especially prompts with carefully designed environments for proportional RL later, but this is mostly about SFT. Getting to the end of it. The part where this SFT distillation is likely very useful is when a model moves into a new domain, a great example being the CripPT physics benchmark, where the US labs are far ahead. The idea would be to get some prompts, and get completions from a frontier model that you can use for an initial SFT. With an initial SFT set, there is A TON of work to still do to get a model in the performance ballpark of GLM 5.2 and Kimi K3. The SFT stage is the start of a long, strenuous process for building a post-training recipe. It involves generating more SFT data and filtering it, something like rejection sampling, and polishing the behavior with extensive RL. For models like Kimi K3 and GLM-5.2, the timeline is such that Fable 5 likely had no impact as a distillation teacher. That could help future models, though. And as post-training becomes more dependent on techniques like multi-teacher on policy distillation, rather than just RL, there is even more complexity in how SFT distillation helps the final model. There are a lot of steps, and RL has been meaningfully scaled up in it's proportion of the final performance. The real kicker for this is that in the above SFT stages, the best teacher models aren't normally the easies to integrate into the post-training recipe! The leading fully open, reasoning SFT works (open thoughts and olmo) have had a very hard time updating their recipes to use the strongest models as teachers. Often it is a smaller, surprising model which is the best teacher. This means that it may not even be the cutting edge models that are enabling distillation! Messy. All together, the evidence in post-training is that distillation is becoming less impactful. Previously, before scaling RL, distillation was more impactful because the relative amount of performance gained from SFT on top of the base model was far higher. I expect this trend to competitive, with how strong the best open weight models are -- and the teacher ambiguity clamps down on arguments that open-weight models are just "distillation washing" by removing the need to use Claude/GPT APIs. It could be that open models are just genuinely easier to distill from, by being easier to modify and tinker with. > **引用原帖 Nathan Lambert (@natolambert):** > Yo @benthompson I'm sorry but the Chinese labs aren't using Fable / the strongest models as teachers during RL, that's not how distillation works. > It wouldn't give that big of a lift (graders during RL are messy) and you cant afford to use Fable like that. > https://x.com/natolambert/status/2079586505476165822 Nathan Lambert (@natolambert): Hopefully this finds its way to @benthompson. Reading list below: Example benchmark: https://t.co/79YstmgM8W Chapter on synthetic data: https://t.co/ymMMfR7QWJ More on on policy distillation: https://t.co/iGjrFfUvnt MOPD paper: https://t.co/pJGdo2HLeZ OpenThoughts 3 (reasoning SFT data): https://t.co/tLyQdbwP06 Olmo 3 (using OpenThoughts in a recipe): https://t.co/e7SHHcmYyO Open agentic sft data work: https://t.co/vifexhtIux