distillation
3 stories and discussions about distillation, aggregated from every source we track.
1.
Nvidia CEO Jensen Huang argued that AI model distillation is "competition". U.S. Treasury Secretary Scott Bessent has called it "theft."
2.
On-policy distillation (OPD) trains a student model by having it generate trajectories, then matching its next-token predictions with an external teacher's next-token predictions. This provides dense, token-level…
3.
Large language models (LLMs) implicitly infer attributes of their users and adapt their behavior accordingly, yet these beliefs remain difficult to inspect and causally manipulate. We introduce Belief Self-Distillation…