Working the Roofline for DeepSeek-V3 on Hopper | Hacker News - But Better

Working the Roofline for DeepSeek-V3 on Hopper deepseek-v3.ezyang.com

By carefully reasoning out the mechanics of how our distributed training works, we can put together a simple set of formulas that let us model how our system will behave, even if we don't actually have a cluster of…

1 points•matt_d•about 3 hours ago•0 comments•

#roofline #hopper #deepseek #carefully #training #mechanics #distributed #reasoning

0 comments

No comments yet.

Related stories

OpenAI halts frontier-model training amid string of agent misalignment incidentsarstechnica.com
Ars Technica · 0 points · 5 days ago
An experimental (toy) cloud-native distributed KV engine for OLTP workloadgithub.com
Lobsters · 2 points · over 4 years ago
A gentle overview of the common ways to tackle the problem of distributed sequential IDs.github.com
Lobsters · 3 points · about 10 years ago
Skyway: connecting managed heaps in distributed big data systemsblog.acolyer.org
Lobsters · 1 points · over 8 years ago
DeepSeek is training a 2T-parameter model and plans to build an 8T-parameter onetwitter.com
Hacker News · 4 points · 12 days ago
Axon Training Pod (ATP) VR for training police officersaxon.com
Hacker News · 2 points · 11 days ago