A) How to prevent malicious injection of bad training data?

B) How to handle copyright violations, will participants be responsible and will they have to pay the creators?

C) Can this create an income stream for content creators and how to avoid abuse, eg feeding with AI content?

So many more, but let’s focus on these before we break things fast because we didn’t think about them.

Here is one really neat bit:

A cutting edge training idea (for agents, it's been used elsewhere for ages) is on-policy RL, basically, it's not enough to say "here is an end to end agentic sequence (including tool calls etc.) that is perfect" you want to say "here is a sequence you might actually have generated that turns out to be correct".

Basically, it's more training efficient to improve models with small tweaks to do more of the right thing they are already doing sometimes than from some perfect oracular "this is the way" answer.

(if you've ever tried to teach humans new skills, you’ve probably noticed this too!)

When you do that, you care about how far the model you are updating (improving) has deviated from the one being used to generate rollouts (agentic rollouts for hard problems can take hours with lots of tool calls, so you can't keep redeploying every slight improvement).

Lo and behold, the dashboard literally has:

partial/avg_staleness (likely the measure of how many micro iterations the "generate answers" model is behind the "improving based on the occasional right answer" model)

train_infer_diff/new_infer/kl (a more direct KL divergence based way of measuring how differently the two models generate tokens)

How cool is that?!

And don't get me started on the clever ideas hiding behind dynsam/avg@n ...

Really the only thing missing was dataset descriptions, the dashboard only had random IDs like "dataset-zrso". I guess it's their lawyers fault.

I think you mean mostly the US

I think your numbers are off.

For a start you are effectively calculating a GPU only number.

I think 100Twh would be the minimum level to think about "all-in". And even that is probably being generous.

Remember, afterall that Google have just bought half the capacity (4.1Twh) of a nuclear power plant in Finland, on top of 630 MW of wind and 94MW of battery.

This is to cater for three new sites at Kajaani, Muhos, and Vaala and expansion at Hamina. So basically 3.5 datacentres.

But Finland is quite a small place. The US has more sites and bigger sites, so the numbers probably grow exponentially very quickly.

Datacenters enable anyone with a computer to use it.

for wwiii it would likely imply a war in asia too, so it's not as if PRC will be a safe place for those researches to run away to.

It's almost as if collaboration is the foundation of scientific progress. Too bad US has lost the notes.

This is also true for Deepseek 4(.1) .

1: https://www.scbench.ai/

Jokes asides, @simonw any plan to include a 3D model version (make a 3D model of a Pelican riding a bicycle in Blender)?

Given that Astra seems to have improved a lot in 3D modeling capabilities (and that matches my experience) I'm actually quite interested to see if/when other models catch up and how they stack against it.

Or if anyone knows what would be a useful existing benchmark for that skill.