Which model wins at Brood War? Results, recordings, costs, and game-by-game analysis of Codex, Claude, and Grok across 171 matches.

346 points•benswerd•12 days ago•156 comments•

156 comments

AntiRush11 days ago
Back in 2010, during the early days of bwapi, there was a Brood War AI tournament held by the Expressive Intelligence Studio at UC Santa Cruz. It's interesting to see how different the approaches were back then, vs this or Deepmind's SC2 work.

https://web.archive.org/web/20091124210529/http://eis.ucsc.e...

There's a great contemporary Ars Technica piece by a competitor:

https://arstechnica.com/gaming/2011/01/skynet-meets-the-swar...

As an undergrad I did a project using genetic programming. It was not very successful, but it was a lot of fun.

https://tomisin.space/archive/starcraft-genetic-programming/

tinco11 days ago
All the hobbyists were using proxybot because it allowed you to use more fun languages than C/C++ but proxybot lacked the features to effectively play Zerg. I really wanted to play Zerg so I built a really sweet API+DSL in Ruby around proxybot and then used that to give the other newbies a hard time with a zergling rush.

Unfortunately the proxybot limitations precluded me from expanding its capabilities so I tried to switch to having ruby embedded in C++ and I basically got mired there and was eventually distracted by real world concerns like actually finishing my degree.

I think I played against Krasi0's bot a couple times in the early days. Hopefully they stuck with AI and have suddenly become crazy rich after 2016. It certainly wasn't a given that AI was going to lead to a fruitful career back then, let alone to unimaginable riches.

mslate11 days ago
I also entered the competition as an undergrad—I contacted Ben Weber 10 years later (2020) and interviewed him on my podcast:

https://theaccidentalengineer.com/adversarial-machine-learni...

dcl11 days ago
In the early days of SC2, I remember people using genetric programming to optimize build orders. I remember a slightly unorthodox Zerg Roach Rush which was _really_ fast.
suby11 days ago
I don't know where else to write this, but I want to throw the idea out there. I have long wanted to take old broodwar televised matches, many of which are terrible quality 240p, and use machine learning to convert them to into perfect Broodwar Remastered frames. This seems tractable to me because you should be able to map the terrain sets to their remastered equivalents, and the game is just a series of sprites rendered at specific frames. Even if the source quality is terrible, I imagine this is able to be extracted at high quality since you can, eg, set up an automated pipeline which generates training data. Maps from original graphics to remastered, and then again for 240p -> tilemap positions for frame camera center + sprite positions / animation index.
mountainplus10 days ago
OSN did it on their youtube for their old OSL series, I think it can be done better but I think original still is much better to look at, at a corresponding resolution.

The key expression for playlists or yt browsing is this `AI 업스케일`

f.e. https://www.youtube.com/playlist?list=PLWQeRMoEALvp75uolIpIE...

efxhoy11 days ago
I would love all the NukeTheStars matches in 4k, his commentary was fantastic
navane11 days ago
Kespa or Gomtv remastered/upscaled their old televised games about a year ago. Just using the upscalers that were the hype at the time.
KeplerBoy11 days ago
Maybe one could try to reconstruct very closely matched replay files?
3eb7988a166310 days ago
Do replay files require consistency or can you spawn new units from the ether as required? I think you could get a system that would get the broad strokes unit placement correct, but having a shot-for-shot perfect replication seems impossible.
duskwuff10 days ago
Not feasible. The VODs are from a spectator's perspective; there's a lot of action happening "off camera" (like players building units) which affects what's seen on camera later, and which would be extremely difficult to reconstruct from what's visible.
silentkat11 days ago
Link to the matches?
pelagicAustral11 days ago
Unrelated to the benchmark...

I love StarCraft. I started playing it right from the beginning, most of my friends right now are from that era. I literally met people that have spread to almost every continent when I was in my early teens. We played at internet cafes and did not have access to the internet, that was priced differently...

I miss those days so much.

Everybody was from a different background back then, and nobody was anything other than a guy that plays StaCraft at the cybercafe... And now, we are in our 40's and I know Math teachers, history teachers, oil rig operators, software programmers, professional gamers, lawyers and more... hahah So crazy to think about it... and I know them, we talk, what a world.

nemo161811 days ago
Even at Burning Man, in the middle of the desert, there is a camp that hosts a StarCraft tournament every year (on the dustiest setups you've ever seen!) :)
jpgvm11 days ago
I hope it's called Mar Sara.
PorciiVorbesc11 days ago
So ... Zerging Man?
benswerd11 days ago
So dope
snicky11 days ago
Oh boy, I've spent more hours playing it than I dare to admit. I won over 10000 battle.net games ... on just one of my several accounts :) When the SC2 beta came out, I played about 50-100 games and never bought the full game, because I knew it would be like heroin to me, and I was already an adult that had to take care of himself.
ddlsmurf11 days ago
I did buy it, all three, loved it. But somehow, broodwar is the one coming back on the scene more than it
layoric11 days ago
Same and Brood War is starting to have a bit of a resurgence. Not anything huge. When Battle.net servers are actually working, tend to play on the ladder a few times a week. Get crushed, but still one of the greatest competitive games ever made.
kentosi10 days ago
How interesting. I too played extensively for hours on end. I was strictly a Terran players, having been inspired by Slayers_Boxer and others at that time. I made "friends" online across my country and maybe 1 or 2 others, but these were move online buddies to play with with side conversations.

We were all teenagers having fun though. The friendships you speak of never really took place for me. I wish they did, because it was always jarring to unplug and get back to the real world (school, uni, work) and associate with people who had no idea of my other parallel life online.

benswerd11 days ago
I agree.

My first time playing StarCraft was at summer camp around a decade after it came out.

All the smartest people played it so I wanted to too. Great decision, I have been continually impressed with the people who StarCraft introduced me to.

faeyanpiraat11 days ago
There is currently a bot beating everyone on the ladder. Just watched it today on Artosiscasts yt channel.
ericpruitt11 days ago
Link to the video for the curious: https://www.youtube.com/watch?v=1vsTqNwHquE . I would note the bot is not APM limited, and comments suggest (https://www.youtube.com/watch?v=1vsTqNwHquE&lc=UgxBrY4CkuZx7... , https://www.youtube.com/watch?v=1vsTqNwHquE&lc=UgyZ0XVFDqzJT...) it's likely using map hacks, both of which make the bot significantly less impressive.
vitaflo11 days ago
If you read the TL thread on this, the bot is technically running at 240 APM, but its commands get split off into separate commands for each unit, so the APM looks higher.

I've watched several replays on this. It using map hacks is lame but it also doesn't always take advantage of them. It sometimes does respect its own fog of war. It mostly wins with incredible micro (kinda has to, it's macro kinda sucks).

Most of its few losses come from drops (it never makes turrets and doesn't know how to handle them), bad macro (blocking its own ramp) or just incredibly unconventional play from its opponent (which is certainly not in its training set).

iammjm11 days ago
It is in fact APM limited, and its creator claims it was trained without the map hack so it should do just as well without it. I think the bot is VERY impressive and has already managed to beat several top Korean pros which is really godlike
readams11 days ago
Bots have been unable to beat even pretty good amateur humans despite those advantages so it's still pretty interesting.
mcteamster11 days ago
I love this. Funnily enough StarCraft has influenced how I approach AI at a meta level

Protoss: powerful and expensive frontier coding agents you directly micromanage for the toughest tasks

Terran: versatile team comps of dedicated agent roles you can delegate well-defined tasks to

Zerg: massive swarms of specialist custom agents inside your apps that you evolve and optimise for speed and cost

Knowing every faction has its strengths and weaknesses helps me decide which tools to use for the job.

alchemism11 days ago
On a tangential note, try run Vibe Island (or equivalent) with Protoss WAVs for the agent sound effects.
mcteamster11 days ago
Committed to the bit; context aliased “change plans” to “pylons” and I have a command to /construct them

Read the full thread on Hacker News →

Related stories