Main idea was, if LEGO parts don't cover the needs for an agent to build a model, let it use the best it can find, and manufacture others if needed, but keeping same compatibility with the rest of the catalog.

And for that, instead of generating new parts from scratch, maybe mutating pre-existing parts into new shapes it needs, but leaving connectors as they were, or even adding new connectors but still compatible.

I'm taking a look at this paper right now, didn't came across it before.

I think it's really good, and also an approach I tried.

In my case, it was working pretty well in terms of geometry and correctness, but not in terms of creativity: too many constraints and an agentic LLM would end up building very similar buildings all the time. Too few ones, and it would drift into unbuildable models.

In the end, I think the current iterative process gives it a good balance: let it try things first and be creative, parts will be placed incorrectly at the beginning, but it will fix their placements, colours and so on bit by bit, until the end result is (hopefully!) correct.

And regarding Opus, according to my experience, I'd say definitely models quality is way better when produced by Opus 5 and even better by Opus 5.5

I've been experimenting with FreeCAD and freecad-mcp too, even managed for Claude to create a new LDraw benchmark for it.

However, given the amount of parts some LDraw models have, FreeCAD performance would not be enough, something to do with its solver trying to work out too many constraints at once.

The thing about how to assemble the parts, yes, that drove me crazy for quite a while.

A way I found that helps is to provide the agents information about contacts, e.g. what parts of a brick are actually connectors.

With that in place, it can create tools to automatically check if two parts are connected "the right way" ;-)

I got LEGO connectors info from here: https://www.melkert.net/LDCad/tech/shadowLib

Yup. I’ve been getting some pretty decent 3d models directly on stl format + the slicer file with all the right configs for my printer

It’s cool to create a simulation, run some visualizations on web directly from Claude desktop, and then choosing a model to print and build

I also added some genetic algorithms to make it fun iterating on different shapes/mechanisms

I really enjoy how much creativity these tools are unlocking by enabling bridging so many elements together

On that note- have you considered adding constraints for part availability? It would be useful to be able to create designs using only parts which are on hand.

I recall seeing a project a while ago which could take a photo of a pile of lego as input and produce models that could be made with those parts. Although I think in that case the models were sorta just random arrangements.

I bought so many sets, and I want to rebuild them, but they're a whole big mess. I want to sort them out.

Makes finding pieces for new builds much easier too