> Great that everyone’s posting their “my secret sauce” cookbooks just to jump on the hype train but it looks like the models have already figured it out for themselves

Unfortunate lesson to be learned here: there's not much leverage here other than just using AI. Previously, us devs could get a head start and build some institutional knowledge but not this time. I'm bearish on all the custom harnesses stuff that people talk about.

What helps me is to understand the failure modes of LLMs - it can't be articulated in easy words but something you can learn slightly by just using it. For example I have an intuition of when to start compacting but Codex already does it for you now haha.

My take: the highest leverage move for us is to write AGENTS.md and provide everything that the model can't learn on its own or might take time to learn.

Yes it’ll eventually get built into the harness/model/interface; just like how “work it out step by step” became thinking, but if you have work to do today, it can be worth trying to improve it.

i've been working on my own toy "software factory" concept (roughly 300k backend and 100k frontend loc across all components of the platform as of today). I've been building it to ultimately run it on my homeserver - its strongly focused on maintenance tasks like automatic library updates and rebuilding base images for deployment (and actual deployment), etc pp.

My explicit goal of the project was to replace my currently manually managed TrueNAS Scale installation, on which i'm currently (more or less) manually managing the lifecycle of various self hosted applications.

So i'm expecting the factory to eg versionbump dependencies, read changelogs, copy the data of the selfhosted applications to prerun the migrations and verify the migration does not encounter issues. Or similarly for my own written software, eg if the framework i'm using in a project has released a new major version... same route, read changelogs, trial migration etc.

While I would be able to build such a system on my own, realistically speaking I would never because the needed time to actually do it far exceeds the amount of time I have available next to my full-time dev job.

So I've basically decided to go with the llm-driven development flow, where I still have a mental model of the system to the bone/internals, and explicitly state how it should be developed. It usually takes roughly 15-30 minutes to properly scope a development, which then takes agents 4+ hours to implement and fully e2e test on the dev platform. eg. yesterday evening i decided to merge the Ticket and Epic/Story/Task concepts into a unified model / same entity table. I scoped it yesterday afternoon, sent of the process to develop it in the evening and it got finished around 4am this morning. (And it totally wasted multiple hours waiting on events to occur which it filtered out via a shell pipe, sigh)

My mindset building it is basically the same as i tread a Factorio, DSP or Satisfactory game: there is some jank to it for sure, but if the process keeps going, its fine. And its kinda fun to see things getting more and more streamlined over the months.

I dont think i'd be comfortable treating my dayjobs codebase like that, however. To a large part because i'm missing essential QA that I automated in my own project, eg you cannot have any unattended mutations unless theyre easily revertable. At my dayjob, if I merge and deploy a faulty commit, i _will_ impact others. When the same happens on my personal project ... some agents may be inconvenienced, which is not particularly high on my priority list.