If we think writing code is dead, and AI is generating all codebases, I still think the bigger problem is people or full teams not knowing anything anymore about the system...

384 points•zazuke•2 days ago•239 comments•

239 comments

zero_shift2 days ago
I was reflecting on this on Saturday in an unformed way, trying to trace the lineage of a decision made at work.

The code change itself doesn't specifically matter. But suffice to say, it was about an AI feature in one of our products.

The code was stamped by Claude driven by a prompt. The prompt was for a ticket generated with the Atlassian AI integration. Atlassian had digested docs made with AI. The docs came from strategy memos I'm 90% sure were written entirely by Claude.

The strategy was chosen by management at the urging of exec leadership. The execs now communicate mostly via AI written memos. I do not know how they make decisions, but they reference tech influencers, market conditions, customer expectations.

This gave me pause. Who had actually made the decision then? Arguably there has been several layers of human review, but the actual source of the decision was hard to pin down.

We were not building the feature because we wanted it. We were building it because we thought other people expected it.

Perhaps reflecting on the state of the market, I thought, could indicate who was actually in control.

Where do investor and customer expectations come from in 2026? It is very murky, at least in tech. There appears to be hype. Some hype comes from true believers, some comes from cynics. But both respond to market incentives that reward bigger and bigger claims.

Where does the market's "action" come from? What is the driver?

Investors do not really seem to understand what the tech is or its limitations. Some are passive operators. Others are just responding to the overall froth and speculation in the market - which becomes a runaway feedback cycle.

This left me lost.

Nobody in this ecosystem, I thought, is actually in control here.

Nobody is actually orienting work and action to real, concrete goals. It's all based on speculation and anxiety about the future.

So it is not only that nobody understands what the code does. It is that we cannot, or at least I cannot, explain the motivation. There doesn't seem to "be" any form of "intention" in this environment.

It has all been hollowed out, replaced either be inscrutable machines, or inscrutable incentives.

Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.

danielmarkbruce2 days ago
In many (most perhaps, but hard to know personally) companies, most people don't really understand the market they are in, the competition, their own products, the customers etc well enough to actually make good decisions. They also aren't likely to be around (or held responsible) for decisions as they play out over years. So what has historically happened is that people use proxies for good decisions and understanding - which are clever sounding documents and presentations.

This is a long winded way of saying "people made up clever/sensible sounding stuff". Now it's easier to do it with AI so the problem is worse. However, I'm not sure what you were looking for was ever really there - the "inscrutable machines" and "inscrutable incentives" were always quite inscrutable.

Supermancho2 days ago
> most people don't really understand the market they are in, the competition, their own products, the customers etc well enough to actually make good decisions

"Product" studies this data and tells engineers what features they want. I am rarely instructed to A/B anything...excepting when it's defensive, to ensure the first rule of "don't interrupt the flow of money" is upheld, if at all. Most of the features are obvious improvements anyway (determined from plain conceptual planning, manual testing, and personal usage).

ofc I don't understand the customer or market or anything else. I'm paid with the expectation that I'm not going to share an opinion about it, especially since am never exposed to the raw data and inner circle decisioning except during a quarterly meeting...maybe. This is part of why AI is so successful. Humans in large organizations are specialized with little creative input and lots of mechanical process. Coding is largely a mechanical black box (turing machine) from the outside looking in. It works seamlessly because I don't need to know about the things product wants, so the AI doesn't know and everyone carries on faster than we were before.

ryaniscool2 days ago
Yes, people can be bad at their jobs. Yes, they can work in an industry and not know anything about the market but I refuse to believe that most people in most companies are just stumbling around faking it. I've never worked in a place like that. People like this exist but there are only a handful of people like this at every job.

The hyperbole feels like it's been contrived to fit into the classic AI counterpoint: But humans do this too!

tarsinge2 days ago
From my experience in big and medium sized companies or even startups I would say that this was already the state of things before AI. Most leaders and top management were following internal committees, that were following consultants recommendations, that were themselves recommending what others were doing and was hype and/or following Gartner like "studies". The motivation was equally dubious, and many developers already had problem with management and product direction, but many didn’t bother to question it. AI just makes it obvious.
epgui2 days ago
To put a finger on a word: David Hume’s “is vs ought” problem.

Data can describe to you what exists. But it can’t tell you what you value.

What you describe is people who can’t tell the difference, and who let the machine (data) make the value judgments.

xhevahir2 days ago
I don't think the problem OP has is anything so metaphysical as the fact/value distinction. This is a business, after all. It has goals (e.g., making money) that the model surely can grasp. It sounds to me more like a breakdown of organizational control owing to a lack of transparency in the tools and a general ignorance among the management.
godwinson__4-82 days ago
Imo, most companies ought not to exist.

The parent comment is interesting, but ultimately I think in this case, the AI is actually revealing something about the true nature about their place of employment, a nature that has always been there versus some mutation caused by the prevalence of the AI itself.

A lot of money can be made purely algorithmically. Think market markers or other algorithmic trading. The ought vs is divide is quite narrow here. It's not a moral question, the "value" is in the money to be made. It's actually not a great example to invoke Hume's problem. Many companies essentially are chasing a similar spread, its just less obvious. Few people ever ask what "ought" to exist. If the power of AI makes more businesses operate more reactively and algorithmically, because of more data or processing power or w/e that really is probably in keeping with their alignment and goals. Because the ultimate ought for a company is we ought to be making more money. So, in many ways the ought is really not that interesting, the is is satisfactory provided the return on whatever their version of a spread is keeps improving.

The number of companies that actually "invent" useful things and thus ask even vaguely meaningful "ought" questions are extremely slim. The vast majority of employees are, at best, accessories to these questions, even in software where even before AI many of us were not doing very interesting work. There is a lot of essentially rebuilding your competitors same layers on top of common libraries and standards where the actual interesting work is done. Really not unlike asking AI to cook you up a boilerplate by leveraging the vast work of a fraction of SWEs who maintain OSS tools. It's the same pattern and the same sort of behavior, just now your "layering" is becoming automated to the point of irrelevance.

What the parent misses - the real promise of AI is paradoxically, that it will allow more people to ask actually interesting ought questions as AI owns more of the spreads. In the same way a human does not compete with an algorithmic trader, and at some level, really doesn't care. The more algorithmic your business becomes, the less any individual human "value judgement" matters. And really this is desirable, because again, most companies are not asking interesting value questions anyway. The end state of this you are missing is these companies are going to cease to exist. In the optimistic case this will free you up to ask more interesting value questions - like how do I value all my UBI enabled free time. In the less optimistic case your value judgments will be more dire - like who do I sacrifice given the Terminators are at the door and we only have x quantity of supplies left.

This is the other paradox. When questions of what ought to happen are of paramount importance, you are probably finding yourself in a very undesirable situation. It's easy to valorize the ought problem from a distance, it is much much harder to actually engage with it when it actually matters. In many ways, the relative luxuries of society and civilization are derived from taking such questions out of most of our hands. This is (perhaps surprisingly) true even as you climb the ladder of power:

I used to think that if there was reincarnation, I wanted to come back as the President or the Pope or as a .400 baseball hitter. But now I would like to come back as the bond market.

BOOSTERHIDROGEN2 days ago
For me, at this company, which is struggling to develop a new value proposition, everyone is unfortunately using AI in a blatant way. It’s even worse when executives and management pressure us to move fast. Presentations, documents, and mockups all use AI, rinse and repeat, feeding it context, but somehow, nothing is moving. It’s just staying static.
RGS18112 days ago
The word I have for this is “commitment laundering”. People pass around AI artifacts that nobody has necessarily read or considered, and the invented assumptions and tagalong commitments just keep piling up. They look like they’ve got real provenance but nobody can say what is being done or why.
saltcured2 days ago
Right up there with responsibility/culpability/liability laundering. These are things being shrugged off and externalized.

And the complement is credit/provenance laundering. These are things being misappropriated.

The grift often does both of these with the same sleight of hand, and this is what gets accelerated with the new tools and cavalier culture around everything.

glouwbug2 days ago
When you write something you constantly remodel your understanding through refactors and rewrites until you internalize it. By internalizing it you gain the capacity to reason about it (during critical downtime) and communicate it. An entire team that can communicate can solve problems together, from one guy's vision to products white boarding to engineering's infrastructure to UX and UI's artistry.

It boggles me we completely forgot that the world operated like this just 4 years ago

breadzeppelin__2 days ago
I feel this in my bones. My team is one of the most communicative in our org and have garnered a widely positive reputation I think largely as a result of our coming together every day to solve problems. Trying to get information out of the less communicative groups is like pulling teeth, i wonder how they get anything done at all
poisonborz2 days ago
Assuming we don't speak about some vibed "one-shot-deploy" situation (for which there were options in the old world as well, with the same bad results), in a well disciplined team, why would AI make this worse?

You can still write anything by doing it either small steps, or at once followed by a lot of refactors while skimming over code and asking tons of questions / making refinements via prompts, guidelines, test guardrails. A team can still reason and whitepaper about the same things. Devs who were previously shy to ask some specific details (to not seem dumb) can now confidently survey big codebases and get insights in whatever style they can swallow.

The bigger problem I see is that all this requires a lot of communication, and most importantly writing skills, something that disappears in thin air in the last decades.

AppleBananaPie2 days ago
My experience is the AI has somehow thrown discipline and accountability out the door.

Folks break everything all the time and looking at their PRs and messages are clearly just having the AI respond to everything for them and actually have no idea what they're doing.

And then other people doing the same thing approve their work (presumably not reading or understanding any of it) because then the other person will do the same for theirs.

Hopefully my experience is an anomaly though :)

binarin2 days ago
There are still some places that operate that way, as outlined in this Jane Street talk https://www.youtube.com/watch?v=zR9PpXWsKFQ (Production Engineering When Trading Billions of Dollars a Day).

> capacity to reason about it (during critical downtime) and communicate it

That's one of the things that's put into limelight in that talk.

I personally think that most of "Web Scale" software is inconsequential (inconsequential for its creators, not for users) - as there are no consequences for bugs and outages at all. A data breach -> slap on a wrist. Reputational damage because on an outage/data loss amidst general public? - almost impossible (clownstrike, anyone?)

The funny thing is that web scale software that's consequential is often in an ethically grey zone - but at least you won't be surrounded by colleagues who don't give a shit.

desmondl2 days ago
> When you write something you constantly remodel your understanding through refactors and rewrites until you internalize it.

Unfortunately I learned that not everybody thinks this way. Some orgs do imperfectly fine without good engineering discipline, and that has been the case before AI... AI has only made it easier to give the appearance of good engineering, which is exactly the pre-AI goal of many orgs

cmrdporcupine2 days ago
I am simultaneously deep down in the agentic treadmill and also supremely frustrated by this.

And I don't think it was ever necessary to go to the point where people just gave up authorship. These were choices made by adopting the "I'll do everything for you" agentic "harness" model that shipped with Claude Code but it was never inevitable.

e.g. we completely dropped fill-in-the-middle completion OG CoPilot auto completion model. That combined AI authorship with a human always in the mix and I actually really enjoyed it. It's just that the models involved were pretty stupid. We totally could have had IDE / shell / tooling integration that kept people in the driver's seat while automating parts of the drudgery away. Instead what we got was a simple chat loop with "oh, whatever, you go do it" being the ultimate result. Cuz, you'll totally review everything after and understand it, right?

The things should end by quizzing you on what was just made and if you don't pass, just throw it away. That'd be funny to watch.

dannyw2 days ago
Our brains are hard-wired to enjoy (at least in the short term) immediate gratification, and the removal of friction. It's a hard battle to fight.
MomsAVoxell2 days ago
I am finding this, professionally, not so dramatic - but mostly because the industries in which I've consistently shipped code at scale involve review as a first principle, as in no un-tested, un-reviewed code gets shipped, because: safety critical/realtime requirements and certification specs, say so.

And having AI code to review is no different than any other code that ever was to review, so the review tooling is - as it necessitates - also benefited by lugubrious application of .. more AI. But: all AI is human reviewed.

So it's not a big impact. We just don't ship code that isn't 100% human reviewed, If that's insurmountable: you're doing it wrong. Use AI to make code readable again.

And then, also, put AI back in its box. Don't give devs 100% full-time API access to subscriptions: give them actual hardware to use, to go 100% local.

Local AI is, thus, the best AI, folks. Don't use more than you can run locally, is a great way to keep AI code properly maintainable.

The industry will prove this, itself, sooner or later: If you can't put your AI in its box for safe-keeping, you're doing it wrong, anyway... and should've already learned this practice as a habit, decades ago, vis a vis future-proof tooling... (See also: not logging everything you do with an AI? Big fail.)

Sure, the absolutely intoxicating addiction of Big Metal AI™ is going to put a lot of consumers in a deep, deep pit of Neo-Illiteracy - however: 'good' AI code is actually just good code.

nicoburns2 days ago
> I am finding this, professionally, not so dramatic - but mostly because the industries in which I've consistently shipped code at scale involve review as a first principle, as in no un-tested, un-reviewed code gets shipped,

> And having AI code to review is no different than any other code that ever was to review

If your company is sticking to "everything needs (human) review" then you wont run into most of this. This issue is that a lot of companies are using AI as an excuse to remove that review process (either partially or entirely).

MomsAVoxell2 days ago
Yes, that is very real to me, the AI "revolution" is more of a battle between traditional devs trying to replace their managers with AI, and professional managers trying to get rid of the devs the same way. There is a wide degree and scope of knicker-twister'ing in these trenches.

Everyone can write code these days. Trouble is, all code has to be SIL-4 code now, because, human, you will never know if your compiler trusts your AI until you trust your compiler. This rule will be true for decades into the future, I'm willing to wager...

But, ultimately, software has to follow certain rules, or it just doesn't work. Proper workflows - involving review - are needed. Because security is pretty much over, otherwise.

Folks are finding it easier to make their own software now, too - rather than use others. I predict an end to the app stores - or at least, the primary interface is going to end up being "describe the app you want to use today" instead of picking words from a list ..

jefftk2 days ago
> And then, also, put AI back in its box. Don't give devs 100% full-time API access to subscriptions: give them actual hardware to use, to go 100% local. Local AI is, thus, the best AI, folks. Don't use more than you can run locally, is a great way to keep AI code properly maintainable.

This glosses over the large capability app between local options and frontier options. Which applies in many ways, but includes drafting more maintainable code. I would definitely not want to limit my devs to what we can run on our own hardware.

joe_the_user2 days ago
The thing is, fully-test-your-code wasn't economical for software houses before the rise of AI. Perhaps it will become economical now but I'm not sure what would force the issue.
bengold142 days ago
I see this everyday. The problem is code is the wrong abstraction for the work we do. LLMs have solved coding, but they haven't solved systems, collaboration or system maintenance.

Edit: Since I seem to have touched a nerve - I've been working on a project to solve this: https://www.archme.io if you want to know my thoughts on the right abstraction

tripleee2 days ago
Outsourcing solved coding a long time ago. You can go on Fiverr and prompt a real human developer for $2/hr - even cheaper than LLMs!
imhoguy2 days ago
Wow, I think you have never had a chance to communicate with someone selling their expertise for $2/h. Unless your time is also worth $2/h, I think you would get better results prompting local 27B model.
bengold142 days ago
Same problem though :). If an overseas employee knows the code and you don't, you have ceded ownership, problem solving ability & future direction of the project.
verdverm2 days ago
When you say "solved coding," what does this mean, what does it look like?

I have strong disagreement because it sounds like, by analogy or proxy, we have also "solved writing"

yetanotherjosh2 days ago
I would say it means you are building software but no longer are concerned with writing or editing literal code. Your concerns have moved up the stack to managing requirements, context, and verification processes.

Just to make this clear: if you can define a really good PRD and sophisticated technical specs, and a strong set of tests cases to pass, at the right level of architectural granularity, plus adversarial code review processes that triangulate and weed out most mistakes, SOTA agents can write the code autonomously, at or above the quality level of most human coding teams. I call that "solved" but only if you meet those context requirements. Which is still hard, not solved, at that layer.

Solving writing is not a good analogy IMO. Writing is for human consumption, and cannot be wrapped in objective requirements and verification processes. Certain forms of writing perhaps could be (can't think of one at the moment but I don't doubt some exist), and those forms might be good analogies for being "solvable" or "solved."

glimshe2 days ago
"solved coding" is type of thing you say if you want to sound smart.

Saying that LLMs have "reduced the cost of coding" would be boring. And using your analogy, pencils, typewriters and computers have all reduced the cost of writing, but writers are still around.

KeplerBoy2 days ago
It solved coding as in it's significantly less of a bottleneck than it was before.

At least that's how I experience it. In the before times each non-trivial code change had a real opportunity cost as it would easily consume two days until I could even estimate whether this is worth looking deeper into.

sampullman2 days ago
Writing is a means of expression and communication. Code can be those things, but that's not its primary purpose.

I think "solved coding" is taking it too far, but for many projects, the mechanical aspect of it has been removed or reduced greatly.

LLMs will have a much harder time "solving writing", because they cannot develop their own style and so are severely limited, creatively. This is less important for coding.

mwillis2 days ago
maybe it’s more like “abstracted” away, in the same way higher-level languages “solved” needing to code via machine instruction sets. Higher level languages allowed humans to think more like themselves. AI puts another abstraction in front of the outcome, making it even more generally open to human thinking and less defined by the need to give machines exactly what they expect.

Today, human-language outlines / briefs / prompts are “compiled” to code which is itself then adapted to hardware. We are stretching less and less across the divide, doing less and less work on the terms of the machine. Now the farthest we’ll stretch is often formatted markdown - the most basic application of machine-parseable structure to very organic human thinking. Because we’re given the chance to be less precise, coherence suffers.

Avicebron2 days ago
Or more broadly, LLMs fundamentally don't "understand". They can simulate understanding and generate text/code/whatever, but they don't have will to engage with something holistically and "own" it.
visarga2 days ago
I have been trying to define "understanding". Is it when you can predict something that you understood it? Or maybe when you can explain it? Or how about when you can control it? Or invent it.

This time I add another definition "when you can own it".

Terr_2 days ago
I find it helps to imagine what is/isn't solved (however, we choose to label it) by an indefinite number of cheap junior-developers.

Except a little worse, since they were raised alone in a library, act mostly the same, and have harsh limits on personal growth.

bengold142 days ago
tbh, I don't think of LLMs as junior-devs. Maybe 6 months ago, but now they are v. senior code-monkeys. They are masters of their craft, but their craft is narrow and lacks big picture, collaboration, org goals, etc.
jplusequalt2 days ago
>The problem is code is the wrong abstraction for the work we do

My team recently spent two weeks on a wild goose chase trying to figure out why TensorFlow Lite was generating nonsensical OpenCL kernels. Well it turns out that LLVM had a few bugs in the RISC-V assembly for our platform that was leading to silent garbage. It took combing through assembly dumps, hexdumps, a lot of pain staking debugging, and going through the TensorFlow Lite source code to to track this down.

In your opinion, if code is the wrong abstraction to be working at, how do you approach this scenario?

bengold142 days ago
Fair question - IMO it's the wrong abstraction for building and collaborating on a new product with a team.

To your point, it's not the wrong abstraction for solving code level bugs. Just like python is not the right abstraction for solving memory corruption or pointer mis-alignments.

vld_chk2 days ago
I feel related to it. I switched companies at the end of July. First 4 weeks AI was giving me the feeling of freedom. No longer I need to understand tens thousand of lines of legacy codebases. Never onboarding was so easy. Just ask Claude and it tells me what happens here and how.

But after 2 months it starts to backfire me. I still know nothing. I have some understanding of the system design and core components but I have zero clue about how certain things are done under the hood. Because AI read code for me and code for me and I take it as my own understanding.

In last week I end up limiting my AI usage and forcing myself (it is really hard) to read and code at least a bit by myself to start having any idea about what is going on here.

Read the full thread on Hacker News →

Related stories