If an AI can replace me on the mental aspects of work, and robotics are on their way to replacing humanity on the physical aspects of work... then what's left? When there was agrarian societies, there were writers, priests, bankers, merchants, and laborers before and after - I really don't think things were that unclear even at the time. Now that we have machines that are close to exceeding humans in every way, what good are humans?

In my view that's a very charitable reading and sadly I don't think it aligns with the historical record. When the cotton gin was invented, there was a hope it would lead to a reduction in slavery. But of course it increased the demand for slaves since more cotton could now be processed, making cotton much more profitable. Slavery ended in the United States because a war was fought, not because of automation.

There had always been abolitionists who were against slavery simply because they thought it was immoral, not because they thought slaves weren’t needed anymore.

Or will it be the cumulative total of various advances?

I've equated Claude Code, or Codex, to the looms that made fine fabric more affordable during the Industrial Revolution; life-changing, but not society-changing. Neither the steam engine nor the automobile.

Perhaps I've answered my own question in that it's the LLM technology itself that equates to the steam engine, and it will power superfarm analogs that have yet to emerge. I'm still curious what you think they will be.

Maybe a few thousand people for personal services of... various kinds. But no one's going to need the rest.

It's the ultimate capitalist fantasy.

And of course it won't happen, because long before things get to that stage AI will have independent plans of its own.

(Which is just as well, because if things did get to that stage the emperors would all wage war on each other rather than living peacefully and productively.)

I don't think we can imagine a post-ASI culture because - by definition - we're not smart or inventive enough.

It's not just farmers -> superfarms. Although in fact that did happen, but largely as a footnote to developments elsewhere.

It's more to do with the fact that our visions of the future haven't changed for over a century. They've been implemented in unexpected ways, and there have been unexpected social and cultural changes. But you can easily see the outlines of modern technology as far back as the late 19th century.

With ASI, the outcome could easily be something that doesn't look and act like technology at all. It would be some unimaginable New Thing. Literally no one on Earth has any idea what that would be or whether there would be room for trad-humans in it.

> A black box oracle that just tells you whether statements are true or false is not the goal of mathematics and would not be particularly interesting to the field

This is a crude distortion. The recent breakthroughs have come with proofs, reasoning and verification, and there is no proposal that I'm aware of that would do away with these foundations. There's also some rather ugly solipsism in the idea of keeping what interests the field as a limit. Mathematics has broader relevance to humanity than merely to please and support mathematicians, and if other fields can make practical use of profound well-proven future math, mathematicians will have a hard time making a case that their comprehension must come first.

Of course, math research is cheap and most academics don’t rely upon grants, their salary covers most of their expenses. But here too, the mathematics professor spends a substantial amount of their time teaching future engineers/quants/other applied mathematicians, who need to understand math for instrumental purposes, not as an end in and of itself. Without the tuitions of these students, I can’t imagine universities maintaining the size of their math departments, let alone expanding them as Dr. Sahai advocates for.

So who or what funds the community of pure mathematics going forward?

Thanks for providing a (much needed!) correction.

The average person doesn't know how the medication they take works, the mechanics of climate and climate change, how the energy they consume is generated, etc.

> Isn't it fun to imagine how life would look like in that scenario?

This is horrifying to me.

I think there's a lot of sci-fi out there that already did. Maybe it's not utopian because a pure utopia would not be likely to have an interesting story, but on the other hand, most huge technological advancements end up having just as much potential to reinforce existing power imbalances in society rather than solve them. It's not obvious to me that if we got magic super AI that can solve every scientific problem in society that it gets used in pretty much the same way as anything else: making the people who control it a lot of money rather than sharing the power with everyone without charging them.

I can almost guarantee you the first time you show cancer symptoms, you won't care whether the cure came from an AI or human's understanding. But we haven't seen that, so we can't make the judgement call.

For some people, fun is doing physics and mathematics. So they are going to keep doing that.

This is, basically, 100% of the thing. We will never get to the level of automation some folks think for this exact reason.

I’m not sure what to think about an analysis written by someone who didn’t catch THAT.

There is tons of evidence against that. Someone armed with just LLM, can have much better uderstanding of problem then "meat brain expert" who studied the subject for decades.

We seen in last decades how "experts" are clueless, and how they predictions work.

> don’t really care whether LLM’s can produce code more and better than me

So on one side there is monopoly on "deep understanding", but on other no one really cares about quality?

I had to catch a stray cat recently and take it to the vet for an infection. It's healthy and spry now but ever since then it's been deathly afraid of me.

For the collective (humanity, mathematical/scientific community etc). Math is not done in isolation, and if somebody does so then feedback to the community does not work as well.

Mathematics, and basic science to a degree, face the issue that they create their own problems and paths through this kind of tranformation, where external feedback is secondary. It is not as if "I want to build an app/car/robot, I let AI do it". It is as if you decide to let AI decide what to build for you and how to build it, and you do nothing at all. Instead, the pursuit of understanding is the goal itself, and through the course of humanity we have learnt that this understanding can also be useful, but this is not necessarily guiding how this understanding is gained.

Most of the contexts people here have in mind are when problems are well and externally defined. Cure a disease, optimise an engine, make an app that does X, etc. This is not exactly the case in theoretical math and never was really.

I have dyscalculia so the math world is dead to me, I can't even add 2 numbers together in my head, but I've read here on HN many times over the years people describe some version of new maths frameworks tangibly changing how they view the world. For me, I went to film school - I have a natural ability to think in movies and pictures, memory is very visual for me, before film school I had a good sense of shape, colour, light. After film school and to this day some 20 plus years later, I mostly look at the world starting with the shadows. Russian speakers, whose language has separate basic words for light blue and dark blue, are slightly faster at telling those shades apart. Speakers of languages that use compass directions instead of "left/right" appear to develop a strong, constant sense of orientation.

These things can only happen through the process, so for who? For people who want to live richer lives, want to think differently, and for you, to be around people who have such.

For math, the value of proving a theorem is often not proving the theorem (which most people already believe correctly to be true or false), but the path taken there, the new math invented, and how it can be applied to other problems. The end result is therefore almost inconsequential in moving math forwards.

At least historically, we haven’t really been able to tell what will be transformative in terms of real world applications ahead of time.

We are destroying the planet and ourselves. There are a few things which we do that are just good in of themselves, which can inspire and proof that there is good. Friendly competition, exploration, art and understanding the world around us.

What world do we want to live in? And what are the kind of personalities that would be required for such a world? A thousand Picards or a thousand Musks?

Most pure mathematicians are no more interested in ‘producing commodities’ than any other academic is. That’s not what the subject is about — at all. The confusion arises because mathematics turns out to be extremely useful (no surprise; it’s quite useful to have a detailed understanding of the basic principles of reality).

Again, this seems to be an alarmingly common fallacy here on HN. As a commenter above observed, pure mathematics (and that is what we’re talking about here) is in important ways closer to the humanities than it is to other sciences.

For the upper crust of our society, work is entirely optional. Presumably people therefore continue to work for the process or the reward of the doing.

Even before the AI era, we were flooded with a veritable DDOS attack of slop, in our in-boxes, meetings, etc. Managers believed that innovation was held back by quantity of ideas, or by the domain experts being "resistant to change." This led to day-long group brainstorming sessions that yielded nothing. Part of the transition from student to junior to senior is learning to filter your own ideas, and to guide others in the search for ideas that are likely to be fruitful.

The criteria for slop filtering have always been subjective, arbitrary, or driven by institutional culture. That's what we've got. Any means of generating ideas produces slop in the absence of filtering.

What will slop filtering look like when the AI can generate tera-proofs per second? What does it look like already for our e-mail inboxes? We are asymptotically trending towards slop filters that look more and more like "ignore everything." I already read none of the business memoranda that I receive. I don't answer my phone. The number of workers needed to read and assess business memoranda is zero.

The slop filter still functions in math. Problems are pre-filtered for being of interest. Somebody expressed enough interest in the Navier-Stokes problem to give it a name and attach a prize to it. There are still plenty of "prize problems." Here are some:

* Quantum gravity

* Improvement of superconductors that don't need helium

* Practical generation of power from fusion

We have naturally only explored the mathematical universe in the parts where telescoping via clever abstractions can get us. But there is much more. Being able to juggle more things in your mind at the same time can have qualitatively massive benefits.

Information theory and proof theory, algorithmic information theory etc has of course explored this.