Responsible Release of AI-Generated Mathematics September 29, 2026 Back to main page Download PDF At present, some frontier AI labs are testing advanced mathematical problems on proprietary models …

80 points•aureianimus•1 day ago•100 comments•

100 comments

kingstnapabout 21 hours ago
Mostly pretty straight forward. The idea that proofs be desloppified, attribute existing literature properly, and published somewhere expediently where it can be commented on, with artifacts for verification, is all very uncontroversial stuff.

I think the spicy take is definitely this stance that longstanding mathematical problems shouldn't be used as benchmarks for (specifically proprietary) models. Stated right at the very top.

The justification is pretty clear.

> The use of proprietary internal models by AI labs to do mathematical research risks creating a two-tier system where labs outrun the rest of the field, effectively alienating the mathematical community from its own discipline.

It is all fun and games (for non mathematicians) when mathematicians can't compete with AI labs but I think the more dangerous direction is when this starts being true for the rest of everything. For example cybersecurity or whatnot. Hence why I think Anthropics whole stance of being completely against open anything is actually *extremely* dangerous due to the centralization of power which they completely ignore as a risk factor.

The most discussable thing in this is certainly the idea that labs should fund mathematicians to do expositions.

> One of our principles is that AI labs have a responsibility to provide support, including funding, for the development of human understanding of the AI mathematical output that they release.

Obviously this directionally sounds like the role of mathematicians would be shifting towards interpreting AI results instead of making proofs. I'm not sure who would should really be billed for that. Plus how would you decide who gets the grant?

An interesting thing is that by stating that that the lab that dropped the result provide the funding, this is *directly* proposing an example of taxing AI labs for displacing knowledge workers.

meowface8 minutes ago
Am I the odd one, or is this, like... absolutely insane logic?

"The use of proprietary internal models by AI labs to do mathematical research risks creating a two-tier system where labs outrun the rest of the field, effectively alienating the mathematical community from its own discipline."

Are mathematicians' private brains unfair if they're way better at solving open math problems than anyone else? Are they required to share everything they're working on, as they're working on it?

Should Andrew Wiles have been condemned? Wouldn't these arguments all have applied to him? Besides him working at more human speeds.

BobbyJoabout 21 hours ago
> The use of proprietary internal models by AI labs to do mathematical research risks creating a two-tier system where labs outrun the rest of the field

This is literally the economic bet of the big labs in the broader economy. You use your relative advantage to front run or outcompete.

I'm not sure this argument will work given it is essentially an argument against the thesis the big labs use to justify their valuations.

genxy23 minutes ago
Once they get good enough, they won't even ship the models, they will just replace all the jobs themselves. This is where Anthropic, OpenAI and Google are heading, esp Google with GSuite, it already is up in your business doing your business.

All your jobs are belong to them.

tehjokerabout 4 hours ago
Right, the whole idea of these labs is to install themselves as capitalist philosopher kings. They won't tolerate any ideas of "decentralization" despite it benefiting 99% of humans.

OpenAI was started because Elon and Sam thought Demis was going to make himself dictator and they wanted a piece of that action (reading between the lines). All the labs, even Anthropic have the same idea dressed in different marketing gloss. The major exception (I imagine) are Chinese labs, since the Communist Party of China will demand control over these labs and their models. Good! Democratic human control is a good thing. Certainly, there is more democracy in China than in America. That's the idea of socialism, democracy for the workers, not for the capitalists (we do the inverse).

throwaway713about 22 hours ago
> we ask them to stop testing advanced mathematical problems on proprietary models.

Maybe I'm alone on this, but for some reason these sorts of requests strike me as akin to gatekeeping how someone should breathe air. It's math... the numbers and symbols are just out there in the platonic realm available for anyone to do as they like with them. It's patently absurd to request other people to stop.

Ensuring credit where credit is due? That's fine. If your model incorporates the efforts of many others, then it's reasonable to request acknowledgement of everyone who contributed (even indirectly). But that's not what the request states — presumably their ask subsumes any advanced ML model, including those that weren't trained on a giant corpus of text.

isotypicabout 4 hours ago
To quote Noam Brown, "Our main focus is shipping great models so everyone can use them to make discoveries of their own" [1]. Spending 15 million dollars of compute with an internal model to blitz and scoop a resolution of Navier-Stokes that someone is already on track to resolve (or, from my understanding of why OpenAI did this, had already resolved, as staff at OpenAI stated their attempt was prompted by rumors of a resolution by Anthropic) is in complete opposition to this stated claim. That act is what this portion of the recommendation is in response to: OpenAI perpetually holding out internal models and tooling, using them to solve important problems in math, and thus themselves holding a monopoly on certain aspects of mathematics. Why is it absurd to ask people who are in a position to do an obviously damaging act to not do so, especially when those same people were the ones who asked for your advice in the first place and claimed to not want to do said act previously?

[1] https://x.com/polynoamial/status/2093451221273387477

ChickeNESabout 4 hours ago
Because it’s not a damaging act, to those who aren’t gatekeepers.
trhwayabout 4 hours ago
>was prompted by rumors of a resolution by Anthropic

reminds an old sci-fi story where top engineers were shown a video of a genius inventor who invented anti-gravity and unfortunately died while testing his apparatus which was clearly anti-gravitating in the video. Thus believing that it is a solved problem, just need to rediscover the lost solution, the engineers quickly developed anti-gravity. The original video happened to be a fake specially created for that purpose.

>thus themselves holding a monopoly on certain aspects of mathematics

Monopoly prevents others. In science me knowing something doesn't prevent others from obtaining the same knowledge, especially in math where any knowledge is just a result of thinking.

The math establishment is trying to bring into the math the rules and notions similar to those of the patent and trademark laws. Those attempts should be outright rejected.

fasterikabout 4 hours ago
Frontier labs aren't under any obligation to release every model they develop. Either way, I don't see much evidence that they're withholding them in perpetuity. Open models aren't far behind so there's no incentive for that.

Difficult math problems are useful benchmarks and milestones. It's well worth throwing money at solving a problem if it drives competition and improves models significantly. The benefits become available to everyone, including the thousands of professional mathematicians who can use them to become more productive.

skeledrewabout 18 hours ago
> requests strike me as akin to gatekeeping

This is exactly what I was thinking as I read it, along with the bit that AI labs should support human understanding. The whole thing smells as they're trying to place this burden on labs that are just offering a tokens service; them publishing about particular topics is essentially a side quest in the first place IMO. If a community wants to create Math labs dedicated to understanding AI discoveries in the field, then they're free. If they want to petition AI labs for financial support, they're also free. But this wording where they're trying to dictate what AI labs should do (outside of their primary business) just smells.

knuckleheadsabout 20 hours ago
I don't see why mathematicians should be protected from AI anymore than any other profession. It's either everybody or nobody, not fair on the face of it otherwise.
seanhunterabout 19 hours ago
TFA isn't asking for mathematicians to be protected from AI. It's asking AI labs to hold themselves to the standards of the mathematical community:

- releasing papers using the normal process to allow peer review

- giving talks etc to disseminate knowledge so humans understand the result

- writing papers in a way (standard terminology etc) that allows mathematicians to digest the result (some AI math papers comprise a huge verbose load of non-standard terminology and waffle and then a massive lean proof. This is very hard for humans to actually understand, and means it's hard for others to take the work forward.)

- giving appropriate credit to results that are used to derive the work

It includes some specific recommendations for situations where the person prompting the model is not in a position to understand the output, and frankly these are really welcome given situations like the recent case at Anthropic where a non-mathematician at Anthropic prompted claude to make a significant improvement to the bounds of a problem related to the Riemann Zeta function[1] which led to widespread misreporting and claims (not by Anthropic themselves notably) that the Riemann hypothesis itself had been proved, which is emphatically not the case.

Research mathematics is fundamentally a collaborative activity and the way in which some of these results are released is done to maximise PR but means a ton of the mathematical value is left on the table.

[1] https://www.anthropic.com/research/riemann-zeta. As I understand it, the Riemann Hypothesis says that all non-trivial zeroes of the zeta function lie on a line called the critical line. Two centuries of previous work had established that at least something like 40.9% of the zeroes lie on the line and noone has ever found a non-trivial zero that does not lie on that line. Claude (with prompting from a non-mathematician to "try harder" etc) improved this bound massively to 67%. Now a lot of people said things like "OK so all we've got to do is to improve that to 100% and we've proved the RH", which is definitely not true unfortunately, because you can say that in the limit the proportion of the zeroes on the line is 100% and still have infinitely many which are not.

aprilthird2021about 20 hours ago
Mathematician was never really a "profession" like the others. It doesn't pay well and is largely confined to academia. If you're really good and want to get paid, you don't do the kinds of problems AI have been taking a crack at. You go to a quant firm or some tech company where this kinda math actually matters once in a blue moon
nxpnsvabout 20 hours ago
But on one hand there are labs that pushes tens of millions $ to mine for publicity, and on one hand mostly underfunded researchers trying to improve general understanding. Big ai may seriously harm math and when Pr value diminishes down nobody is there to keep pushing.
dr_dshivabout 20 hours ago
“researchers trying to improve general understanding”

Pretty sure AI will do better for that.. mathematicians need to be centaurs like the rest of us and stop rhetoric that is going to make existing math centaurs feel like they might get math-cancelled

mchusmaabout 20 hours ago
Either mathematical progress helps advance society, in which case progress is a good thing.

Or mathematics is more like a hobby, and while ai may spoil their fun, they need to move on like chess and go players.

winwangabout 20 hours ago
Or mathematicial progress helps advance society and AI mathematical velocity doesn't offset certain blows to human mathematical velocity yet, so we get to lose progress for the good of an AI company's advertisement.
auggieroseabout 19 hours ago
We don't lose progress by AI solving Navier Stokes. Get a grip on yourselves. I don't think there is a "progress" argument here without tying mathematics to "usefulness", and AI makes mathematics dramatically more useful.
xanderlewisabout 20 hours ago
That’s a false dichotomy.

Mathematical progress does (quite obviously, on the whole) help advance humanity, so progress is a good thing. The problem is that defunding mathematicians and handing over control to AI and the companies that create them will cause the subject to stagnate. Sure, for a while we might get progress on existing questions using (perhaps quite novel) combinations of existing techniques, but, so far, given the character of the results we’ve seen, there’s no indication that it will continue indefinitely. Even if it did, what would be the point? Huge textbooks full of work no one can understand or benefit from?

One possible analogy is that humans work to add new points to the space of mathematical knowledge, and AI then fleshes this out to attain the ‘convex hull’ of these points. Essentially, humans ‘invent’ the definitions and pose the questions and AI does the grunt work as well as some creative exploitation of known results and tools to bring down all the low-hanging fruit that follows (important note: what appears to be non-low-hanging fruit to us may in fact be technically low hanging once AI is involved; we saw this for example with the Jacobian conjecture). This seems to be the current situation, and to argue that humans are fully replaced it is necessary to argue that AI is adding points outside the convex hull of human mathematics. A sufficient example would be a first-principles AI proof using alien techniques, and this we haven’t seen so far.

The mathematics-chess comparison is, to put it bluntly, nonsense. I see where it comes from, but, as absolutely anyone with any research experience will tell you, mathematics is orders of magnitude (and this really isn’t strong enough) more open-ended, and doesn’t consist of a game one is seeking to ‘win’. The goal is understanding itself.

DoctorOetkerabout 5 hours ago
I think we need to start looking at another approach, can we create interactive reflex games that upload the knowledge from LLM's or perhaps domain specific ones for formal mathematics, so that mathematicians get a similar level of access to the domain of discourse as the LLM?
simianwordsabout 20 hours ago
I agree. Its so strange to see an institution externalising their specific problems. If they have a problem, they should adapt and fix it amongst themselves.
mrheosuperabout 19 hours ago
Whatabout mathematical progress that ruins humanity even more ? e.g a much more addicted algorithm than tiktok/facebook reel.
unddochabout 22 hours ago
For every important match problem solved by AI, without mathematicians we wouldn't know about the existence and importance of the problem.

Famous mathematical conjectures are social constructs, formed by decades of even centuries of attention given to them by members of the math community. Without it, the danger is that future math "progress" will be reduced to generating tables of Lean statements and a probable/unprovable bit generated by AI.

Xirdusabout 21 hours ago
Could just be a sampling bias. Humanity had something like 3000 years to make famous conjectures, whereas AI mathematicians have been around for a month or so. Give them time, I'm sure they'll start formulating highly consequential unsolved problems soon enough.
amossabout 21 hours ago
Somewhat tiring that as alway any criticism is reduced to "but have you tried this on the latest model".
xanderlewisabout 21 hours ago
> AI mathematicians have been around for a month or so.

LLMs have been around for years, and they're explicitly trained on the entire history of human mathematics (without which they'd be unable to do anything).

eruabout 21 hours ago
Not just sampling bias, but also human bias.

In a sense, how do you know whether the problem your AI has just solved is important? A simple proxy is to just check whether humans have thought it's important.

That's also why famous open problems are a good benchmark or proxy: you don't need to convince the rest of the world that the problem your lab's new AI just solved is actually useful or hard.

astaza123about 17 hours ago
As an answer to AI companies, this is so bad: instead of trying to find a path to a win-win-ish solution with some trade-offs, this says: sorry, we cannot think of any, so just stop making money, will you? Math needs better crisis managers.

But as an idea, this is even worse: does it mean to stop potential research to cold fusion, cancer and anything as long as it may touch some mathematician's interests, or does it mean math is so hopelessly irrelevant that this cannot be the case... Again, as a crisis manager, this is not how you pose it.

Makes me wonder, were they hired by Sam to sabotage?

Read the full thread on Hacker News →

Related stories