Wall Street is growing skeptical of the data center boom

Wall Street is growing skeptical of the data center boom nytimes.com

72 points•mikhael•9 days ago•87 comments•

#data #street #wall #boom #center #skeptical #wall street #growing skeptical

87 comments

Any data center that can remain profitable selling open source tokens at commodity prices will be fine. Any data center that relies on OpenAI/Anthropic level token prices and margins might be in trouble. After clearing their debts through bankruptcy, they'd likely be quite profitable selling open source tokens at commodity prices.

FuriouslyAdrift9 days ago

Eventually these models will get commoditized (we're already at the "good enough" stage for real work), then they will get turned into custom hardware and get 1000x faster, then that hardware will get commoditized (like DSPs) and they'll be everywhere and cost $1.

bryanlarsen9 days ago

From a very high level view DSP's and GPU's are the same thing: highly programmable number crunchers. A better example would be Google's Tensor chip. It's not as general purpose as a DSP or GPU, it's more optimized to do inference.

That trend will continue.

I figure there are 6 order of magnitude events that could happen in the next decade to lower token prices:

- more specialized / better chips

- IC technology: smaller feature size, higher clocks, etc.

- more efficient algorithms

- solar power is getting cheaper at an order of magnitude per decade, batteries even faster.

- pricing pressure from open source models

- breaking of the Nvidia monopoly and it's 75% gross profit margin

Maybe all 6 won't happen, but certainly a 1000x reduction in price in the next decade seems highly likely. Jevon's paradox says that the 1000x reduction in price will likely result in more spend on AI, not less.

binary1329 days ago

It blows my mind that anyone hasn’t grasped that the token pricing game is a race to the bottom forever.

greyface-9 days ago

Any data center that is in the business of selling tokens rather than space, cooling, power, and connectivity is going to be in trouble eventually.

bix69 days ago

Quite profitable at commodity prices? I don’t buy that. And all of them are building with debt / equity that expects high token prices?

bryanlarsen9 days ago

After bankruptcy they likely have no debt, so can out-compete those that didn't go through bankruptcy. It's the bankruptcy that makes them profitable -- it's a common pattern in nascent commodity industries.

nomel9 days ago

I think there's still some low-hanging fruit with thin clients and colocated-to-AI applications. I don't think the math for beefy personal computers is going to hold up, for most use cases.

Apes9 days ago

If it costs you more to generate the tokens that the market is willing to pay for those tokens, then not even bankruptcy will save any of the costs invested in one of these datacenters.

If the cutting edge OpenAI token prices are $80 per 1M token, and the open source tokens are $1 per 1M token, that's a huge gap of "this will never be able to make money under any scenario if the bubble bursts" that will catch a lot of these new datacenters. No one will run a datacenter that costs $5 per 1M token to sell at $1 per 1M token even if the debts are cleared.

nostrademons9 days ago

The point being made here is that most of those costs are amortized capital costs, which get wiped in bankruptcy.

That $5 per 1M token doesn't literally cost $5 per 1M token. It's more like they had to build a datacenter for $500M that can service 100T tokens over its lifetime. They did this by borrowing money on the capital markets, and now they have to pay interest to those bondholders, interest that they can recoup with their $80/1MT prices. But if it turns out they can't charge $80 and have to charge $1, they won't be able to make those interest payments. They enter bankruptcy, the court wipes the debt clean, and now they don't have to pay interest, only the actual operating costs, which may be more like 50c/1MT. The company gets recapitalized with the new owners being largely the bondholders, the existing equity holders get wiped out, and they can compete with the commodity producers now.

Apes9 days ago

I'll believe it the day SpaceX - the AI company that is mostly making money selling datacenter compute using gas turbines for energy - takes a single day 90% or greater drop in stock price.

Zigurd9 days ago

Currently nobody knows when the first big financial crisis is fully locked in. For example, if OpenAI can't close another round, and they default on their contracts with Oracle, there's your sign. Until then it looks like everybody is enjoying the communal hallucination.

downrightmike9 days ago

Probably the next nvidia chip drop because the old chips where priced at their highest level at a depreciation of 5 years instead of the 2 year norm. What happens when the next chip is way better(hearing 67x better, not just the 10x from initial claims)? Well all those chips need to be dumped fast and depreciate that second and the datacenter gets bankrupted and parted out

nostrademons9 days ago

OpenAI's already announced they're not going public in 2026:

https://www.reuters.com/legal/litigation/openai-ipo-will-not...

They could potentially do another private bridge round, but for a company that was gearing up for the largest IPO in history a couple months ago, the reversal is a pretty bad sign. For investors that are looking for a fire sale, there's already smoke in the air.

TacticalCoder9 days ago

> Currently nobody knows when the first big financial crisis is fully locked in.

What do you mean the "first" big? 1929? 2001? 2008?

Do you mean 1929 wasn't a big financial crisis and that, this time, we'll have the first "real" big financial crisis?

I'm confused.

Dependance9 days ago

Be careful what you wish for. The repercussions might be titanical.

Apes9 days ago

We're already seeing repercussions from an economy that has been retooled not to actually produce anything of value, but to produce more air to fill up the largest economic bubble in the history of the world.

National debt through the roof, inflation through the roof, PHD and research programs gutted, non-ai startups dead and unfunded for the last 4 years. These are just a few things that have been sacrificed on the altar of this bubble - there's far more I haven't recounted.

We're already in a widespread long term economic collapse, but the delusion just hasn't broken yet.

lil-lugger9 days ago

Is there anyone serious who thinks that the future is local models anyway? All computers used to be the size of rooms like these data centers and then they got smaller and faster until the home computer came. Is that not a possibility down the line as we improve efficiency of the models and increase compute?

Zetaphor9 days ago

I have a shoebox sized computer (Framework Desktop) running Qwen 3.8 Flash Next. It has completely replaced my use of proprietary models in my personal life. 6 months ago I would have told you this was impossible. Based on the current trajectory I expect 6 months from now I'll have a Mythos class model at home. The best part is not having to concern myself with token cost has unlocked all kinds of experimentation and use cases. I have been pushing over a billion tokens per week for multiple weeks now, all for the $52/year it costs to keep this machine running 24/7

lostmsu9 days ago

What's the output rate and config?

Analemma_9 days ago

I have no strong opinion on whether the endgame of AI services is local or in-cloud, but I think your historical analogy is pretty suspect: it's true that computers got smaller and faster, but it's also true that most people have shifted most of their workloads from local and on-prem to datacenters since the turn of the century. Why would AI be an exception?

stult9 days ago

Do you mean is there anyone who thinks that the future is NOT local models anyway? Since that seems to be the gist of your other points

lil-lugger9 days ago

Yes that’s what I meant thanks for clarifying. I think local models are the future but I don’t see anyone serious arguing it. If it were true these city sized data centers will be graveyards in 5-10years

Zigurd9 days ago

AI budgets and AI pricing have too much squish in them currently. Frontier models are being sold at a loss, and AI budgets are experimental. And there is still a whiff of FOMO in the air.

Also, Google and Facebook are still spending like drunken sailors. Nobody has stubbed their toe on hard limitations yet. So yes of course people will figure out how to optimize the cost of AI in their products. Just probably not this year.

johnbellone9 days ago

Google and Facebook have profitable businesses.

lenerdenator9 days ago

Like everything else, "it depends".

There will be people who want to host things on-device. At some point, you could probably do most day-to-day tasks with a Siri-like agent, so you don't necessarily need it to be on a datacenter rack somewhere.

More complex tasks being run quickly opens up a choice: insanely beefy individual devices, on-prem hosting, or cloud hosting, whether that be some data center running FOSS models, or ones from people like Anthropic or OpenAI.

Beefy hardware for individual users? Not cost-effective. Could have people share that hardware by putting it in a data center. Do you want to operate that data center? For proven business cases, sure, why not? If you're still working out what your scale will be, maybe you ask the Googles, Amazons, or Microsofts of the world to rent you the hardware so you don't have wasted or too little capacity.

The real question is, how much value is there in a few companies that talk about how their eventual goal is to create AGI as opposed to just giving you enough intelligence to augment your current workers?

The answer is "probably not enough to justify more than one company having a valuation of over a trillion dollars, and that's generous".

fedecaccia9 days ago

There are two different clocks here. Financials could change in months, but the electrical infrastructure needs to be planned with years.

Gas turbines are all practically sold out till 2030, delivery time changed from 2-3 years to 5-7yrs.

Many announced data centers where destined to delay, independently from financial markets, due to the available infrastructure.

If the AI/data centers fomo dissapear, the effect on electrical production will take years in take effect, because capacity is already reserved and paid.

In oposition to software, energy infrastructure is not flexible: you can't speed up a turbine projected for 2032 neither cancel the order without a considerable cost.

Read the full thread on Hacker News →

87 comments

Related stories