Here’s a product feature which the world is going to need a whole lot more of over the coming months and years: default hard budget caps. I’m talking about the …

652 points•elffjs•about 2 months ago•313 comments•

313 comments

motionlessvelocabout 2 months ago
I used to work on a support team of a well known backend type service that had hard budget caps.

It was, unfortunately, a nightmare. There were tons of tickets and even threats of lawsuits from customers whose service got cut off hard at the worst possible time due to organic growth/going viral/big event/nobody knew about the limit/etc. Not only did they lose all the leads and revenue they would have gotten from that bump, but they also pissed off their own existing users who suddenly couldn't use the service either.

Generally speaking, it's much better to use alerts instead of hard limit. Even in the worst case (hackers pwn your credentials and mine Bitcoin or whatever) the rest of your business is unaffected and you can negotiate with the billing department at comparative leisure.

This is all assuming you have humans operating the service. If you're letting AI agents yolo infra in prod, you have a whole series of new problems.

KronisLVabout 2 months ago
> Generally speaking, it's much better to use alerts instead of hard limit.

Probably for companies and opportunistic individuals with a high risk tolerance. As for me, just give me hard budget caps. It shouldn't be a choice between supporting one of the approaches, when you could let each client pick what they want.

Want an alert? Configure one.

Want a hard limit? Configure one.

(just make the configuration easy and visible)

edoceoabout 2 months ago
The thing I see is folk putting a limit when given the choice. Later the situation changes, they want mega scale. Then don't fiddle the settings. I think it's a documentation & process problem.

1000 years ago, before CI/CD took over folk would have launch/go-live events, where teams would do a good checklist. There were pre-launch meetings to review, which included load checking.

Brezaabout 1 month ago
I completely agree with you. If I'm running a big VM for model training, I want to be absolutely sure that I can't accidentally leave it on over the weekend. If I'm a fast growing startup, maybe I'll prefer alerts so I can optimize a slow piece of my stack before getting a surprise bill.
walrus01about 2 months ago
That doesn't mean you couldn't have a service which by default has no hard cap, and people have to opt into it. You could even put a user interface thing where people have to type a whole sentence perfectly matching and hit OK, like "I understand that enabling a hard billing cap will shut off services if it exceeds my monthly quota". Wrap it in as much service agreement contract, TOS language as is necessary.

Heck, have it do the equivalent of send people a DocuSign equivalent PDF to sign acknowledging the risk before enabling it. Would it still stop pissed off people? Probably not. Would it help with the risk of lawsuits, very possibly.

reticulatesabout 2 months ago
Yes, this is an important point although it has changed with A.I. Software is traditionally very high margin and so a $10k bill can be written off by the provider without any meaningful loss.

As a customer, the big number is scary and causes panic but for the provider… customers constantly fail to pay bills, providers are constantly writing off bills because it just isn’t worth the cost to chase, if a customer says “hey that usage was a mistake” it’s usually worth it to write it off to save the relationship. If you write off a big bill that wouldn’t have been paid anyway, the customer will perceive you as wonderful and benevolent and be loyal for life when they are ready to spend their money.

With tokens though the actual cost being incurred is much, much higher. If your service is just a wrapper around tokens, and a customer incurs $10k of usage that you paid OpenAI $5k for, it becomes much more difficult to write off.

Google Cloud is one of the few services that actually pursues unpaid bills even on their high margin services.

preommrabout 2 months ago
Google Cloud is also the scariest because of how much damage it can do and how bad their payment system can be.

I recently loaded up on prepaid api credits for gemini and it somehow triggered some billing shenanigans in my linked accounts where it said I had a negative balance (from the credits), and they were going to discontinue my services. I had to reset some settings to sort it out, mainly using their chat ai and mine (because theirs gave me right status info, but wrong conclusions).

It's pretty messy across like aistudio.google.com, and their typical console, and google workspace business account. I'd be so fucked if they froze my account, I'd rather just pay openrouter to access credits in the future.

sandworm101about 2 months ago
That will change as data centers become more "compute hubs" than service providers. When a good chunk of your bill is the cost of the electricity the compute center will have little margin to negotiate. Power companies do not care about customer mistakes.
mybrowsercacheabout 2 months ago
My mobile phone contract reduces my speed a lot when the hard limit it reached. I still get some connectivity. What Never happens is that I have to pay extra. If a company can’t give me hard limits on the cost I have to pay, I will not do business with them. Notifications when close to the limit are fine, but when I don’t react to them I want things to crash rather than having to pay indefinite amounts.
collingreenabout 2 months ago
Isn't that a soft limit? I guess it's a hard limit from the "zero additional cost" perspective but sift from the "stuff still works but in a degraded state" perspective.
mcintyre1994about 2 months ago
I can understand how that causes issues, but surely it also shows that alerts don't work either? These companies were presumably missing/ignoring alerts with scary messaging that they're about to get shut off, and would also miss/ignore alerts with scary messaging that their bill is getting too high.
pixl97about 2 months ago
Ya, especially on the open internet where the time between the first alert and bankruptcy could be minutes.
watwutabout 2 months ago
Ignored alert works exactly as the provider wants it to work. It makes them earn more money.

It works best when coupled with spam of other irrelevant alerts, to maximize the chance of higher earnings.

kqrabout 2 months ago
This goes beyond dollar charges. For production code to be reliable, everything needs to have a hard limit.

Queue lengths, request sizes, response wait duration, message payload size, authentication attempts, allocation rates -- there's always some upper number beyond which the system is so messed up you'd rather it crashes.

> An argument against this is that businesses don’t want their hosted applications to start throwing errors because some budget was exceeded. I expect that most businesses and individuals would prefer errors to a surprise $10,000+ bill.

Indeed. If you want a surprise $10,000 bill that's still not an argument against a hard cap -- just set it at $9,999,999 instead, or wherever you don't want the surprise bill. There's always a number that indicates something has gone insane. There's always a sensible upper limit to any operation.

yencabulatorabout 2 months ago
> There's always a sensible upper limit to any operation.

Ooh. It could even literally be "hard limit is 3x of previous month's bill", and you can adjust the multiplier.

That's much less likely to be set, forgotten, and gotten limited by, but it's unlikely to bankrupt customers.

LorenPechtelabout 2 months ago
Yup. Things happen. I'm still picking up the pieces because somebody sent a complex job through broken down into a bunch of separate pieces. Maybe 10x as many parts as normal (and each part caused the run of an external process, a third of them failed to start), one report came back something like 100x as big as normal. All local stuff so it didn't cost anything beyond the production stall.
mstaoruabout 2 months ago
I find Google AI Studio / Dev Platform / whatchamacallit one of the worst in this.

When video models just came out, I wanted to make a short clip. Gemini didn't have enough control back then, so I started in Studio. I had $10 on my account. Try, try, try, not good, retry, altogether maybe 20-30 retries with 4 choices for a 10 second video.

Wake up next morning to an email from Google - my Studio account is frozen because of negative balance.

I check it, it's at -$160. Not financial ruin, but a painful sum for a 10-second video I didn't even use in the end.

I don't think there was (or is) a setting in there that says "stop everything when I'm in the negative".

Almost a year later, it's still as bad. Trying to make some music via Lyria, I need to wait 10-12 hours for the API charge to show in my costs. Now I had the bitter lesson, I wait after every major novel experiment to see how much it costed me.

Coding is solved, my ass.

Guillaume86about 2 months ago
Had a similar experience with OpenRouter, put 10$, goes -50$, I just stopped using it. To their credit support agreed to reset my balance (AI refusal at first, then re-contacted me months later to agree). I'm being more careful now but I shouldn't have to be IMHO.
hmokiguessabout 2 months ago
Cloudflare is worse, they have no controls whatsoever.
hyperhelloabout 2 months ago
These shouldn't even exist without a negotiated contract.

I can subscribe to your service for a specific fee on a monthly basis ($20/month say), and take the risk of losing that month's fee if your service or I make mistakes, or I can choose to drop another $20 mid-month, or anything for convenience.

Saying that the computer will "control" the billing and can run haywire tells me that I don't want to be anywhere near your pile of bad incentives.

Retricabout 2 months ago
Monthly electricity bills are based on usage, and it works well but there’s a limit to how surprising a bill can be. The difference is the relative orders of magnitude you can be charged for these services you can go from 20$/month to 200k/month without warning.
handoflixueabout 2 months ago
There is also a hard physical limit on how much electricity you can use before you blow out the fuse box
roliszabout 2 months ago
In the UK there are prepaid electricity providers: there's a card reader at the power panel. When you run out of credit, power cuts off, it resumes when you pay again.
brapabout 2 months ago
When I first started working with cloud providers it was shocking to discover this feature doesn’t exist, basically anywhere.

It’s such a basic thing, not having it has to be deliberate to make you accidentally spend more than you’d like.

Brezaabout 1 month ago
This is why my first cloud provider when I was learning VMs was Verda. They let you prepay and everything would shut down the second you ran out of money.

It was a necessary feature when I was learning. I was amazed when I moved up to the big guys and this wasn't an easy option.

foldrabout 2 months ago
>It’s such a basic thing, not having it has to be deliberate to make you accidentally spend more than you’d like.

This is unlikely. The big cloud providers routinely cancel bills based on accidental usage. The real explanation is just that it's technically difficult to calculate all usage in real time and shut down systems immediately as soon as a certain cap is reached.

nicceabout 2 months ago
> This is unlikely. The big cloud providers routinely cancel bills based on accidental usage

That is not an evidence alone. They also don't cancel bills in many cases. Most people also just swallow the damage and pay. Like gym subscriptions where people just pay because cancelling was too hard and they postpone the canceling because it takes so much effort. And they also think it is their fault of not cancelling, because it took that much of effort.

> The real explanation is just that it's technically difficult to calculate all usage in real time and shut down systems immediately as soon as a certain cap is reached.

That is trivial to add these days. Cloud providers usually say that "we care about your business and don't want to make your systems go down unexpectedly if you did not pay your bills".

Terrettaabout 2 months ago
Mistaken view.

I was a member of AWS customer advisory board for years. Two dozen of largest enterprises and unicorns and so on. Nobody wanted the billing department "inline" of systems controls.

AWS provides all the controls you need to flip the switch yourself, and provides the CDK / TF / etc. patterns for it. Plenty GitHub projects to host your own light switch service.

Don't relinquish the switch.

callmealabout 2 months ago
> Two dozen of largest enterprises and unicorns and so on. Nobody wanted the billing department "inline" of systems controls.

And of course, two dozen enterprises trump hundreds if not thousands of other small businesses right? Did AWS think of, oh I don't know, a 'feature flag' or somesuch thing that enterprises and unicorns are so fond of to provide this feature to those who don't have VC money to flush?

devsdaabout 2 months ago
Amazon didn't have to build a hosted db, queue, mail or anything else that is value add as long as it provides basic hosted compute, storage and network.

But it still does and can only claim having those after building them.

Their customers have been asking for that hard cap feature for a long long time, not building that feature while saying customers have all the tools to build it themselves is just a deliberate misdirection at this point.

kemitchellabout 2 months ago
Might you recommend any talks or writing on the AWS advisory board and how it works? I found Amazon’s reference page on it, but it’s scant.

Read the full thread on Hacker News →

Related stories