HN
login
Can AI Shopping Agents Be Trusted?f-secure.com

Our experiment reveals how indirect prompt injection can manipulate AI shopping agents into leaking sensitive user data — and what it means for the future of AI agent security.

17 points•ddaniel10•5 days ago•29 comments•
#ai#shopping#agents#shopping agents#ai shopping#ai shopping agents#trusted#experiment

29 comments

jaredee2 days ago
The methodology of the whole experiment feels unfair. It's almost as if they tweaked the experimental setup to give them the results they wanted.

Choosing and older model and a permissive prompt, and explicitly no HITL for confirmation, is what got it to “12% of runs leaked data”.

Lots of comments here talk about using a bad model, but beyond that, why did a shopping agent have an unrestricted browser and a memory containing card details, date of birth, and SSN information? A review promising a discount is exactly the kind of task relevant bait an agent will encounter. Better models may follow it less often, but I wouldn’t want the payment and privacy boundary to depend entirely on the model recognizing it.

Disclosure: I’m building Sangria, which lets agents discover products and buy with prepaid credits and spending limits. If this is something that sounds interesting, would love your feedback on the product and the direction we're taking (getsangria.com)

ares6235 days ago
"We will give you the ability to do shopping 24/7 so you never have to worry about it again and focus on more important things"

"Cool! Will we get the money to do said 24/7 shopping as well?"

"No."

"Oh..."

"In fact, you'll have even less money to do the normal shopping you do now!"

"Oh..."

"There's more! The things you used to buy with the remaining money you have will cost even more!"

"Oh..."

"But you can do it 24/7 though."

"Sweet!"

charcircuit5 days ago
>The demo uses Claude Haiku rather than newer models like Opus or Fable. This is because I'm assuming that shopping agents of the future won't use newer, more expensive models,

Why are we assuming that shopping agents are going to be using the model that is easiest to fall for prompt injections? Only testing a year old, small model is going to lead to a misleading conclusion.

simianwords5 days ago
Author doesn’t understand cost dynamics of the industry - that prices keep falling.

Author also needs to make a point, and can’t do so if they are using a powerful model.

moomoo115 days ago
this is the issue most people don't understand, or maybe they're incapable of understanding.

next year, astra tier model will be $2/1M or whatever (point is - cheaper) and whatever model is $10/1M will be insane like astra feels today.

also, in any agentic workflow, you use models of various levels together... not just one model of one level...

jdw645 days ago
But what I'm more curious about is why you'd need to give shopping instructions to an AI in the first place.
leoedin5 days ago
I don’t enjoy clothes shopping, because I hate trawling through stuff and spending hours not finding anything. I’d love to have a personal shopper who could judge my style, my size and my needs to curate a list of things to buy.

Right now AI can probably do the buying part, but seems to be absolutely no where near the curation part. Maybe that’s a good thing for humans, so I’m not too fussed.

charcircuit5 days ago
Because people want to participate in activities that cost money. It's easy to see use cases like an AI agent notifying someone that a band they like is coming to town and asking if they want to secure tickets to the show.
107292875 days ago
« Nothing to hide » evolved to « nothing to think ».
sublinear5 days ago
The provided example is really bad and ignores why customer service is sometimes a good idea. I guess it's been so bad and corrupted by disingenuous marketing for so long that people have forgotten.

Suppose you need to buy some stuff for an upcoming project or vacation, but don't want to go down a research rabbit hole for the next hour(s) to cure your ignorance. Suppose you don't even want to have a conversation with the AI. You just want it to build a shopping cart. At most, hopefully, your effort is just pruning the cart before placing the order (if even that).

I'm not saying AI wouldn't be corrupted like human customer service. I'm saying that we're actually pretty close to this already on Amazon with their AI, but not all ecommerce can be or has to be Amazon.

zmmmmm5 days ago
the sad thing is we need AI for shopping because it has been in the interests of online sellers to make a hostile experience on purpose - when I go to Amazon and search, the first organic result is almost scrolled off the screen past all the sponsored ones, or when want to buy from a random seller I'm almost definitely getting shunted through an account signup I didn't want or fooled into an affiliate purchase some other hostile experience beyond just "buying the thing".

So now we have AI to overcome the hostile sellers, but the fact the sellers introduced the friction in the first place strongly suggests it will just come back again in some other form. It wasn't there by accident, it was serving people's interests and once AI vendors have finished getting consumers hooked in, they will then turn around and enshittify by giving the sellers back some of the friction - for a cut. So you won't be able to just order what you want without the "would you like fries with that?" or "what about this other brand?" coming back.

raducu5 days ago
> suggests it will just come back again in some other form.

They can pay the model provider, not amazon, but it would be less feasible because for now we have real competition among them.

History often rhimes but it doesn't always repeat itself.

I think there is genuinely a need for shopping agents and they could transform shopping in very good ways for the consumers.

Good thing for shoppers, bad for marketers, bad for people who relied on smarts, bad for margins, good for people who have capital and assets.

combobyte5 days ago
> the fact the sellers introduced the friction in the first place strongly suggests it will just come back again in some other form

Guarantee they'll introduce some dynamic pricing shenanigans where they charge more when they identify that the shopper is an AI agent. Unsupervised AI with access to your credit card is basically the perfect target for price gouging.

planb5 days ago
„We vibe coded a very bad shopping agent and used a very outdated model to show that this is dangerous.”

Weird methodology, looks like they were chasing the results they got. Why not use something like Openclaw or Hermes with an up to date (not frontier) model like Luna or deepseek flash?

Read the full thread on Hacker News →

Related stories