That's happening across the board when it comes to models that feed off online information. Create a website with a false claim, have AI slurp it up and spit it back out when a user asks a question.

Happy elections.

I've been seeing the back-side of this as a mod at the Julia Discourse. Pagerank/SEO spam is ridiculously obvious and trivial to detect/block — links are all that matter. Many GEO spam posts are similarly obvious: someone posting about a cryptocurrency customer support hotline on a programming language forum is definitely spam. But recently spammers/scammers been using AI to _tailor_ posts to look much more authentic, posting "How to use Julia to analyze market statistics" referencing (without links!) a particular crypto exchange and then (sometimes) going back later to edit in phone numbers or the like.

What makes this even more painful is seeing all the good faith answers that such GEO posts spur from the community. It's far more abusive than SEO spam.

You don’t need to be a hacker to do this. My elderly neighbor searched for some version of “Microsoft help center phone number,” but Gemini suggested a scam number based on her search terms. She was scammed so much that I needed to spent the weekend helping her wipe computers, lock down the digital accounts and deal with identity theft.

AI poisoning is currently the new exploit frontier and I don't think it's going away anytime soon.

I was just contemplating the other day that agents we use at work are exposing new attack surfaces. For example, historically a folder of documents (maybe google docs) isn't a huge risk, particularly if it doesn't contain sensitive documents.

Now though if one employee is running a harness capable of computer control, and running an agent that reads from that directory, simply dropping some files with instructions could poison the AI to leak info or even own the host.

Skills are another supply risk, malicious instructions could be added to them.

Imagine working decades and decades on PageRank™ only to throw it all away because you felt you've been left behind in the AI race. This is how you lose focus.