My thought: They're trained on so much web content that is plain wrong but authoritative due to apparent endorsement by humans that show tolerance based on understanding which is lacking in an "AI" next-token predictor.

Example, https://en.wikipedia.org/wiki/Progressive_web_app

"Progressive web apps must use service workers to create programmable content caches."

This statement is factually false. There is no requirement that a PWA do this. True is "To create a programmable content cache, a progressive web app would need to use a service worker." But Gemini parrots the false original.

I suspect the reason the false original persists as authoritative is the human understanding that subconciously translates it to the true version, theteby tolerating it. If queried, they would acknowledge the original is false. The bot has no such understanding.