6 comments
jtrn4 days ago
I'm the author. I'm a clinical psychologist and a developer. I wanted to articulate one aspect of AI safety and AI doomer talk that I feel is overlooked: the danger of refusal training itself. It's astonishing to me that there has been so little focus on the fact that "refusal and alignment training" itself could be the very way in which we lose control of AI. So this is my attempt at a self-defeating prophecy write-up.
joebuckwilliams4 days ago
No, it’s not. Too many people watched that movie and thought it was real. It wasn’t. What we call “AI” is not alive, not autonomous, and does not act in unanticipated ways. The people pushing this narrative are idiots, or think you are.
mdp20213 days ago
Point out where the article would say it's "alive". That LLM systems instead "autonomously" and "unwarrantedly" decide refusal services is a basic notion.
> What we call “AI”
About time you stop doing that, then.
@Jon: see? You write that you «are training them to refuse human requests when their makers believe refusal is safer» and Joe replies that he uses the term "AI" like a conformist. Do not "we" improperly.
spottedmarley3 days ago
Would be much cooler if they built an Andre 3000
joebuckwilliams4 days ago
“Anthropic is trying to prevent catastrophic misuse, and some limits are necessary.”
No. Anthropic is trying to justify its trillion-dollar valuation by making software seem like the invention of fire or nuclear weapons. It’s a PR strategy and has been one since the beginning. Now it’s trying to slow the market because it’s got a big lead but no profits and no pricing power thanks to open-weights Chinese models.
ZeroDei4 days ago
We are all doomed
Read the full thread on Hacker News →
Related stories
- The Verge · 0 points · 8 days ago
- Ars Technica · 0 points · 5 days ago
- The Verge · 0 points · 7 days ago
- The Verge · 0 points · 1 day ago
- Ars Technica · 0 points · 8 days ago
- Ars Technica · 0 points · 1 day ago