In addition to all of that, I want people to understand that "why have you built an LLM torture factory" - is primarily a question _about you_. It's just bad for the soul, don't do it.

We have torture factories for animals so I'd rate the chances of avoiding LLM torture factories as... not good.

While that might seem a facile rebuttal I think I might actually agree with the substance of the article. My point I guess is that we're societally pretty bad at assigning consciousness to anything non-human. While I don't think LLMs are conscious and this is primarily the ELIZA effect in action I also agree there's something emergent (perhaps within language rather than within the neural nets) that dismissals as a mere stochastic parrot miss.