Just this year, our cat has been having digestive problems. We got special food for her, which she hates, with the suggestion that she'll need to eat it for the rest of her life. Six vet visits later, Fable 5 suggested two tests that my doctor recommended we didn't get. Both found issues that explain her symptoms, and the vet says she will probably only need a supplement and infrequent two week courses of medicine if she has a flare-up.

All that to say, the helplessness of not knowing what's wrong and the people who could know not really caring enough is something that LLMs do a really great job of mitigating. If you don't have any way to know what's wrong with you or a loved one, or how to find out, you're stuck spending a ton of money (in the US at least) and crossing your fingers that someone gets it right.

These are pretty old. I'd be curious how performance compares with the latest frontier models.

So with regard to LLMs, for me the question is not purely "how often does it get it right?" the question is "how does it compare to the sources and self-diagnosis methods people use otherwise?"

Of course, I agree people should be careful with any form of self-diagnosis or LLM-diagnosis.

The pace of progress is so fast that many studies are totally outdated by the time they release

https://openai.com/index/introducing-mentalhealthbench/

I also appreciate privacy even though I "have nothing to hide" - just because I "have nothing to hide" it doesn't mean I want companies scanning my camera roll.

Then when brainstorming on a team, you'd start with "as a user"/"as an admin"/"as a Power-User"/"as Eva" and then use the first person. It framed the product story as something requested by that person.

Was just one way to go about it. Idk the origins of it though but it dates back to at least 2010 if memory serves, probably way before that.

Who is "we"? I, working in an LLM startup, know exactly what drives the base "voice" in the LLMs we train, because we have a process to select for it. OpenAI and Anthropic surely do too. Saying broadly that something is not well-understood in a scientific paper because it's not understood to casual observers is, uh, not very rigorous.

> The strange thing is that the base models (before RLHF) use the "experiential" voice, even though they are not incentivized to do that.

(Replying to your quote from another comment)

This is a matter of the training material. We have trained models that do not do that. I'm not exactly divulging trade secrets here. It should be really, really obvious that if you train a model on chat-conversation-like patterns of speech it will infer probabilities for how to continue a textual sample that will differ from the probabilities learned from being trained on narration, prose, or informational patterns of speech, even without RLHF.

Those don't have a themselves, because they can only continue text. A base model can only plausibly continue along the lines of what a character would say in a novel or what the narration would say in a story or in an article. Post-trained models may tie "I"-talk to actually observable effects they caused in some RL environment, or to how RLHF humans rewards its self-talk. But there is no themselves in a base model.

"You are a Large Language Model" in (system?) prompt would do the trick..

You know when chatbots ask you which answer you prefer between two. People tend to chose the "as a langage model..." one, so it stuck.