Login
You're viewing the mstdn.social public feed.
  • Paul Cantrellinthehands@hachyderm.io
    Feb 9, 2026, 4:34 PM

    There’s a lesson here, perhaps, about the tangled relationship between what is •typical• and what is •correct•, and what it is that LLMs actually do:

    When medical professionals ask medical questions in technical medical language, the answers they get are typically correct.

    When non-professional ask medical questions in a perhaps medically ill-formed vernacular mode, the answers they get are typically wrong.

    The LLM readily models both of these things. Despite having no notion of correctness in either case, correctness is more statistically typical in one than the other.

    3/

    💬 6🔄 19⭐ 64

Replies

  • Paul Cantrellinthehands@hachyderm.io
    Feb 9, 2026, 9:38 PM

    RE: girlcock.club/@miss_rodent/116

    This is a different, crisper way of saying what I meant by the previous post: if it sounds like a medical textbook, you’re more likely get a diagnosis; if it sounds like a tweet, you’re more likely to get a shitpost

    The tone, vocabulary, and style of the question change the likelihood that the answer is correct.

    4/

    💬 1🔄 4⭐ 16
  • 💬 0🔄 0⭐ 0
  • Greg Whiteheadgrwster@mastodon.social
    Feb 9, 2026, 5:10 PM

    @inthehands I continue to be well-served by treating LLMs as fancy autocomplete and not anthropomorphizing them. I feel like the chat interface is where things went sideways, making it too easy to believe that they "think"

    💬 0🔄 0⭐ 9
  • Garrett Wollmanwollman@mastodon.social
    Feb 9, 2026, 5:56 PM

    @inthehands Worth noting, however, that when the training set captures a lot of outdated or irrelevant information, because the field has advanced rapidly since the model was trained, "typical" can start to diverge again. This can be mitigated if the practitioner knows to consult the latest information (either by reading it or by feeding it to the model as a part of the query) but of course they have to be aware of that. This is I suppose no worse than relying on the practitioner's knowledge.

    💬 1🔄 0⭐ 3
  • Garrett Wollmanwollman@mastodon.social
    Feb 9, 2026, 5:58 PM

    @inthehands OTOH, as practitioners come to rely on stochastic information retrieval for more and more diagnoses, as it confirms what they already know, it may cause them to assign more weight to the information in the model than is justified, overruling their own second thoughts. ("Computer says...")

    💬 0🔄 0⭐ 2
  • mirthmirth@mastodon.sdf.org
    Feb 9, 2026, 6:24 PM

    @inthehands One of the factors in this mess is the heavily-boosted notion that LLMs contain facts or knowledge. Coincidentally, sort of, but not really. A safer mental model is to think of them as a fuzzy virtual machine of sorts, not unlike a vibe-y JVM but programmed in something dressed as plain language. Garbage-in-garbage-out. Often anything-in-garbage-out.

    💬 0🔄 1⭐ 0
  • Matthias KrämerKraemer_HB@mastodon.social
    Feb 9, 2026, 10:21 PM

    @inthehands Interesting results. The credo of many specialized ChatBot firms is that "the human in the loop" is still needed, meaning an expert in their field can make the best use of support by an AI. I thought that was mainly due to the expert spotting hallucinations. But the expert making the more expert-sounding inputs what results in more expertish outputs is a new aspect of this.

    💬 0🔄 1⭐ 1
  • tuban_muzurutuban_muzuru@beige.party
    Feb 9, 2026, 10:14 PM

    @inthehands

    I use Claude in my IDE every day. The LLM can only return what it identifies as Appropriate.

    And the LLM will be the first to tell you so.

    Particularly good:
    >Despite having no notion of correctness in either case, correctness is more statistically typical in one than the other.

    💬 0🔄 0⭐ 0