Login
You're viewing the lou.lt public feed.
  • DrYakdryak@mstdn.science
    Oct 24, 2026, 10:38 PM

    @jap this, by the way, is a good way to see the actual limits of nodern chat-bots: take a personnal hobby you're very good at and query the slop machine to extrude an explanation or answer for something non trivial that you know very well. Be prepared to be extremely disappointed. This is the same "quality" of "answers" the slop machines have been extruding each time you queried about something you're not fluent in, you merely didn't notice how much of a garbage the "answer" was.

    💬 9🔄 58⭐ 108

Replies

  • 💬 2🔄 11⭐ 27
  • Hyweldaycoder@toot.wales
    Oct 25, 2026, 10:56 AM

    @etp @dryak @jap this an example of the Bader-Meinhoff phenomenon.

    (I hadn’t heard the Gell-Mann effect until a couple of days ago, but have experienced it)

    💬 0🔄 1⭐ 2
  • Jaap Zeldenrustjozeldenrust@social.sdfeu.org
    Oct 25, 2026, 12:02 PM

    @etp
    TV Tropes was one of my formative websites, so I know this phenomenon as "Cowboy BeBop at His Computer".

    I like how both terms are nearly completely useless for conveying the idea. They both require explanation. Which is good.
    @dryak @jap

    💬 1🔄 1⭐ 4
  • 💬 1🔄 0⭐ 1
  • Jaap Zeldenrustjozeldenrust@social.sdfeu.org
    Oct 25, 2026, 3:25 PM

    @etp It's an anime (animated series), not a manga (comic book). I hear it's very good, considered a classic of the medium, but I've never seen it myself.

    💬 1🔄 0⭐ 0
  • Bruce Heerssenbruce@darkmoon.social
    Oct 25, 2026, 4:36 PM

    @jozeldenrust @etp
    It is very good, and one of the few animes that has an English dub that is as good or better than the original Japanese. But the real standout, IMO, is the soundtrack by Yoko Kanno and The Seatbelts.

    💬 0🔄 0⭐ 2
  • Kagan MacTane (he/him)kagan@wandering.shop
    Oct 25, 2026, 2:57 PM

    @jozeldenrust @etp @dryak @jap Not quite the same thing, though! "Cowboy BeBop at His Computer" is when someone gets something so hilariously wrong, it might as well be gibberish.

    The Gell-Mann Amnesia Effect is when you (anyone in general, not just you in particular) see someone pull a Cowboy BeBop at his Computer, recognize it as such because you know the topic, and then _continue to trust them_ about other topics anyway.

    💬 0🔄 0⭐ 2
  • Jaap Zeldenrustjozeldenrust@social.sdfeu.org
    Oct 25, 2026, 3:43 PM

    @kagan I think you're partially wrong here. It doesn't have to be (or even approach) gibberish to be "CBahC", it just has to be clearly incorrect to someone familiar with the subject.

    But you're also right that GMA includes how a reader proceeds to relate to the source, and CBahC doesn't. That's a real distinction.

    💬 1🔄 0⭐ 2
  • Kagan MacTane (he/him)kagan@wandering.shop
    Oct 25, 2026, 4:30 PM

    @jozeldenrust I just rechecked and yeah, you're right, I was overstating how wrong something has to be to qualify as CBahC. I was going off the trope naming example, where practically every word was wrong, but the article says it just needs to be egregiously and obviously wrong to someone who knows the material, in a way that makes it clear someone did hardly any fact-checking.

    Thanks for the correction. (I should have done my own fact-checking!)

    💬 0🔄 0⭐ 1
  • 💬 0🔄 0⭐ 0
  • argv[-1]argv_minus_one@mastodon.sdf.org
    Oct 25, 2026, 6:32 AM

    @dryak

    I actually did notice that mastodon.sdf.org/@argv_minus_o and of course this makes me wonder what *else* it's blatantly wrong about.

    And as I mentioned in that thread, it gave its blatantly wrong answers with the utmost confidence…while also adding a freaking security vulnerability to the code I asked it to fix.

    These kinds of bots are now being used to develop the operating system and much of the software I use. God help me.

    @jap

    💬 2🔄 2⭐ 11
  • No Fate: Butlerian Jihad Nowglitzersachen@hachyderm.io
    Oct 25, 2026, 12:18 PM

    @argv_minus_one @dryak @jap

    God helps only those that help themselves. <Taps his nick>.

    (Longer answer: There are two kind of Gods --- the old ones and the new AI Gods who once want to converge into Rokko's Basilisk. The new ones will be gone in the blink of an eye --- either sucked into the popping bubble or transcended, the interpretation depending on whether the speaker is a cult member or not. The Old Ones are still sleeping and we better pray it stays this way).

    💬 1🔄 0⭐ 1
  • argv[-1]argv_minus_one@mastodon.sdf.org
    Oct 25, 2026, 5:51 PM

    @glitzersachen

    Then God grant me the time and energy and tranquility to help myself, because I've been in constant crisis mode at both home and work for over a year. It's exhausting. How am I supposed to explore replacements for my whole entire computing environment while being threatened from 5 different directions?

    Not to mention that the alternative I choose could go full AI at any moment. Even Debian, which once held famously lofty ideals, has betrayed my trust.

    @dryak @jap

    💬 0🔄 0⭐ 0
  • DrYakdryak@mstdn.science
    Oct 25, 2026, 8:35 PM

    @argv_minus_one @jap Read that thread, yup. Your experience mirrors mine: wrong answers, missing the point, being excessively wordy, suggestion about stuffing the code with not very useful bits, etc.
    Once one starts throwing real-world problem at it, one quickly realises that these are - as name implies - just *language* models.
    They are just good at putting English words and bits of code that superficially sound nice and could seem to make sense in the context, but lack any actual value.

    💬 0🔄 0⭐ 0
  • Océaneoceane@syndicat.solidaires.org
    Oct 25, 2026, 6:37 AM

    @dryak @jap LLMs are good at providing precise answers on the surface of the topic you’re interested in at the moment, not at doing deep dives.

    For example, they’re good at debugging students-level R code.

    They’re good at doing research online for you and giving you sources. But that’s because the cyberspace is in a pretty bad shape. There are far-right platforms, paid platforms, techies platforms (static hosting/prose.sh/etc.), and there’s the Gemini protocol.

    💬 2🔄 0⭐ 0
  • No Fate: Butlerian Jihad Nowglitzersachen@hachyderm.io
    Oct 25, 2026, 12:21 PM

    @oceane @dryak @jap

    > good at providing precise answers on the surface of the topic you’re interested in at the moment, not at doing deep dives.

    1/ Counting teeth is not a deep dive.

    2/ "Precise". Nope. Plausibly sounding, yes. But so can any manipulative sociopath. And the sociopath does it on purpose, whereas the the LLM doesn't even know what it's doing. An LLM is the Dunning-Kruger effect manifested as a machine.

    💬 2🔄 0⭐ 5
  • Océaneoceane@syndicat.solidaires.org
    Oct 25, 2026, 2:58 PM

    @glitzersachen I’m sorry but you’re extremely condescending, and you seem completely clueless about the topic you’re discussing. Did you POST this from a Windows/Android/iOS device? Because these environments are extremely boring. I think we can sort you out.

    💬 1🔄 0⭐ 0
  • No Fate: Butlerian Jihad Nowglitzersachen@hachyderm.io
    Oct 25, 2026, 3:11 PM

    @oceane

    LOL. I don't think you have done the basic homework on AI (and I am not talking about LLMs) and read any of the more critical literature on the current type of LLM-based AI if your are taking offense at me refuting the use of the word "precise" and the idea that counting something is a "deep dive" into a topic.

    The "device" I am posting this from is not your business[1]. But, as a clue, in my days we called this a computer (or even more precisely a workstation). Do you want to know the exact brand and model? Do you seriously think this has some bearing on how AI works or how good it is?

    But I agree with you: I am condescending. In the face of (let's call it) badly informed opinions.

    {1] We also called what you are trying here an "ad hominem" attack. Look it up. It's no recognized as valid reasoning, usually.

    💬 1🔄 0⭐ 0
  • Océaneoceane@syndicat.solidaires.org
    Oct 25, 2026, 3:23 PM

    @glitzersachen I’m asking you what’s your environment because most people – even the kernel developers who belong to my union – generally don’t see the difference between free software and open source software. So the bad news are you’re not more knowledgeable about this topic than the kernel devs I’m talking with at my union.

    This isn’t an ad hominem attack. If you change your mind, you know where to find me.

    💬 1🔄 0⭐ 0
  • No Fate: Butlerian Jihad Nowglitzersachen@hachyderm.io
    Oct 25, 2026, 3:44 PM

    @oceane

    > even the kernel developers who belong to my union – generally don’t see the difference between free software and open source software. So the bad news are you’re not more knowledgeable about this topic

    You are aware you're answering in a thread about the failure of a certain chatbot to count teeth on a sprocket correctly, aren't you?

    Because all this kernel developers / awareness of the difference between Free and Open Software (which doesn't even matter here) certainly doesn't fit.

    And, BTW, since knowing the difference is literally my job: LOL to the square.

    > If you change your mind, you know where to find me.

    About what, exactly? About the idea that the system I *use* has zilch bearing on the question of how well AI works. This is getting downright surreal.

    💬 2🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • Océaneoceane@syndicat.solidaires.org
    Oct 25, 2026, 3:49 PM

    @glitzersachen No, I’m trying to address the fact that you’re answering my messages within 20-minutes delays in an absolutely non-constructive manner. If you’re doing that to yourself, that’s because you think you deserve it.

    💬 1🔄 0⭐ 0
  • No Fate: Butlerian Jihad Nowglitzersachen@hachyderm.io
    Oct 25, 2026, 3:51 PM

    @oceane

    People gotta eat sometime and do other things. Can't give you lower latency.

    And it's still ad-hominem, since you don't address the issues with which we started, but invent new supposed failures of your opposition.

    💬 1🔄 0⭐ 0
  • No Fate: Butlerian Jihad Nowglitzersachen@hachyderm.io
    Oct 25, 2026, 3:56 PM

    @oceane

    BTW, "non-constructive", yep that really hurt. Given how constructive "what device you are using, I am asking because people with those boring devices usually don't know anything" arguments are.

    I feel chastened. Totally.

    💬 1🔄 0⭐ 0
  • Océaneoceane@syndicat.solidaires.org
    Oct 25, 2026, 5:49 PM

    @glitzersachen So there are demographics of knowledgeable Windows users, but some people, like you and me, are far more knowledgeable when we use free software on top of POSIX systems.

    POSIX compliance isn’t the point. It's a prerequisite. I was asking you whether or not you used POSIX-compliant systems so you could use free software without having to twist your config around the WSL.

    And I don’t mean to hurt you. This conversation seems to hurt you and I promise you I’m not enjoying that.

    💬 0🔄 0⭐ 0
  • Zimmiebob_zim@infosec.exchange
    Oct 25, 2026, 3:47 PM

    @glitzersachen @oceane @dryak @jap It’s definitely precise. 36 is an exact answer, and prompting it again apparently returned the same answer. What it isn’t is accurate, even compared to itself.

    For basically all the things people want LLMs to do, error in accuracy is much worse than error in precision.

    💬 0🔄 0⭐ 0
  • No Fate: Butlerian Jihad Nowglitzersachen@hachyderm.io
    Oct 25, 2026, 3:49 PM

    @bob_zim @oceane @dryak @jap

    > It’s definitely precise.

    Damn. You are right[1]. Should have been "precises but inaccurate" answers then. Just shows that providing precise answers is not worth anything on its own.

    [1] I blame the fact that I am not a native speaker that I didn't notice this ;-).

    💬 1🔄 0⭐ 1
  • 💬 1🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • Ariaflameariaflame@masto.ai
    Oct 25, 2026, 1:08 PM

    @oceane @dryak @jap Judging from what my students submit they are NOT good at providing actual sources. They often make those up.

    💬 1🔄 0⭐ 1
  • Océaneoceane@syndicat.solidaires.org
    Oct 25, 2026, 3:08 PM

    @ariaflame I guess they make up sources at an academic level. If you ask them precise sources for a deterministic question, (I don’t use LLMs because I’m not a racist but) it looks like you need to verify them of course, but that they can point you to e.g. the correct open access datasets.

    I don’t endorse in any way LLMs. But there’s the consumption problem, and there’s the production problem. We’ve got shiny new tech to replace them, and actually produce the cyberspace, as a democratic space.

    💬 1🔄 0⭐ 0
  • Océaneoceane@syndicat.solidaires.org
    Oct 25, 2026, 3:09 PM

    @ariaflame If you merely consume democracy, if you don’t produce it, someone else will for you. In the context of the cyberspace, that’s Facebook. LLMs are a consequence of the centralization of our communications on mediocre, far-right, Twitter-like platforms.

    💬 1🔄 0⭐ 0
  • Ariaflameariaflame@masto.ai
    Oct 26, 2026, 2:48 AM

    @oceane I don't go on facebook, or twitter any longer. I live somewhere that voting is mandatory and ranked choice, so it's pretty good.
    LLMs seem to be a bit like genies. If you are skilled enough to ask exactly the right question, you might be getting a correct answer. But they are being used by a load of people who don't know how to ask the right questions, and there are cheaper, easier and less hard ways on the environment and communities ways to get the answer.

    💬 1🔄 0⭐ 0
  • Océaneoceane@syndicat.solidaires.org
    Oct 26, 2026, 11:18 AM

    @ariaflame I totally agree with you. I actually despise LLMs, but I think we need to be correct about them if we want to terminate them.

    By the way, this was an impersonal “you”. I apologize if this wasn’t clear: I was referring to your students.

    💬 0🔄 0⭐ 0
  • The Sleight Doctor 🃏🍉ApostateEnglishman@mastodon.world
    Oct 25, 2026, 6:40 AM

    @dryak @jap Or ask them a series of loaded/leading questions on a topic you know well, and observe them generate different, contradictory answers - effectively proving they they have no internal representation of the world against which to fact-check their own outputs. They're just confabulating whatever responses you're statistically most likely to expect based on how you phrased the prompt.

    In similar fashion, if you ask them to explain a nonsensical saying that you just made up, they will. 🤷🏻‍♂️

    💬 0🔄 3⭐ 14
  • 💬 0🔄 0⭐ 1
  • DrYakdryak@mstdn.science
    Oct 25, 2026, 4:16 PM

    @reflex @jap > are we looking at the same code man?

    I feel your pain. (Esp. as someone who professionally has to do long term maintenance of projects)

    💬 1🔄 0⭐ 2
  • 💬 0🔄 0⭐ 1
  • 💬 1🔄 0⭐ 1
  • 💬 0🔄 0⭐ 2
  • 💬 0🔄 0⭐ 0
  • imhotepimhotep@infosec.exchange
    Oct 25, 2026, 4:48 PM

    @dryak @jap Fairly solid method for finding the limits of modern humans too.

    Where the chatbots seem to do well is that right or wrong they will actually talk about stuff no one else gives a shit about.

    💬 0🔄 0⭐ 0
  • Very Human RobotStompyRobot@mastodon.gamedev.place
    Oct 25, 2026, 9:46 PM

    @dryak @jap The frontier models in high/ultra research mode are typically very good at being accurate for these questions. It's the free models and the automatic search summarizers that do slop. Unfortunately, most people don't pay for the good results, so that's something like a poverty tax right there...

    FWIW a typical human news journalist is also pretty bad at things you know something about. But I'm sure they are accurate with all the things I know nothing about....

    💬 1🔄 0⭐ 0
  • DrYakdryak@mstdn.science
    Oct 26, 2026, 7:55 AM

    @StompyRobot @jap The "Free models" of today are previous generation of "Frontier models" now made available for free to entice you to fork money for the latest iteration of "Frontier models".

    What we get for free now is what 6 months ago all the AI fans were saying is much better than the free tier and if we were paying for those model we would see how much better AI actually is.

    This has been going on for ages. The old is used for free as ads for the (incremental) improvement.

    […]

    💬 2🔄 0⭐ 0
  • DrYakdryak@mstdn.science
    Oct 26, 2026, 7:56 AM

    @StompyRobot @jap […]
    But for the last 1.5~2 years, I haven't seen any actual real improvement in the answers, jut more wordy answers and better style.

    Because improving those would require the slop machines to have an actual idea of what their talking about…

    …which they obviously lack. Because they are merely output words to word queries or, in the top poster's case, outputting pixels to a query of pixels.
    [...]

    💬 1🔄 0⭐ 0
  • DrYakdryak@mstdn.science
    Oct 26, 2026, 7:57 AM

    @StompyRobot @jap […]
    Up to a certain point it's possible to brute-force around those limitations by throwing insane amount of compute on gazillon-parameters large models, which could marginally improve on some corner cases.

    But we're way beyond the point of diminishing returns, and the problem of who's going to pay for this once the US corporations stop shovelling investors' money into it isn't solved. (And don't get me started on all the indirect cost on environment, culture, etc.)

    💬 0🔄 0⭐ 0
  • Very Human RobotStompyRobot@mastodon.gamedev.place
    Oct 26, 2026, 5:50 PM

    @dryak @jap
    No they are not. The free models are small distilled versions of the leading models.
    The compute power of a model is limited by both training and size, and this generations free models are still much smaller than last generations leading models.

    💬 0🔄 0⭐ 0
  • Deep Muddeepmud@mas.to
    Oct 26, 2026, 2:03 AM

    @dryak I simply asked how to take 3 percent hydrogen peroxide and make a 1 percent solution. AI said to mix with ten parts of water. 🫩

    💬 0🔄 0⭐ 0