Login
You're viewing the mastodonapp.uk public feed.
  • Gabriel Pettiertshirtman@mas.to
    Oct 25, 2026, 3:10 PM

    @alan @veronica @pluralistic For your first use case (completion), yes, qwen models from last year did that pretty well, very small distillations of 3B or less, did useful work, but that's still depending on you validating each suggestion.

    For agentic coding, the code is not generated first try reliably, you need the agent to be ble to run the code, see breaking tests, syntax errors, etc, and self-correct.

    I have been disapointed so far with the ability of local models, even when it works, it's too slow, though i had a quick test of the newly released "flint" on friday, that runs at 50 tokens/s on my macbook pro M2 max, and reportedly a lot faster on an M4, it looks good enough to attempt actual work, in replacement of claude.

    Regarding productivity, i think closed/frontier model are good at building things very fast, but the bottleneck of choosing what to build, and making sure we understand what we have, don't really go away.

    💬 1🔄 0⭐ 1

Replies

  • Alan Langford 🇨🇦🧤🧊摏alan@mindly.social
    Oct 25, 2026, 3:49 PM

    @tshirtman @veronica @pluralistic My personal feeling is that it should be starkly obvious that the current algorithms are woefully inefficient. In fact, so inefficient that no computer scientist would say that it's reasonable to deploy them commercially.

    Unfortunately, a bunch of entrepreneurs and VC's seem to have missed or ignored this fact and we've taken this path of building it out instead of waiting for reasonably efficient innovations.

    As a result we have this looming, massive bubble that when it bursts, in addition to the massive real economic harm it will do, will set back the theoretical development, research that could discover more efficient approaches, for years.

    💬 1🔄 0⭐ 1
  • Gabriel Pettiertshirtman@mas.to
    Oct 25, 2026, 4:00 PM

    @alan @veronica @pluralistic Inefficient compared to what? I agree they are energy intensive, but they are doing a thing we didn't know how to do before, which is to have computers write, operate, improve, verify programs from natural language prompts (not even just English, but about any human language sufficiently documented).

    And while they are indeed energy intensive, the efficiency has been increasing a lot over the last couple years, although it's hard to get clear data, the companies are secretive about it, the massive energy costs seems to be mostly on the training on newer, bigger models, not producing output from the existing ones, and we can run relatively large models, that are doing useful work, today, on customer hardware (admittedly high end).

    I think what the VC have missed, is that they won't be able to capture the value created, because their product will be commoditized and run for cheaper on hardware they don't own.

    I agree the bubble popping will hurt, yes.

    💬 1🔄 0⭐ 0
  • Alan Langford 🇨🇦🧤🧊摏alan@mindly.social
    Oct 25, 2026, 5:35 PM

    @tshirtman @veronica @pluralistic I would use as a baseline for efficiency comparisons, the power consumption of a human brain. I don't expect we'll reach that anytime soon, but it seems clear that we're at least a few orders of magnitude away from that at the moment. However we're far enough away that I expect some big steps in closing that gap are not that far out of reach.

    I think that's an excellent point about model commoditization. Maybe that's a disincentive for VCs to seek more efficiency.

    💬 1🔄 0⭐ 0
  • Gabriel Pettiertshirtman@mas.to
    Oct 25, 2026, 5:54 PM

    @alan @veronica @pluralistic interesting point, but human brains sadly don't run on solar panels, and feeding us is pretty energy intensive, given all the loops energy has to jump through before it gets to occasionally allow a brain to fix issues in a program, i'm not sure the comparison is as clear cut. Also, getting more brains apt at this particular task is a very lengthy and energy intensive process, while spawning a new inference context for a model is very straightforward, if still not cost-free, and that model will work at speed i can't imagine humans rivaling with.

    I'm vegan, so i imagine my brain is a bit less intensive in primary-energy to run, as eating plants is more efficient, but still, a frontier model doing what used to take weeks, in hours if not minute, gives a solid ratio to start with to the machine.

    I'm all for making things more efficient though, and i think it'll happen whether VCs want it or not, competition is a rough mistress.

    💬 1🔄 0⭐ 2
  • Alan Langford 🇨🇦🧤🧊摏alan@mindly.social
    Oct 25, 2026, 6:08 PM

    @tshirtman @veronica @pluralistic Then to be fair in that wider estimate you need to include the energy cost of extracting, refining, packaging, shipping, and replacing the components in an AI data centre. I think that balances it out to some extent, even when ignoring the environmental impact of those activities.

    Excluding all these inputs, can these processes become more efficient than the human brain? My guess is that they can, and that's when concerns of AGI become much more pertinent. It also illustrates just how far I think we are away from getting there.

    💬 0🔄 0⭐ 0