You're viewing the mstdn.social public feed.

Federated feed Local feed

an actual busrenardboy@mastodon.social
Jun 24, 2026, 1:03 AM
@benroyce @hackersquirrel @emilymbender IME you need substantial computer muscle for it to run models large enough to be useful. Not that it can't be done, not at all, but it's more accessible if you already have a powerful rig for other reasons.
And then, it doesn't necessarily improve some of the issues. Since models need lots of ram, it's actually less useful to fill, say 32 gigs of ram to serve one user, times 10000 users, than it is to have 320 terabytes of ram serving 10000 users.
💬 1🔄 0⭐ 3

Replies

an actual busrenardboy@mastodon.social
Jun 24, 2026, 1:07 AM
@benroyce @hackersquirrel @emilymbender Cause the second solution allows use to split cpu time between users, whereas the first one has one cpu per user that spends most of its time idling. So energy-wise you're not necessarily winning either unless your use case involves AI running around the clock, but then while you might be more *efficient*, you're still using gobs more energy.
💬 2🔄 0⭐ 1
Ben Royce V͎O͎T͎E͎ ᶜʸⁿⁱᶜⁱˢᵐ ⁼ ᵃᶜᶜᵉᵖᵗᵃⁿᶜᵉbenroyce@mastodon.social
Jun 24, 2026, 1:34 AM
@renardboy @hackersquirrel @emilymbender
well distant future
like we walk around with computers in our pockets that are 10,000x what we used to send astronauts to the moon. in 50 years, we'll walk around with petabytes and petaflops in our pockets (or stuck in our heads... cyborgs)
💬 1🔄 0⭐ 2
an actual busrenardboy@mastodon.social
Jun 24, 2026, 1:38 AM
@benroyce @hackersquirrel @emilymbender Personally, my hope for better generative "AI" involves distributed models running on home servers that channel their heat usefully.
You'd have a bunch of servers with water cooling that feeds into appliances like hot water heaters, and each would hold some focused fragment of a large language model, and when you'd prompt you'd have some kind of router language model that'd send it to whoever is likely to have an answer.
💬 1🔄 0⭐ 1
an actual busrenardboy@mastodon.social
Jun 24, 2026, 1:40 AM
@benroyce @hackersquirrel @emilymbender I don't dare to expect this but I allow myself to hope.
It's not that far-fetched as I understand, there are existing LLMs that are internally structured as components, they are called "Mixture-of-Experts" (MoE) LLMs.
💬 1🔄 0⭐ 1
an actual busrenardboy@mastodon.social
Jun 24, 2026, 1:42 AM
@benroyce @hackersquirrel @emilymbender all the energy use would displace energy normally used simply to run resistive heaters in hot water heaters. You'd have models sized for a number of average power consumptions (and corresponding heat outputs) to match with water use patterns of different buildings.
💬 1🔄 0⭐ 2
Ben Royce V͎O͎T͎E͎ ᶜʸⁿⁱᶜⁱˢᵐ ⁼ ᵃᶜᶜᵉᵖᵗᵃⁿᶜᵉbenroyce@mastodon.social
Jun 24, 2026, 1:58 AM
@renardboy @hackersquirrel @emilymbender
AI as water heaters is very Matrixy. I like it
💬 0🔄 0⭐ 0
Marsh Raymarshray@infosec.exchange
Jun 24, 2026, 4:11 AM
@renardboy @benroyce @hackersquirrel CPUs and GPUs slow down or even shut down and consume far less energy when they’re not actively crunching numbers.
E.g. a phone or tablet may have peak performance numbers surprisingly competitive with a desktop, yet the battery could last multiple days if left idle.
💬 0🔄 0⭐ 1