Login
You're viewing the mastodon.coffee public feed.
  • David Chisnall (*Now with 50% more sarcasm!*)david_chisnall@infosec.exchange
    Aug 14, 2026, 8:23 AM

    RE: mastodon.world/@anttipeltola/1

    I really like the Pentium 4 as an analogy here.

    The NetBurst architecture in the P4 was designed to be able to scale to 10 GHz. It initially shipped in the 1-2 GHz range. The performance was not that impressive compared to even the previous generation but, Intel told us, that didn’t matter: it would rapidly reach clock speeds that were completely impossible with the P3.

    On the face of it, this wasn’t unreasonable. The Pentium Pro and Pentium 2 shared a fairly similar microarchitecture (P2 added MMX and a few other things). The Pentium Pro was introduced at 150 MHz and scaled up to 450 MHz after process shrinks.

    When the Pentium 4 was in early design stages, it was completely reasonable to expect that 3-5x clock speed increases were possible.

    And, technically, they were. The clock speed for a part is determined by how long it takes a signal to propagate across everything in a pipeline stage. Make the things smaller and the signal goes further.

    That kept happening. The thing that stopped (completely by about 2007, but it wasn’t an abrupt thing) was Dennard Scaling. Beyond a certain point, making the transistors smaller meant you had a lot more leakage current and so the power went up. And power was already polynomial in terms of clock speed.

    All this meant that, yes, in theory, you could run a Pentium 4 at 10 GHz, as long as you didn’t mind all the fire. Someone did eventually run one at this speed with liquid nitrogen cooling, but that wasn’t appropriate for a consumer device. The range topped out at 4 GHz, and those parts had infeasible high cooling and power requirements (and were outperformed by other designs at lower speeds).

    The entire ‘AI’ bubble is predicated on the same assumptions of scaling of something foundational that the people building the end systems are not really paying attention to.

    💬 8🔄 145⭐ 125

Replies

  • 💬 0🔄 0⭐ 3
  • wraptilewraptile@fosstodon.org
    Aug 14, 2026, 9:36 AM

    @david_chisnall I think people were aware that transistor size will be a limiting factor in this example, so the limit was well known. Also this is physical limitation not a software one, LLMs are mostly just information (compression/code).

    💬 2🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • wraptilewraptile@fosstodon.org
    Aug 14, 2026, 9:43 AM

    @david_chisnall yes but people in P4 timeline were aware of the size of an atom and how small the transistor could get. It was not some new physics we discovered. So everybody knew that size will be limiting factor, probably much much before P4 was created.

    We don't have such thing with software scaling. There's no physical limit we can deduce for certain.

    💬 1🔄 0⭐ 0
  • David Chisnall (*Now with 50% more sarcasm!*)david_chisnall@infosec.exchange
    Aug 14, 2026, 10:18 AM

    @wraptile

    I don't understand the point that you're trying to make. The fact that transistors can't be made smaller than an atom was known at the time, but it was predicted that they could be made smaller at a specific rate. That prediction was met. The size of transistors throughout the P4's lifetime shrank almost exactly as expected. That was not the problem. As I explained in the original post in this thread.

    💬 1🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • David Chisnall (*Now with 50% more sarcasm!*)david_chisnall@infosec.exchange
    Aug 14, 2026, 11:13 AM

    @wraptile

    Both were assuming that something would keep scaling as it had previously. In the case of the P4, it was assuming that Dennard Scaling would continue and power consumption wouldn't be a factor when running at higher clock speeds. In the case of LLMs, it's assuming that adding more parameters increases utility. Both were based on extrapolating from insufficient data points and both were shown to be incorrect as the technology evolved.

    💬 1🔄 0⭐ 0
  • NetravenNetraven@hear-me.social
    Aug 14, 2026, 11:52 AM

    @david_chisnall @wraptile

    Wow, is this even a real conversation? Am I asleep right now? I can't tell and I just took mental damage from trying to read this. Wraptile just... have a snickers... or something.

    💬 0🔄 0⭐ 1
  • wraptilewraptile@fosstodon.org
    Aug 14, 2026, 12:08 PM

    @david_chisnall sure but these are clearly differently constrained scenarios so what's the value in this analogy? It's like saying "fishing will not scale because we'll run out of fish" and "prose won't scale because we'll run out of fictional ideas" - both are kinda true but the comparison doesn't really add anything.

    💬 0🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • Ken Milmorekbm0@mastodon.social
    Aug 14, 2026, 9:46 AM

    @david_chisnall I seem to remember the P4 design was also intended for use with RAMBUS memory which got rejected by the rest of the industry. Intel were reluctant to go DDR I think?

    💬 1🔄 0⭐ 1
  • David Chisnall (*Now with 50% more sarcasm!*)david_chisnall@infosec.exchange
    Aug 14, 2026, 10:23 AM

    @kbm0 Yes, Intel thought RAMBUS memory was the future. That wasn't really a problem for performance, but it was a big problem for cost early on. RAMBUS memory cost a lot more. I remember someone I knew buying a P4 machine and boasting about the clock speed. My Athlon at 75% of the clock speed had twice as much RAM, cost less, and outperformed it in every benchmark we ran.

    💬 0🔄 0⭐ 2
  • JavensbukanJavensbukan@cosocial.ca
    Aug 14, 2026, 9:53 AM

    @david_chisnall Here, have a Dell Optiplex GX 270 as a prize.

    *LOUD FAN NOISES* *IS INCREDIBLY HOT*

    My high school was loaded with these boat anchors - absolute trash.

    Showing a mid 2000s flat mini tower PC in a dark grey clamshell style case in its side
    💬 2🔄 0⭐ 3
  • 💬 0🔄 0⭐ 1
  • 💬 1🔄 0⭐ 1
  • 💬 0🔄 0⭐ 1
  • 💬 0🔄 0⭐ 2
  • twosky2000twosky2000@mastodon.social
    Aug 14, 2026, 10:42 AM

    @david_chisnall yeah feel the same. But it's even worse with AI I think, I see AI more in heuristical landscape, the optimum is not 100%, but 80%. Not sure to explain it more and the terminology, but interested of learning more on a general level(not specific llm techs).
    More get a reconable anwser what is to much in different areas like programming and communication. My experience is that it get's out of control as soon as AI is integrated, not just a rubberduck. So AI rejection is reasonable.

    💬 0🔄 0⭐ 0
  • David Chisnall (*Now with 50% more sarcasm!*)david_chisnall@infosec.exchange
    Aug 14, 2026, 10:55 AM

    @twosky2000

    The 80% number is pretty close to the worst possible case for the Paradox of Automation.

    If a task requites effort but a machine can get it right 20% of the time, then you get a pretty good improvement in productivity: you check the result and most of the time you reject it and do it yourself, but when the machine did a good job you move quickly onto the next task. This benefit goes up a bit as the machine's efficiency improves but there's a threshold at which it's hard to pay attention to checking because it's right most of the time, and so you start letting through things that you wouldn't let through if you either did it yourself or if the machine were less reliable and forced you to check more carefully.

    This is closely related to something I've said many times before: Machine learning techniques are great for situations where the cost of a wrong answer is low and the benefit of a correct answer is high. The problem with LLMs is that very few things involving text actually fit in this category.

    💬 0🔄 0⭐ 1
  • 💬 1🔄 0⭐ 0
  • 💬 0🔄 0⭐ 1
  • Santiago 🔭🪐santiago@mastodon.uy
    Aug 14, 2026, 11:01 AM

    @david_chisnall I get the point, but i think it is a bit too harsh for the Pentium 4. The Netburst architecture introduced the SSE2 instructions and branch prediction algorythms, that all CPUs use today although better ones. Also the extreme heat produced by the CPU required mandatory forced air circulation, that all CPUs need today, Its not like the CPUs went back to run cool again after ditching Netburst. And also the Pentium 4, although expensive, it was superior to its AMD competitors in almost every benchmark. It got viewed as a failure because off course it could not deliver 10GHz speeds, but it introduced many things common today in CPUs. The same with the AMD FX, the big AMD failure, given that those CPUs still run fine today, pretty much future proof and also introduced many new instructions sets on their own. The AI is a different kind of failure

    💬 1🔄 0⭐ 0
  • David Chisnall (*Now with 50% more sarcasm!*)david_chisnall@infosec.exchange
    Aug 14, 2026, 11:30 AM

    @santiago

    The Netburst architecture introduced the SSE2 instructions and branch prediction algorythms, that all CPUs use today although better ones.

    The P4 wasn't the first chip to do branch prediction, it wasn't even the first Intel chip to do branch prediction. A few of the techniques it used were taken from elsewhere. The reason the P4 talked so much about branch prediction was that it had far more instructions in flight at a time than any prior Intel chip. With around 180 instructions in flight at a time and with an average of one branch per seven instructions (still roughly true today), the P4 needed to correctly predict 25 branches ahead to keep the pipeline full. A branch predictor accuracy of 98% (which is fairly easy) gives you a probability of an incorrect prediction of over 40%, which would absolutely kill performance. Most modern CPUs depend at least as much on branch prediction but use completely different techniques.

    SSE2 was a fairly incremental set of extensions. Double-precision floating-point was obviously useful but not exactly novel. Nowhere near as big a jump as from MMX to SSE. The main reason that we still talk about SSE2 today is that the original Opterons included a lightly extended SSE2 (more registers), which meant that it could be part of the ABI for 64-bit systems, in the same way that x87 was for 32-bit x86.

    And also the Pentium 4, although expensive, it was superior to its AMD competitors in almost every benchmark

    Only if you didn't measure performance per Watt or performance per dollar. And they were trounced by Opterons once they came out. And that's including Intel being one entire process generation ahead of AMD at the time.

    The chief architect on the P4 has given some good retrospective talks on it. I was in one about 20 years ago when the project was still fresh in his mind. He said, in particular, that it was the last really fun CPU to design because it was the last one that wasn't designed with power as a key optimisation target. He also said that it was one generation after the last one that should have been designed without power as an optimisation target and the failure to realise that is a big part of why AMD was able to massively jump in market share during the P4 era.

    💬 1🔄 0⭐ 0
  • Santiago 🔭🪐santiago@mastodon.uy
    Aug 14, 2026, 12:45 PM

    @david_chisnall Thanks for remind me the fact, that P6 were the first ones using branch prediction, tho P5 also had that right? I remember the SSE2 was a fairly big deal to crunch multimedia, encoding and such. Anyways, processors are like car engines, there are better ones, efficient and others not so much.

    💬 0🔄 0⭐ 0