A few years ago, I was working on a server chip with confidential computing support. We had a fairly big list of things that could undermine the guarantees if we got them wrong and so needed careful auditing before the chips shipped. I can’t help wondering if AMD stole that document and treated the potential bugs as a to-do list.
Sleeping with the balcony door open last night and the friendly kitten decided to come in at 5:20am.
Me (woken up by a loud noise, looking across the room): Cat!
Me (a few seconds later, seeing the cat on the bed): Cat on the bed!
Partner (waking up, opening eyes, field of vision full of cat): Aaah!
Still haven’t quite managed to explain to friend kitten that it is an outside friend.
RE: https://mastodon.world/@anttipeltola/117092344692403567
I really like the Pentium 4 as an analogy here.
The NetBurst architecture in the P4 was designed to be able to scale to 10 GHz. It initially shipped in the 1-2 GHz range. The performance was not that impressive compared to even the previous generation but, Intel told us, that didn’t matter: it would rapidly reach clock speeds that were completely impossible with the P3.
On the face of it, this wasn’t unreasonable. The Pentium Pro and Pentium 2 shared a fairly similar microarchitecture (P2 added MMX and a few other things). The Pentium Pro was introduced at 150 MHz and scaled up to 450 MHz after process shrinks.
When the Pentium 4 was in early design stages, it was completely reasonable to expect that 3-5x clock speed increases were possible.
And, technically, they were. The clock speed for a part is determined by how long it takes a signal to propagate across everything in a pipeline stage. Make the things smaller and the signal goes further.
That kept happening. The thing that stopped (completely by about 2007, but it wasn’t an abrupt thing) was Dennard Scaling. Beyond a certain point, making the transistors smaller meant you had a lot more leakage current and so the power went up. And power was already polynomial in terms of clock speed.
All this meant that, yes, in theory, you could run a Pentium 4 at 10 GHz, as long as you didn’t mind all the fire. Someone did eventually run one at this speed with liquid nitrogen cooling, but that wasn’t appropriate for a consumer device. The range topped out at 4 GHz, and those parts had infeasible high cooling and power requirements (and were outperformed by other designs at lower speeds).
The entire ‘AI’ bubble is predicated on the same assumptions of scaling of something foundational that the people building the end systems are not really paying attention to.
If you applied the same logic to Count Binfaces elections as is being applied by boosters of the AI bubble, then good news: he’s on track to win the next election he stands in by a majority of over 1000%.
The problem with spending so long working with capability-based security is that you read announcements about things like 'expanding capabilities' or 'wide-ranging capabilities' and you think 'someone needs to explain the principle of least privilege to these clowns'.
If you want to understand how CHERI works and the relationship between memory safety and compartmentalisation, and why starting from a programmer model is essential if you want to build useful hardware, here you go!
I wish people wouldn’t have screaming arguments at 6:45am while I’m trying to sleep.
And, by ‘people’, I mean ‘cats and herring gulls’.
And now the Sun has been eclipsed by a tree. Arboreal eclipse!
Starting to suspect the Moon is a hoax.
This article starts with a story from someone who tried Claude Code and found it amazing, but then switches to the same person a few months later seeing what a disaster it’s been. This quote is key:
The problem is that code produced by an AI agent looks reasonable, but can contain ‘hard-to-spot bugs’ that end up causing major problems
LLMs, by their nature, generate statistically plausible output. That is often a set that overlaps with correct output. But the things that are not correct look exactly the same as the ones they are.
Learning to review code is hard. You look for the bugs that you expect to be possible implementing it by using your theory of mind for the person writing the code and the kinds of things that they might overlook (not necessarily a specific person, but the kinds of things people miss) and also common bug classes.
And the big help is that the person writing the code is not thinking adversarially. They are not trying to sneak bugs in. Normally. Unless they’re a supply-chain attacker, and we’ve a depressing amount of evidence that code review doesn’t catch supply-chain attacks.
An LLM is not trying to do anything. It has no intent. But it is a machine that is trained on code that made it past code review. The kind of bugs that it will generate are ones that look like code that appeared in production. This is exactly what an attacker would do: try to write code that looks correct but is subtly wrong.
I’m only being slightly flippant when I say LLMs are a mechanism for brining supply chain attacks in house.
One of the video game tropes I find most annoying: You are a soldier in an elite unit, with the best equipment your employer can possibly provide. The first low-level enemy grunt will drop a better weapon than anything you are carrying.
The absolute worst offender for this was The Witcher 2, where I started the game with two swords. One was given to me by a king, from his personal collection. The other was given to me by a god. Enemy generic infantry dropped better swords than both within the first few minutes.
Today's instalment of 'people who criticise #Signal should understand why Signal made specific choices and provide better solutions to those problems':
Signal uses a phone number to sign up (boo, PII!). This serves three primary purposes:
The one I don't really like is that it limits account-creation rate for antispam purposes. This is largely a problem because Signal conflates identifiers (your phone number or username) with capabilities: anyone who has your identity can contact you, there's no way of saying 'this person may add me but does not have any token that allows them to pass on the ability to contact me'. And this is a thing that should be table stakes for any messenger, but isn't something I've seen done well anywhere.
More importantly, it enables discovery. In a world where most people used SMS, WhatsApp grew rapidly by reusing this identifier. If you had a contact list of people you used SMS to talk to, WhatsApp would automatically create a contact list with everyone who was already in your contact list. That's how WhatsApp grew to a billion users in a few years. Apple's iMessage copied this flow. Signal added some attempt at making this privacy preserving (though the space of valid phone numbers is so small that it's not hard to exhaustively enumerate it) and ideally they'd have some protocol so that you could find people by phone number only if you, specifically, were allowed to (some kind of zero-knowledge system that checked that both of you had the other in their contacts).
This isn't just a marketing things. Growing the network rapidly is important for privacy. Privacy cares a lot about the concept of an anonymity set: the set of users that might be mistaken for you. If you're using something like XMPP, with a one-user server, your anonymity set is just you: any message from your server comes from you. If you're using a server with a hundred users, it's fairly easy to correlate messages sent from a client to the server with messages sent from the server elsewhere (even if they're end-to-end encrypted) because normal humans don't send messages all of the time and so there will be moderately long periods (seconds, if not minutes) where you're the only one sending messages. If you're on a server with a million users, there will be very few times when you're the only person sending a message and so this kind of correlation is hard. But, since Snowden, we know that at least one (and I would be shocked if it is exactly one) nation state is routinely running this kind of correlation, so this is something that any private messenger needs to consider in scope.
What about other identifiers? The theory with XMPP was that email providers would also offer XMPP and you'd use the same identifier for both. Yay! Except most email providers didn't (except, briefly, gmail), so now you had two different identifiers that looked the same for reaching you. And here's the sad fact: most people don't have email addresses in their systemwide address book. They have mail clients that remember people they've contacted before. Unless they give your app access to their email account (and they absolutely should not be encouraged to give messenger apps access to their email), you can't get a list of email addresses for contacts (and, if you do, it will also include a load of companies and probably spammers). So it doesn't work as a discovery mechanism.
The final one is recovery. If you can't log into Signal, you can reacquire the account with a SIM and a passcode. Getting a replacement SIM is normally easy. You could do this with email but most people have free email from a single provider. A phone number is something that you pay for but the free emails come with no guarantee of service and can be cancelled at any time without giving reason. Email-based recovery is nice to have as an alternative option but for most people it's strictly worse (in fact, their email address probably has an SMS-based recovery flow!).
So, if you want to criticise Signal for using phone numbers, please do! I don't like it. I think there are better solutions to these things.
But don't pitch your favourite messenger as better than Signal because it doesn't need a phone number unless you come with an explanation of how you solve those problems. And if you do have a better solution to those problems, don't bother comparing to Signal, tell me about why your messenger is amazing because it has good solutions to those problems!
Rather than putting Rust in the kernel, I'd love to see FreeBSD build infrastructure for talking to RedoxOS drivers (which run in userspace already). Adding more Rust code to the kernel is maybe better than adding more C code to the kernel, but removing code from the kernel is even better.
We really don’t talk about election interference enough. #CountBinface is moving his spaceship to blot out the Sun as a show of strength the evening before the election and the mainstream press is running cover stories.
Yay, engineering!
The friendly kitten next door is now old enough that he has learned to teleport. His brother, fortunately, still moves around with traditional Newtonian methods.
Does anyone have experience with modern DLP projectors? The bulb in my current projector is approaching end of life and I’m pondering replacing the projector with something brighter. Modern LEDs should generate less heat than my current LED projector’s halogen. The last DLP projectors I used had an annoying effect where you’d get rainbow patterns if your eyes moved quickly: is that still the case with more modern ones?
This story should terrify any open source project or company that allows LLM-generated code into their project.
Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.
When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.
Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.
Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.
Our patent attorney just referred to the parts of a computer that interact in the system that we're drafting a patent for as the 'antagonists' and, honestly, he's not wrong. I am now going to refer to all of the parts of an SoC as 'the various antagonists'.
UK folks: If you are organising a petition that you want to be presented to the government and you use any platform that isn't https://petition.parliament.uk, then I assume it's just a data-harvesting organisation. I don't care if it's some charity that hasn't had all of their systems compromised (or not simply sold the data to Facebook) yet. I don't care that 48rabbitfeet or whatever is a really good organisation that's totally trustworthy and promises not to sell this valuable data to anyone.