Looking at Torvalds using generative AI for Linux, Andrew Tridgell using it for rsync, and a bunch of other very bright and techically very capable people using it to assist them with various tasks makes me think again about the time a very, very intelligent friend of mine suddenly fell into Scientology.

It started me thinking about why it is that clever people fall for these scams, and I have a theory.

I think clever people learn to trust their minds. They learn that their mind will give them correct answers without much effort, and that it will also show them flaws in arguments with the same ease. Because it generally does!

But it's also a weakness. If something comes long that doesn't trigger the automatic defences, the clever mind may let it in.

The cleverness needs to be combined with vigilance, and that cannot be left to an automated system. I think clever people tend to become less vigilant, because their mind has always stood them in good stead.

Once the scam is in it combines with the general human unwillingness to admit to error. Or even to re-evaluate things just in case they are not true. We have a whole complicated social construct called "science" designed to work around that, and not even that works all the time.

And here's the kicker: The scam that tells clever people how clever they are will have the greatest chance of success, because they ARE clever. No false modesty. The scam they fall for is wrapped in the truth that they are clever people, and presented as a thing that only clever people can truly appreciate. The scams that clever people fall for are built up of details, layers, interrelated things that appear to explain each other and support each other. That hide and distract from the basic falsehood at the bottom.

To avoid such a scam you must take a step back from the complicated, dazzling, fascinating, alluring detail. Look at the whole thing and ask what it is, why it is, who is it from? Where does it connect to the world? Who does it benefit, who does it harm? Is it too good to be true?

Asking those questions about generative AI gets you pretty obvious answers.