Cassandrichdalias@hachyderm.io
Jun 12, 2023, 12:38 AMI think a lot of ppl who are skeptical of the criticisms here really don't understand how it's *burning* anything because they haven't thought about how value is derived from provenance.
Replies
- Cassandrichdalias@hachyderm.ioJun 12, 2023, 12:41 AM
Suppose you have a $100M batch of medicine but an inside saboteur has put poison in a few bottles and you can't determine which ones. How much is your inventory worth? $0 - or less since you have to dispose if it too. This is because the value depended on the provenance - having a reasonable basis to believe it's what it appears to be.
- Cassandrichdalias@hachyderm.ioJun 12, 2023, 12:44 AM
For a lower stakes example, why is the organic produce in the grocery store more expensive? If you took the labels off and mixed it all together with the rest, it wouldn't be. The value at sale time is derived from a meticulous record keeping process that makes faking provenance comparable in cost to just doing it right.
- Cassandrichdalias@hachyderm.ioJun 12, 2023, 12:50 AM
Provenance of human written knowledge comes from a lot of places. Just because something was written by a human doesn't make it accurate or non-garbage. But the labor cost in producing misinformation that's hard to distinguish from meaningful writing in the same domain, together with a lot of systems we have in place, makes evaluating provenance a tractable problem.
- Cassandrichdalias@hachyderm.ioJun 12, 2023, 12:51 AM
The ability to produce unlimited amounts of plausible-looking garbage at essentially no cost, and to crowdsource that kind of vandalism to millions of randos by disguising it as something fun, destroys that capability. It's a DDoS attack on written knowledge.
- Cassandrichdalias@hachyderm.ioJun 12, 2023, 12:55 AM
I've seen folks arguing that good and accurate info can come out of "AI" too, so we can't dismiss it as garbage.
This misses the point entirely.
Even if "AI" "says" something 100% accurate, the provenance is still garbage. It's like a broken clock. It's like waiting for the nazi to say something non-offensive and saying "wow they're at least right about some things".
- Cassandrichdalias@hachyderm.ioJun 12, 2023, 1:05 AM
And quite coincidentally, all this comes right at the moment Elon burned down the greatest tool we had for realtime vetting of information across a vast range of knowledge domains through a shared domain-knowledge trust graph. We're still a long way from rebuilding that here.
- Cassandrichdalias@hachyderm.ioJun 12, 2023, 2:57 PM
So how do we move forward? We can't entirely put this shit back in the shitter. The models are large but tractable for bad actors to keep and continue using even if we somehow banned them.
But there's a lot we can do...
- Cassandrichdalias@hachyderm.ioJun 12, 2023, 3:00 PM
We can recognize the people pushing this shit as charlatans, enemies of humanity, rather than the geniuses they want us to see them as. We can stop falling for their scam of the day. We can organize to tear down their power, devalue their wealth, deliver them consequences.
- Cassandrichdalias@hachyderm.ioJun 12, 2023, 3:02 PM
We can preserve and strengthen our standards for provenance of knowledge. In particular, open and community based projects that deal with knowledge can clearly and unequivocally ban model-generated bullshit and users who try to sneak it in.
- Cassandrichdalias@hachyderm.ioJun 12, 2023, 3:05 PM
A lot of the problems with provenance of knowledge didn't begin with LLMs. LLMs just greatly accelerated the threats that fashy political aspirations and capitalism/adtech already incentivized. LLMs make the defender's role a lot harder, but not fundamentally different.