1
2

as returns to investing in production decline due to demand saturation, engineering gets replaced with manifesting. the only thing left that people don't already have is status, and the way to get that is to form a highly regarded cult. revivalist preachers wear the trappings of industrialists. MLM schemes LARP as work.

I've thought this for a while but I think it's even more true with the platforms getting clogged with slop; the way to make money from art isn't to sell the art it's to make money from the community around the art. you might think your job is a cartoonist or a novelist but you're actually a professional discord mod

Former xAI executive Uday Ruddarraju, who was promoted to chief technology officer of computing capacity at OpenAI this summer, is now set to take over the company’s computing strategy. He reports to President Greg Brockman.

more openai exec churn https://www.wsj.com/tech/ai/openai-computing-chief-to-depart-6ffb7d7e?eafs_enabled=false

right so an LLM is a random walk. that's not an analogy, that's literally what an LLM is. it gives you the probability of all the next context windows with a new token at the end given the current context window, and you generate text by picking next context windows repeatedly using those probabilities until the context window is full of the amount of output you want. that's a random walk over the space of context windows in the same way that brownian motion is a random walk in the space of particle positions. like all random walks we can talk about its stationary distribution, which is the probability that an infinitely long random walk is at each point in the state space. since our state space is the space of context windows, that assigns a probability to each context window. most context windows are low-probability gibberish; there's only a relatively small region of the state space that's higher probability real text. because gibberish is low probability there's also very little training data out in that part of the state space, so all possible next states are more or less equally low probability. so once a random walk finds itself in a low probability part of the state space everything after that point will be gibberish. it "fell off the [stationary] distribution" in the sense that it fell off the edge of the map. you're in low probability, no training data terra incognita.

if I say an LLM "fell off the distribution", do you know what I mean?

I still think if you really want to use an LLM to generate code the way to do it is to use a diffusion language model and a set of property-based tests, and jointly MCMC sample from the output space of the LLM (which is what diffusion is) and from the output space of the properrty-based generator. ie you're searching for a high likelihood piece of text/code that has a low likelihood of containing a failing case

big study on claude code adoption using fine-grained data from over 700 firms. zero productivity improvement https://fion.ac/jellyfish.pdf

we've had post-scarcity for 30 years already

who still uses GNOME?

no one is maintaining it because facebook captured all the old maintainers and locked them in a dungeon. the plus side of that is that (unlike onnxruntime) it isn't super buggy now because of slopification. the minus is that there's code rot that you have to fix to get it to build

nnpack doesn't build anymore :/

if you read enough Lem goodreads decides your native language is polish

sometimes you want to have a copy of all human knowledge, you know, just in case stuff starts getting locked down

does anyone sell copies of commoncrawl on ultrum tapes? downloading it takes forever and you probably have to buy more storage to store it anyway (it's on order 100TB)

Paradise Valley

the people who are excited about LLMs and use them a lot generally come in three flavors:

1. people who want the social status of being someone who makes a certain thing (writer, artist, software developer) but they don't care at all about the thing. the sort of people who would pay to have a book published and written by a ghostwriter that says nothing about nothing, just so they can identify as "an author"
2. people who have things they want to do that are actually extremely easy and don't even require making anything (the thing they want already exists on the internet), but they don't know that and they feel like they need someone's permission to look. LLM hype is implicitly giving them that permission.
3. people that just really like jumping on bandwagons, whether that's AI or NFTs

don't underestimate how much mileage you can get from generating your own tokens for your particular type of data