gaytabasedysfun@treehouse.systems
Oct 14, 2026, 9:50 AMif you see a comment saying a test or script has been generated by LLM in a github repo, how does that make you feel about the repo it's in?
Replies
Philippa Cowderoyflippac@types.plOct 14, 2026, 9:55 AM@dysfun unless I see a "tests only and no fs access" policy with matching reviews, it's potentially all slop - and a script that isn't eg "build a filter out of these other filters in the app" means it's all slop until proven otherwise
- gaytabasedysfun@treehouse.systemsOct 14, 2026, 9:56 AM
@flippac this one is a benchmark with a comment at the top of it. as it happens, it's the first file i looked at, which isn't encouraging.
- Philippa Cowderoyflippac@types.plOct 14, 2026, 9:58 AM
@dysfun slop benchmark = untrustworthy project, end of
- gaytabasedysfun@treehouse.systemsOct 14, 2026, 9:59 AM
@flippac what if i told you it's not the benchmarks that went into the paper? it's testing the performance of the pext primitive used by the HOT they compare against.
- Philippa Cowderoyflippac@types.plOct 14, 2026, 10:01 AM
@dysfun then I might not seek to have the paper retracted without actual data of my own ;)
but extremely in "if you don't know how to do it adequately yourself..." and "if you can't be arsed to do it yourself..." territory
- gaytabasedysfun@treehouse.systemsOct 14, 2026, 10:05 AM
@flippac i mean, i think they knew how to do this https://github.com/m-mueller678/btree-cpp/blob/sigmod25/pext_bench/pext.cpp
but they could have gotten that information quicker from uops.info. they do benchmark on amd in the paper, but it's newer than zen 2 so not microcoded and obscenely slow.
- Philippa Cowderoyflippac@types.plOct 14, 2026, 10:07 AM
@dysfun yeah: some of what I'm saying is normative rather than strictly factual judgment
(also, people love to forget that the current generation of consoles-not-hybrids are on zen 2 still)
- gaytabasedysfun@treehouse.systemsOct 14, 2026, 10:08 AM
@flippac i'm on zen2 still.