Oct 16, 2026, 7:45 PM@Li @cwebber Not all the training data is putting strain. There are training datasets that were crawled respecting robots.txt.
I am most interested in the scenario when there is a training dataset (crawled once without hitting servers repeatedly), and used for open weight model.
Open data+weight still does not address attribution. That's why my question.
Oct 16, 2026, 7:50 PM@yantar92 @cwebber robots.txt is kind of for 20 different purposes, like e.g to know what parts of your site are pointless to try access (like things behind a login or paywall or something) so dont bother, its also things you want to be delisted from the site and also apparently also things you dont want used for training (which is still an opt out and thus manufactured consent but ok)
also thats still ignoring every other issue AI has thats not related to training data, it has a bad impact on people and the world as a whole too.
Oct 16, 2026, 8:01 PMOct 16, 2026, 8:03 PMOct 16, 2026, 8:05 PMOct 16, 2026, 8:06 PMOct 16, 2026, 8:17 PMOct 16, 2026, 8:24 PMOct 16, 2026, 8:29 PMOct 16, 2026, 8:38 PM@Li @cwebber I did not imply that solving attribution automatically resolves every other issue. I am asking about specific thing - AI not propertly attributing its sources used for the output. That's a concrete problem. I asked if attributing to *all* sources is good.engough to solve this particular problem.
Oct 16, 2026, 8:40 PMOct 16, 2026, 8:43 PM@Li @cwebber I conclude from your answer that referring to all sources, even not really used in the specific output is practically solving attribution (at least, from your perspective). I'd like to point out that may or may not be enough for other people who are looking for their work to be attributed.
So, no, I did not say "consider one issue, then solve that issue". What I proposed may or may not be a solution.
Oct 16, 2026, 8:43 PMOct 16, 2026, 8:48 PM@Li @cwebber Let's me give you exaggregated example - what if every slop code starts to attribute *every possible website on the internet*. Or even non-slop code using a code licensed with attribution does not attribute just to that code, but also to 1000 other codes (that have very little to do with this non-slop code). That feels not exactly right, although might technically fulfill the license. But maybe that's just me.
Oct 16, 2026, 8:53 PM@yantar92 @cwebber oh thats what you mean, yeah thats not obvious at all from what you said,
but also movies and books and such have for awhile includes a "special thanks" section of ppl who were not actually really involved at all,
and tbh over-attribution is usually not a problem- agree it feels kinda shit here
however i will note that in movies and books and such attribution is usually also in context, of like "these ppl filmed it, these ppl did sfx, this ppl wrote the script.. " etc. which AI doesn't really do that, nor can it, in this case its not even that they "did" it, really its like "??? (edited? generated? idfk how you'd even describe what AI does) from this work here from this person here"
Oct 16, 2026, 9:03 PM@Li @cwebber That's what I thought rightaway after I came up with this idea with overattribution. Obvious for me, and I wanted to see if it is the same for others. (It is not).
For the attribution context, one can always invent new word for it (like "statistically inspired" of whatever).
The bigger issue is for authors who may not even care about open source, but just want attribution to see usefulness of their work. Over-attribution is an insult for them.