@ljwrites @MutedTrampet I did read that, and I agree with it. Pointing an LLM at unsolved problems and getting incomprehensible yes/no answers can be counterproductive, though there are cases where having an answer from an oracle can itself be useful since it can ground further work.
I agree I was missing the reason you bring up the credit issue, though I cannot accept the framing of it as stealing given my views on intellectual property (it should not exist). But let's set that aside.
If I understand what you're saying correctly, forgive me if I'm reading your claim as stronger than you mean it, you're stating that any AI output can only embody knowledge that has already previously been produced by some other human and is present in its prompt or training corpus.
If so, I don't think this holds. The declaration also seems difficult to reconcile with that position. In particular, it says: "AI offers the potential of enhancing and accelerating genuine mathematical study and understanding." It's hard to read that, together with its claim that AI is increasingly capable of solving problems directly, as claiming AI cannot produce novel results.
Consider the unit distance result.
The result itself was new, externally verified and surprising. The conjecture was widely believed to be true. Naturally everything an LLM generates is based in some way on its corpus, but the particular construction to solve this hadn't been produced by a human before. If deriving a novel result from previously existing mathematics doesn't count as novel, then I'm not sure what does.
Incidentally, this is also something psychics can't do, unless they're also mathematicians; in which case, the analogy holds because they're mathematicians, not because they're psychics.