@david_megginson @a_cubed @gabrielesvelto Transformer models can do really quite well when the underlying ground truth is smoothly differentiable. This is the case in quite a few scientific domains (because science looks for this).
But human thought turns out to not be a projection on a smooth differentiable hypersurface. It's got elements that are more complex. (It's also necessarily dynamic, but that's a different problem.)