Finally, I think I'll leave you with some of the feedback given by the control group:

- "This was a lot of fun but the recording aspect can be cumbersome on
some systems and cause a little bit of anxiety especially when you can’t
go back if you messed up the recording."

- "I think I could have done much better if I could have accessed the coding tasks I did at part 2 during the quiz for reference, but I still tried my best. I ran out of time as the bug-finding questions
were quite challenging for me."

- "I spent too much time on this quiz, but that was due to my time management.
Even if I hadn’t spent too much time on the first part, though, it still
would have been a tight finish for me in the 30 minute window I think."

To me, these read like stress. It's so disappointing that the study was designed in such a stressful way. Even moreso that the subject's stress doesn't seem to have been considered as a factor at all. That plus the tooling handicap of the control group make it impossible to draw the kind of conclusions that the authors and Anthropic seem to be doing.

/end