Login
You're viewing the front-end.social public feed.
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 5:27 AM

    Found it! n=52. wtf. I reiterate: 20 billion dollars, just for this current funding round, and they only managed to do this study with 52 people.

    But anyway, let's return to the methods themselves. They start with the design of the evaluation component, so I will too. It's organized around 4 evaluative practices they say are common in CS education. That seems fine, but their explanation for why these things are relevant is weird.

    1. Debugging. According to them "this skill is curcial for detecting when AI-generated code is incorrect and understanding why it fails.

    Maybe their definition is more expansive than it seems here? But it's been my experience, professionally, that this is just not the case. The only even sort-of reliable mechanism for detecting and understanding the shit behavior of slop code is extensive validation suites.

    💬 1🔄 5⭐ 35

Replies

  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 5:32 AM

    2. Code Reading. "This skill enables humans to understand and verify AI-written code before deployment."

    Again, not in my professional experience. It's just too voluminous and bland. And no one has time for that shit, even if they can make themselves do it. Plus, I haven't found anyone who can properly review slop code, because we can't operate without the assumptions of comprehension, intention, and good faith that simply do not hold in that case.

    💬 3🔄 10⭐ 68
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 5:37 AM

    3. Code writing. Honestly, I don't get the impression they even understand what this means. They say "Low-level code writing, like remembering the syntax of functions, will be less important with further integration of AI coding tools
    than high-level system design."

    Neither of those things is a meaningful facet of actually writing code. Writing code exists entirely in-between those two things. Code completion tools basically eliminate having to think about syntax (but we will return to this). And system design happens in the realm of abstract behaviors and responsibilities.

    💬 2🔄 4⭐ 36
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 5:42 AM

    4. Conceptual. As they put it, "Conceptual understanding is critical to assess whether AI-generated code uses appropriate design patterns that adheres to how the library should be used.

    IIIIIII guess. That's not wrong, exactly? But it's such a reverse centaur world view. I don't want to be the conceptual bounds checker for the code extruder. And I don't understand why they don't understand that.

    💬 1🔄 4⭐ 34
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 5:46 AM

    So anyway, all of this is, apparently, in service to the "original motivation of developing and retaining the skills required for supervising automation."

    Which would be cool, I'd like to read that study, because it isn't this one. This study is about whether the tools used to rapidly spit out meaningless code will impact one's ability to answer questions about the code that was spat. And even then, I'm not sure the design of the study can answer that question.

    💬 2🔄 0⭐ 27
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 6:11 AM

    I guess this brings me to the study design. I'm struggling a little to figure out how to talk about this. The short version is that I don't think they're testing any of the effects they think they're testing.

    So, they start with a warmup coding round, which seems to be mostly to let people become familiar with the tool. That's important, because the tool is commercial software for conducting coding interviews in a browser. They don't say which one, that I've seen.

    Then they have two separate toy projects that the subjects should complete. 1 is a non-blocking ticker, using a specific async library. 2 is some async I/O record retrieval with basic error handling, using the same async library.

    And then they take a quiz about that async library.

    But there's some very important details. The coding portion and quiz are both timed. The subjects were instructed to complete them as fast as possible. And the testing platform did not seem to have code completion or, presumably, any other modern development affordance.

    💬 1🔄 0⭐ 22
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 6:21 AM

    Given all of that, I don't actually think they measured the impact of the code extruding chatbots at all. On anything. What they measured was stress. This is a stress test.

    And, to return to their notion of what "code writing" consists of: the control subjects didn't have code completion, and the test subjects did. I know this, because they said so. It came up in their pilot studies. The control group kept running out of time because they struggled with syntax for try/catch, and for string formatting. They only stopped running out of time after the researchers added specific reminders for those 2 things to the project's instructions.

    💬 1🔄 3⭐ 29
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 6:26 AM

    So. The test conditions were weirdly high stress, for no particular reason the study makes clear. Or even acknowledges. The stress was *higher* on the control group. And the control group had to use inferior tooling.

    I don't see how this data can be used to support any quantitative conclusion at all.

    Qualitatively, I suspect there is some value in the clusters of AI usage patterns they observed. But that's not what anyone is talking about when they talk about this study.

    💬 1🔄 7⭐ 30
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 6:32 AM

    And then there's one more detail. I'm not sure how I should be thinking about this, but it feels very relevant. All of the study subjects were recruited through a crowd working platform. That adds a whole extra concern about the subject's standing on the platform. It means that in some sense undertaking this study was their job, and the instruction given in the project brief was not just instruction to a participant in a study, but requirements given to a worker.

    I know this kind of thing is not unusual in studies like this. But it feels like a complicating factor that I can't see the edges of.

    💬 2🔄 1⭐ 21
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 6:58 AM

    But now it's 1am. I may pick this up tomorrow, I'm not sure. If I do, the next chapter is their analysis. Seems like there would be things in there that merit comment

    💬 2🔄 0⭐ 4
  • 💬 3🔄 0⭐ 13
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 4:25 PM

    I might get back to this after breakfast.

    There's 2 particular things i keep thinking about.

    1. The quiz scores dropped significantly for people who started out using the chatbots in interrogatory ways and then evolved to having it do major code generation, as compared to people who persisted in their interrogative use. Or as compared to people who used it for major generation, but followed up with interrogation.
    2. Even with syntax reminders, the control group had difficulty completing the coding problems due to struggling with syntax. I want to know if this had an observable effect on the quiz scores.

    💬 1🔄 1⭐ 9
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 6:06 PM

    They didn't really explore either phenomenon. It also doesn't seem like they included enough data that i could do much of an analysis myself. And I'm not sure I would want to, anyway

    💬 2🔄 0⭐ 6
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 7:41 PM

    Let's finish this off.

    They had like ~~2~~ 4 paragraphs at the end of the results for a quantitative analysis. And, as it turns out, I also don't have much more to say about this thing, quantitatively. I continue to think that the study design doesn't actually allow for meaningful comparisons between the control and test groups. That sucks. Maybe they should talk to some methods people, I dunno. I can think of one I'd recommend.

    💬 1🔄 1⭐ 13
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 7:55 PM

    Qualitatively, I think there are things worth discussing. And it seems the authors agree, because there's 7 pages of it.

    They identified 4 "axes" of interaction with the AI that they think are explanatory. As they put it:

    - AI Interaction Time: The lack of significant speed-up in the AI condition can be explained by how some participants used AI. Several participants spent substantial time interacting with the AI assistant, spending up to 11 minutes composing AI queries in total
    - Query Types: The study participants varied between conceptual questions only, code generation only,
    and a mixture of conceptual, debugging, and code generation queries. Participants who focused on
    asking the AI assistant debugging questions or confirming their answer spent more time on the task
    - Encountering Errors: Participants in the control group (no AI) encountered more errors; these errors
    included both syntax errors and Trio errors (Figure 14). Encountering more errors and independently
    resolving errors likely improved the formation of Trio skills (note, Trio is the library that was the focus of the test problems)
    - Active Time: Using AI decreased the amount of active coding time. Time spent coding shifted to time spent interacting with AI and understanding AI generations (Figure 16).

    💬 1🔄 0⭐ 6
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 7:59 PM

    I continue to find this frustrating, because they absolutely will not let go of notions like time to completion or rate of output as being meaningful, or even vital. But this is supposed to be a study about learning outcomes. I'm not an expert on the latest in learning/teaching research, but I don't think those are useful measures. Please, correct me if I'm wrong, though.

    They also have remarkably little to say about the control group in this section, and I think _that_ says a lot.

    💬 2🔄 2⭐ 20
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 8:15 PM

    Still, they then identified 6 clusters of interaction patterns with the AI that correlated with performance and related in varying degrees to those axes.

    Personally, I think I would call it 5 patterns, plus a group that moved from 1 pattern to another. But maybe that's not an important distinction.

    They broadly fell into low-scoring and high-scoring groups. The low scoring group was:
    - AI delegation. Just having the chatbot generate all the code.
    - "Progressive AI Reliance". Or, starting with conceptual inquiry, and then later just having the chatbot generate everything.
    - Iterative AI Debugging. Just having the chatbot generate all the code, and then just showing the chatbot whatever errors resulted and instructing it to fix the problem.

    What I find *really* interesting here is that the group who started with what they call conceptual inquiry and then moved to delegation scored _30 percent_ lower on the quiz than the group who only engaged in conceptual inquiry.

    That is an ENORMOUS effect for a tiny intervention. They actually performed comparably the group that engaged in pure delegation the whole time. I don't see any discussion of this from the authors, and that also sucks.

    💬 1🔄 0⭐ 12
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 8:16 PM

    I might have expected the initial approach that was more oriented around understanding would have some protective effect against the switch to a production orientation. I also wonder if it reflects a disengagement with the task? Apparently the authors don't share my curiosity.

    💬 1🔄 0⭐ 7
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 8:29 PM

    Then there are the high-scoring patterns:

    - "Generation then comprehension". Generate code, and then ask followup questions about it.
    - "Hybrid code-explanation". Generate code and simultaneously ask for explanation.
    - "Conceptual inquiry". Don't generate code, just ask questions for understanding.

    The authors propose that "spending time" and "encountering errors" do a lot to explain the difference in quiz scores. The relative results from these groups make me doubt that. I suspect that the actual differentiator is having assumptions revealed and invalidated. The generate-then-follow-up pattern is the only one of the three that actually offers a chance to incorporate some result from a change into the explanation of the change. This group scored 16% - 19% higher than the other two, for nearly the same amounts of time spent on the task.

    💬 1🔄 0⭐ 3
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 8:36 PM

    Finally, I think I'll leave you with some of the feedback given by the control group:

    - "This was a lot of fun but the recording aspect can be cumbersome on
    some systems and cause a little bit of anxiety especially when you can’t
    go back if you messed up the recording."

    - "I think I could have done much better if I could have accessed the coding tasks I did at part 2 during the quiz for reference, but I still tried my best. I ran out of time as the bug-finding questions
    were quite challenging for me."

    - "I spent too much time on this quiz, but that was due to my time management.
    Even if I hadn’t spent too much time on the first part, though, it still
    would have been a tight finish for me in the 30 minute window I think."

    To me, these read like stress. It's so disappointing that the study was designed in such a stressful way. Even moreso that the subject's stress doesn't seem to have been considered as a factor at all. That plus the tooling handicap of the control group make it impossible to draw the kind of conclusions that the authors and Anthropic seem to be doing.

    /end

    💬 1🔄 0⭐ 22
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 2, 2026, 1:08 AM

    Actually, one last thing.

    I don't think this study was well designed, but I don't want to go much farther than that.

    Some people have made something out of the paper not being peer reviewed. That's not a secret, though. This is arxiv, it's a prepublication host.

    Also, I glanced at the lead author's other work, and I get the impression that she's just not accustomed to working with human subjects. I think that's hubris, but not malice. It's just the standard attitude in tech that being good at computer touching qualifies one to do virtually anything else they want.

    💬 0🔄 6⭐ 26
  • jonny (nonvenomous)jonny@neuromatch.social
    Feb 1, 2026, 5:35 AM

    @jenniferplusplus the latter part is especially true and i don't have any sort of strategy for handling it. i have to read every single line of LLM code because the space of possible mistakes it can make is so large. with humans, even if someone really doesn't know what they are doing, there are only so many kinds of things that could conceivably screw up.

    💬 0🔄 1⭐ 8
  • Hrefna (DHC)hrefna@hachyderm.io
    Feb 1, 2026, 6:01 AM

    @jenniferplusplus That paper is _extremely damning_ of the use of AI for all that it bends over backwards and ties itself into knots to try to find some way of making it seem less catastrophically bad.

    💬 1🔄 1⭐ 8
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 6:02 AM

    @hrefna it certainly doesn't make them look good. But I'm honestly not sure we can draw *any* conclusion from this study. Which I'm getting into now

    💬 0🔄 1⭐ 5
  • 💬 0🔄 0⭐ 3
  • 💬 0🔄 0⭐ 1
  • Thomas DepierreDi4na@hachyderm.io
    Feb 1, 2026, 10:52 AM

    @jenniferplusplus i have a different take on what they actually tested.

    If you look in the problems they had, people had problems finishing in time without AI due to Syntax errors.

    What they tells me is that they tested ... Using basic syntax highlighting and basic linters/snippets.

    The AI help "simulate" badly these by offering a way to generate plausible code.

    Basically they tested
    1. that LLM can produce plausible language
    Which is their purpose so eh
    2. Tooling matters and help productivity. We knew that!

    💬 1🔄 0⭐ 2
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 4:05 PM

    @Di4na yes, that was a factor. It's not clear if it had any impact on what they were measuring, because they did such a shit job of measuring it. I'd be interested to see if the quiz scores are predicted by whether a control subject completed both coding problems, for instance.

    If they share data at that level, I haven't seen it yet.

    What is clear to me is that we can't actually make comparisons between the two groups in this study. I'm undecided if we can make comparisons within a group

    💬 0🔄 0⭐ 0
  • 💬 1🔄 0⭐ 4
  • 💬 0🔄 0⭐ 0
  • 💬 0🔄 0⭐ 2
  • 💬 0🔄 0⭐ 1
  • Joshua Barrettojsbarretto@social.coop
    Feb 1, 2026, 5:53 AM

    @jenniferplusplus Kind of a funny statement given that the whole point of abstraction, encapsulation, high level languages, etc. is to provide a formal basis for much of a program to be designed in terms of high level concepts

    💬 1🔄 0⭐ 1
  • Jenniferplusplusjenniferplusplus@hachyderm.io
    Feb 1, 2026, 6:53 AM

    @jsbarretto That's not what people mean when they say system design.

    They mean which way do dependencies flow. What is the scope of responsibility for this thing. How will it communicate with other things. How does the collection of things remain in a consistent state.

    For example.

    💬 1🔄 0⭐ 1
  • Joshua Barrettojsbarretto@social.coop
    Feb 1, 2026, 10:43 AM

    @jenniferplusplus Yeah, I get that. I've been around the block. I'm saying it's bizarre that the paper seems to be implying that AI might be a route towards automating the production of lower level code when this has been the goal of pretty much every form of developer tooling since forever.

    💬 0🔄 0⭐ 1
  • pixxpixx@merveilles.town
    Feb 2, 2026, 1:04 AM

    @jenniferplusplus
    Why is that any different from reviewing slop code written by incompetents? Most foss maintainers have to deal with some of that too...

    [Agreeing with most of what you're saying, btw]

    💬 0🔄 0⭐ 0