Login
You're viewing the discuss.systems public feed.
  • Nat Bennettnat@ruby.social
    Oct 12, 2026, 10:16 PM

    The linked PDF Is to a Harvard study that I think can be fairly summarized as follows

    "Among firms that use JIRA, firms that adopted AI earlier produced more code, more PRs, and more commits, but did not increase the number of completed JIRA tickets. We considered whether this is because the JIRA tickets got larger but we don't think they did. Instead, we believe that it's mostly because any time that got saved writing code is now spent reviewing and verifying code."

    fion.ac/jellyfish.pdf

    💬 1🔄 0⭐ 0

Replies

  • Nat Bennettnat@ruby.social
    Oct 12, 2026, 10:20 PM

    There are some interesting exceptions to this general "coding agents probably just move the work around" thesis. In particular, I find certain stories about using co-ordinated agents to do large pieces of work entirely plausible, particularly for domains where verification is highly specifiable, like "migrate data from one relational database format to another"

    cockroachlabs.com/blog/experim

    💬 1🔄 0⭐ 0
  • Nat Bennettnat@ruby.social
    Oct 12, 2026, 10:23 PM

    On the other end of the spectrum, I also find it plausible that agents built on LLMs are good at building "my personal version of X" especially when you need a pretty small subset of the features of X

    They're very good at copying. They're bad at fit-and-finish. There are a lot of cases where fit-and-finish just doesn't matter very much, so being bad at it is fine.

    💬 1🔄 0⭐ 0
  • Nat Bennettnat@ruby.social
    Oct 12, 2026, 10:28 PM

    But in general I don't think that AI-assisted programming helps very much with either

    1) the absolute hardest problems in your software
    2) the long tail of small detail issues that were already mostly "decide what it should do"

    And in fact I think it's very possible that they're making both of those parts of software development worse/more expensive

    You're getting the bulk of already-pretty-easy coding work faster at the cost of making the time consuming parts harder

    💬 2🔄 0⭐ 0
  • Nat Bennettnat@ruby.social
    Oct 12, 2026, 10:30 PM

    Still, this is at the level of Hypothesis

    Mostly what I'm sure about is that productivity in software development is already pretty poorly measured, so it's hard to make statements about one way or another

    We've known for decades that lines of code is a bad measure but you'll still see people who should know better talking about how great their agents are in terms of LoC

    💬 1🔄 0⭐ 0
  • Nat Bennettnat@ruby.social
    Oct 12, 2026, 10:30 PM

    I think in aggregate it's likely that what we're doing right now, as an industry, is stressing everyone TF out and spending a bunch of money without actually... getting much of anything out of it

    💬 1🔄 0⭐ 0
  • Nat Bennettnat@ruby.social
    Oct 12, 2026, 10:31 PM

    We're at the part of the Hype Cycle where MongoDB is the Future because it's Web Scale even though obviously a lot of datasets are... relational... and so should be modeled with... a relational database

    💬 1🔄 0⭐ 0
  • Matthias Vkronn@ruby.social
    Oct 17, 2026, 8:59 PM

    @nat
    OMG i haven't heard webscale in a while (didn't scale well, i assume). The comparison fits well and I tend to agree with the hypothesis. I will counter the AI-webscaleys with that like it is a fact. If they ask for proof, I ask for proof of their case.

    Also, I assume every dev has caught a slop-grenade by now. I know I have. So anecdotal evidence (yeah...) is available.

    💬 0🔄 0⭐ 0
  • alphaalpha@hey.kejadlen.dev
    Oct 13, 2026, 3:17 AM

    @nat My personal thinking (right now, at least) is that AI is really only a difference in cost/scale - it was always possible to outsource code production, but previously you'd do it by spending half a million dollars on contractors of dubious quality, but now that same code is available at a fraction of the time and money.

    So it's extremely effective at actually a pretty wide range of tasks - I think not just "already-pretty-easy" coding work, but also stuff that just previously wasn't worth the cost or time to do previously.

    💬 0🔄 0⭐ 0
  • Nat Bennettnat@ruby.social
    Oct 13, 2026, 5:24 PM

    @alpha Can you give me an example of a task that isn't "already pretty easy," wasn't worth the time/cost before, and is worth the time/cost with AI?

    I want to be sure we're talking about the same class of things.

    💬 1🔄 0⭐ 0
  • alphaalpha@hey.kejadlen.dev
    Oct 13, 2026, 6:57 PM

    @nat A couple recent ones are diagnosing issues with my self-hosted setup and building a CardDAV server. I mean, a lot depends on what "pretty easy" means for an individual, but regardless of difficulty, time/cost to value is hugely differentiating, even for things that are pretty easy.

    💬 0🔄 0⭐ 0
  • alphaalpha@hey.kejadlen.dev
    Oct 13, 2026, 6:59 PM

    @nat I just don't think it's a new problem in the general sense - we've always had to balance scaling with understanding code, even before AI.

    💬 0🔄 0⭐ 0
  • Nat Bennettnat@ruby.social
    Oct 13, 2026, 7:20 PM

    @alpha Yeah I definitely don’t think the issue is “no one wrote it.” And those strike me as good examples of the kind of work AI is most useful for — you wouldn’t have attempted to do it without AI.

    This is the kind of work I had in mind as “relatively easy.”

    💬 1🔄 0⭐ 0
  • Nat Bennettnat@ruby.social
    Oct 13, 2026, 7:21 PM

    @alpha I do think the outsourcing metaphor is useful but breaks down in the same way that “commissioning an artist” breaks down for AI art generation.

    There’s no way that a human being would produce some of those restaurant menu images, for instance, and I think there are code equivalents to that they’re just harder to see because we’re less sensitive to weirdness in code than we are to weirdness in food.

    💬 0🔄 0⭐ 0