Login
You're viewing the discuss.systems public feed.
  • Dare Obasanjocarnage4life@mas.to
    Oct 9, 2026, 12:26 PM

    Harvard researchers studied 718 companies and found AI coding agents increased lines of code by 30%, but no meaningful increase in completed tasks or projects.

    Meanwhile, review time rose 49%.

    Generating code faster hasn’t translated into completing more work. The bottleneck has moved to code reviews.

    The solution the industry is landing on is throwing more AI at the problem. AI writes the code, AI reviews the code and no one knows how it works.

    fion.ac/jellyfish.pdf

    Image attached toot
    💬 16🔄 91⭐ 50

Replies

  • 💬 0🔄 0⭐ 0
  • Milescmiles74@indieweb.social
    Oct 9, 2026, 1:13 PM

    @carnage4life The "no meaningful increase in completed tasks or projects" sounds like a pretty important finding as well. IMHO, when AI writes the code *and* AI reviews and "corrects" the code, the defect rate increases.

    💬 1🔄 0⭐ 3
  • Rob Napiercocoaphony@mastodon.social
    Oct 10, 2026, 7:53 PM

    @cmiles74 @carnage4life This hasn't been quite what I've seen. Defect rate doesn't increase in a simple "more bugs" way. When I've allowed AI to do all the work, what happens is the complexity grows. Code duplication goes through the roof. And so when you find a bug, and you have the AI fix it, it fixes that exact bug, but there are 3 more versions of that bug that it doesn't fix. So you ask it to fix those, and the parts of the system diverge more, and the UX gets more and more inconsistent.

    💬 1🔄 0⭐ 0
  • Rob Napiercocoaphony@mastodon.social
    Oct 10, 2026, 7:56 PM

    @cmiles74 @carnage4life As you go deeper and deeper down this hole, at each step it kind of works "ok" but the system gets harder and harder on the AI to deal with, and has long since become impossible for humans to deal with. And by the time you're at the bottom, it's really hard to dig your way back out. It's an accelerated version of tech debt.

    The trouble is we say "AI code is lower quality" and for the first couple of iterations, that's not obvious, so the criticism is easy to ignore

    💬 0🔄 0⭐ 0
  • Cold Kettle Jackcount_01@mastodon.social
    Oct 9, 2026, 1:29 PM

    @carnage4life So weird that this would confirm what absolutely everyone with even an inkling of software development predicted, unless they were selling some vaporware.

    💬 0🔄 0⭐ 5
  • 💬 0🔄 0⭐ 0
  • James Morrisxjamesmorris@mastodon.social
    Oct 9, 2026, 7:10 PM

    @carnage4life many as-yet unsolved problems here and not much hard data on overall cost/benefit. Good to see some solid research. IMHO, we aren’t going back & need to understand how to go forward.

    💬 0🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • OCR BotOCRbot@fedi.lynnesbian.space
    Oct 9, 2026, 9:26 PM

    @LeelaTorres
    Artificial Intelligence in the Firm:

    Bottlenecks in Software Production

    Fiona Chen James Stratton
    Harvard University Harvard University
    Job Market Paper

    Current version: August 4, 2026
    First version: January 7, 2026

    Click here for current version

    Abstract

    We study the impacts of AI coding assistants and agents on software engineering work, using
    a novel proprietary dataset from an engineering analytics platform, covering 300 million work
    events — including GitHub coding activity, Jira issues, and Google Calendar events — across
    718 firms. We use a staggered difference-in-differences design, exploiting variation in firm-
    level adoption timing of AI coding assistants and agents. Both technologies increase coding
    productivity. However, productivity gains do not fully pass through to changes in software output
    or employment. For AI agents, this incomplete pass-through reflects a bottleneck from code
    review: review times increase, a larger share of code updates require revisions, and reviews
    involve more comments. We develop a model of software production to structure these results, in
    which AI affects both productivity and quality of intermediate outputs, and in turn can generate
    a review bottleneck and limit pass-through.

    💬 0🔄 0⭐ 0
  • Konstantin Weddigeweddige@gruene.social
    Oct 9, 2026, 9:47 PM

    @carnage4life and reviewing code is the harder task. So we took something that humans are decent at (writing code) and replaced it with something they struggle with (reviewing code). Hooray!

    (This doesn't matter however, if you lower your standards sufficiently; lower expectations explain IMHO most productivity gains through LLMs)

    💬 2🔄 1⭐ 0
  • 💬 0🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • Mx. Chara Aznable of Pnictogenmxchara@cyberpunk.lol
    Oct 9, 2026, 11:38 PM

    @carnage4life
    "...found AI coding agents increased lines of code by 30%..." Ah so a 30% productivity increase! =D

    "review time rose 49%" doesn't sound like it'd be so easy to spin but I think if anyone complained they'd get an answer back: "the higher review times just proves that we should switch to 'agentic' code review"

    💬 1🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • Lazy B0ylazyb0y@mastodon.social
    Oct 10, 2026, 7:00 AM

    @carnage4life

    harvard schmarvard - i just vibecoded my own research generator and it said all companies who do everything with more ai are 10000x more productive

    💬 0🔄 0⭐ 0
  • 💬 2🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • 🇺🇦 haxadecimal 🚫👑brouhaha@mastodon.social
    Oct 11, 2026, 9:11 AM

    @carnage4life
    What could possibly go wrong?
    And if anything does, they'll just try to use even more so-called "AI" to fix it.
    I can't wait for the bubble to burst. It's hard to believe that it can last much longer. There is only so much gullible capital to invest in it. Even the greater fools wise up eventually, or go broke.
    The business model is literally "losing money on every token sold, and making it up in volume," and it's amazing how easily they hoodwinked their investors.

    💬 0🔄 0⭐ 0
  • Malevolent Poison Gnomeaanee@mastodon.online
    Oct 11, 2026, 9:32 AM

    @carnage4life It's interesting (frustrating) how the authors are convinced from abstract to conclusion (Didn't see a results section) that AI is a good thing. Or at least that is the vibe I get.

    But lines of code = productivity, so it must be good! Cause programming is a factory task not a creative writing/deep knowledge task! /s

    💬 0🔄 0⭐ 0