We conduct a randomized controlled trial to understand how early-2025 AI tools affect the productivity of experienced open-source developers working on their own repositories. Surprisingly, we find that when developers use AI tools, they take 19% longer than without—AI makes them slower.
Anecdotally, this sounds very close to my own experience. The fact that AI can generate code quickly creates a false sense of speed. While experienced engineers are better at vetting and correcting generated code, that still causes significant overhead. There are also delayed impacts when bugs, vulnerabilities, and oversights slip through. The best case scenario is that you break even, but when you pay attention to the big picture, you realize that you are actually taking longer to reach milestones than you used to. I also hypothesize that this overhead gets worse over time as you are far less familiar with the codebase you leave behind than if you had written it yourself.
Remember that AI is incapable of reasoning, so it can’t actually apply logic to the code it generates, which is a problem because all code is literally a representation of logic and reasoning.
Anecdotally, this sounds very close to my own experience. The fact that AI can generate code quickly creates a false sense of speed. While experienced engineers are better at vetting and correcting generated code, that still causes significant overhead. There are also delayed impacts when bugs, vulnerabilities, and oversights slip through. The best case scenario is that you break even, but when you pay attention to the big picture, you realize that you are actually taking longer to reach milestones than you used to. I also hypothesize that this overhead gets worse over time as you are far less familiar with the codebase you leave behind than if you had written it yourself.
Remember that AI is incapable of reasoning, so it can’t actually apply logic to the code it generates, which is a problem because all code is literally a representation of logic and reasoning.