We conduct a randomized controlled trial to understand how early-2025 AI tools affect the productivity of experienced open-source developers working on their own repositories. Surprisingly, we find that when developers use AI tools, they take 19% longer than without—AI makes them slower.
Interesting study. Also similar to my own observations. I’ve tried AI coding to some degree. Some people recommend it. And it definitely can do some nice things. Like write boilerplate code fast and do some tech-demos and exploring. And it’s kind of nice to bounce off ideas to someone. I mean the process of speaking out things loud and putting it into words helps me think it through. AI can do that (listen) and it’ll occasionally give back some ideas.
The downside of coding with it in real life is, I end up refactoring a lot of stuff. And that’s just tedious and annoying work. I’ve had this happen to me several times now. And I think the net time balance is negative for me as well. I think I’m better off coming up with an idea how to implement something, and then just type it down. Rather than skipping the step and then moving around stuff, changing it, and making sure it handles the edge-cases correctly and fits into the broader picture. Plus I still end up doing the thinking step, just in a different way.
Interesting study. Also similar to my own observations. I’ve tried AI coding to some degree. Some people recommend it. And it definitely can do some nice things. Like write boilerplate code fast and do some tech-demos and exploring. And it’s kind of nice to bounce off ideas to someone. I mean the process of speaking out things loud and putting it into words helps me think it through. AI can do that (listen) and it’ll occasionally give back some ideas.
The downside of coding with it in real life is, I end up refactoring a lot of stuff. And that’s just tedious and annoying work. I’ve had this happen to me several times now. And I think the net time balance is negative for me as well. I think I’m better off coming up with an idea how to implement something, and then just type it down. Rather than skipping the step and then moving around stuff, changing it, and making sure it handles the edge-cases correctly and fits into the broader picture. Plus I still end up doing the thinking step, just in a different way.
Related to what you said, I found it actually helpful to just write and discuss ideas with some LLM without letting it code.