New GitHub Copilot Research Finds 'Downward Pressure On Code Quality'

theodp writes: Visual Studio Magazine reports on new research on the effect of AI-powered GitHub Copilot on software development which sought to investigate the quality and maintainability of AI-assisted code compared to what would have been written by a human. Countering the positively-glowing findings of some other studies, the Coding on Copilot whitepaper from GitClear cites some adverse results. "We find disconcerting trends for maintainability," explains the paper's abstract. "Code churn -- the percentage of lines that are reverted or updated less than two weeks after being authored -- is projected to double in 2024 compared to its 2021, pre-AI baseline. We further find that the percentage of 'added code' and 'copy/pasted code' is increasing in proportion to 'updated,' 'deleted,' and 'moved 'code. In this regard, AI-generated code resembles an itinerant contributor, prone to violate the DRY-ness [don't repeat yourself] of the repos visited." The paper concludes, "How will Copilot transform what it means to be a developer? There's no question that, as AI has surged in popularity, we have entered an era where code lines are being added faster than ever before. The better question for 2024: who's on the hook to clean up the mess afterward?" Further complicating matters, Computing Education in the Era of Generative AI (Feb. 2024 CACM) notes that "generating and inserting large blocks of code may be counterproductive for users at all levels. This requires users to read through code they did not write, sometimes at a more sophisticated level than they are familiar with." Interestingly, the AI-generated code maintenance worries are reminiscent of concerns cited in the past for 'Google programmers', Stack Overflow copy-and-pasters, and stitchers of not-quite-compatible libraries, as well as earlier iterations of code generators, including C++ and other 'Next-Next-Finish' code wizards of the 90's and COBOL and PL/I applications generators (PDF) of the 80's. Everything old is new again, including code maintenance challenges.

Read more of this story at Slashdot.