'OK, So ChatGPT Just Debugged My Code. For Real'

ZDNet's senior contributing editor also maintains software, and recently tested ChatGPT on two fixes for bugs reported by users, and a new piece of code to add a new feature, It's a "real-world" coding test, "about pulling another customer support ticket off the stack and working through what made the user's experience go south." First... please rewrite the following code to change it from allowing only integers to allowing dollars and cents (in other words, a decimal point and up to two digits after the decimal point).ChatGPT responded by explaining a two-step fix, posting the modified code, and then explaining the changes. "I dropped ChatGPT's code into my function, and it worked. Instead of about two-to-four hours of hair-pulling, it took about five minutes to come up with the prompt and get an answer from ChatGPT."Next up was reformatting an array. I like doing array code, but it's also tedious. So, I once again tried ChatGPT. This time the result was a total failure. By the time I was done, I probably fed it 10 different prompts. Some responses looked promising, but when I tried to run the code, it errored out. Some code crashed; some code generated error codes. And some code ran, but didn't do what I wanted. After about an hour, I gave up and went back to my normal technique of digging through GitHub and StackExchange to see if there were any examples of what I was trying to do, and then writing my own code. Then he posted the code for a function handling a Wordpress filter, along with the question: "I get the following error. Why?"Within seconds, ChatGPT responded... Just as it suggested, I updated the fourth parameter of the add_filter() function to 2, and it worked! ChatGPT took segments of code, analyzed those segments, and provided me with a diagnosis. To be clear, in order for it to make its recommendation, it needed to understand the internals of how WordPress handles hooks (that's what the add_filter function does), and how that functionality translates to the behavior of the calling and the execution of lines of code. I have to mark that achievement as incredible - undeniably 'living in the future' incredible... As a test, I also tried asking ChatGPT to diagnose my problem in a prompt where I didn't include the handler line, and it wasn't able to help. So, there are very definite limitations to what ChatGPT can do for debugging right now, in 2023... Could I have fixed the bug on my own? Of course. I've never had a bug I couldn't fix. But whether it would have taken two hours or two days (plus pizza, profanity, and lots of caffeine), while enduring many interruptions, that's something I don't know. I can tell you ChatGPT fixed it in minutes, saving me untold time and frustration. The article does include a warning. "AI is essentially a black box, you're not able to see what process the AI undertakes to come to its conclusions. As such, you're not really able to check its work... If it turns out there is a problem in the AI-generated code, the cost and time it takes to fix may prove to be far greater than if a human coder had done the full task by hand." But it also ends with this prediction. "I see a very interesting future, where it will be possible to feed ChatGPT all 153,000 lines of code and ask it to tell you what to fix... I can definitely see a future where programmers can simply ask ChatGPT (or a Microsoft-branded equivalent) to find and fix bugs in entire projects."

Read more of this story at Slashdot.