The researchers tested ChatGPT's performance using the QuixBugs bug-fixing benchmark. The automated program repair (APR) systems appear to be at a disadvantage as they were developed prior to 2018.