Contract-based program repair without the contracts : an extended study

Abstract

Most techniques for automated program repair (APR) use tests to drive the repair process; this makes them prone to generating spurious repairs that overfit the available tests unless additional information about expected program behavior is available. Our previous work on Jaid, an APR technique for Java programs, showed that constructing detailed state abstractions - similar to those employed by techniques for programs with contracts - from plain Java code without any special annotations provides valuable additional information, and hence helps mitigate the overfitting problem. This paper extends the work on Jaid with a comprehensive experimental evaluation involving 693 bugs in three different benchmark suites. The evaluation shows, among other things, that: 1) Jaid is effective: it produced correct fixes for over 15 percent of all bugs, with a precision of nearly 60 percent; 2) Jaid is reasonably efficient: on average, it took less than 30 minutes to output a correct fix; 3) Jaid is competitive with the state of the art, as it fixed more bugs than any other technique, and 11 bugs that no other tool can fix; 4) Jaid is robust: its heuristics are complementary and their effectiveness does not depend on the fine-tuning of parameters. The experimental results also indicate the main trade-offs involved in designing an APR technique based on tests, as well as possible directions for further progress in this line of work.