TY - GEN
T1 - Automatically generated patches as debugging AIDS
T2 - 22nd ACM SIGSOFT International Symposium on the Foundations of Software Engineering, FSE 2014
AU - Tao, Yida
AU - Kim, Jindae
AU - Kim, Sunghun
AU - Xu, Chang
N1 - Publisher Copyright:
Copyright 2014 ACM.
PY - 2014/11/16
Y1 - 2014/11/16
N2 - Recent research has made significant progress in automatic patch generation, an approach to repair programs with less or no manual intervention. However, direct deployment of auto-generated patches remains difficult, for reasons such as patch quality variations and developers' intrinsic resistance. In this study, we take one step back and investigate a more feasible application scenario of automatic patch generation, that is, using generated patches as debugging aids. We recruited 95 participants for a controlled experiment, in which they performed debugging tasks with the aid of either buggy locations (i.e., the control group), or generated patches of varied qualities. We observe that: a) high-quality patches significantly improve debugging correctness; b) such improvements are more obvious for difficult bugs; c) when using low-quality patches, participants' debugging correctness drops to an even lower point than that of the control group; d) debugging time is significantly affected not by debugging aids, but by participant type and the specific bug to fix. These results highlight that the benefits of using generated patches as debugging aids are contingent upon the quality of the patches. Our qualitative analysis of participants' feedback further sheds light on how generated patches can be improved and better utilized as debugging aids.
AB - Recent research has made significant progress in automatic patch generation, an approach to repair programs with less or no manual intervention. However, direct deployment of auto-generated patches remains difficult, for reasons such as patch quality variations and developers' intrinsic resistance. In this study, we take one step back and investigate a more feasible application scenario of automatic patch generation, that is, using generated patches as debugging aids. We recruited 95 participants for a controlled experiment, in which they performed debugging tasks with the aid of either buggy locations (i.e., the control group), or generated patches of varied qualities. We observe that: a) high-quality patches significantly improve debugging correctness; b) such improvements are more obvious for difficult bugs; c) when using low-quality patches, participants' debugging correctness drops to an even lower point than that of the control group; d) debugging time is significantly affected not by debugging aids, but by participant type and the specific bug to fix. These results highlight that the benefits of using generated patches as debugging aids are contingent upon the quality of the patches. Our qualitative analysis of participants' feedback further sheds light on how generated patches can be improved and better utilized as debugging aids.
KW - Automatic patch generation
KW - Debugging
KW - Human study
UR - http://www.scopus.com/inward/record.url?scp=84965149079&partnerID=8YFLogxK
U2 - 10.1145/2635868.2635873
DO - 10.1145/2635868.2635873
M3 - Conference contribution
AN - SCOPUS:84965149079
T3 - Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering
SP - 64
EP - 74
BT - 22nd ACM SIGSOFT International Symposium on the Foundations of Software Engineering, FSE 2014 - Proceedings
PB - Association for Computing Machinery
Y2 - 16 November 2014 through 21 November 2014
ER -
