on the adoption of mc/dc and control-flow adequacy for a tight integration of program testing and statistical fault localization

ISCAS OpenIR

	on the adoption of mc/dc and control-flow adequacy for a tight integration of program testing and statistical fault localization
	Jiang Bo; Zhai Ke; Chan W.K.; Tse T.H.; Zhang Zhenyu
	2012
发表期刊	Information and Software Technology
ISSN	0950-5849
页码	-
摘要	Context: Testing and debugging consume a significant portion of software development effort. Both processes are usually conducted independently despite their close relationship with each other. Test adequacy is vital for developers to assure that sufficient testing effort has been made, while finding all the faults in a program as soon as possible is equally important. A tight integration between testing and debugging activities is essential. Objective: The paper aims at finding whether three factors, namely, the adequacy criterion to gauge a test suite, the size of a prioritized test suite, and the percentage of such a test suite used in fault localization, have significant impacts on integrating test case prioritization techniques with statistical fault localization techniques. Method: We conduct a controlled experiment to investigate the effectiveness of applying adequate test suites to locate faults in a benchmark suite of seven Siemens programs and four real-life UNIX utility programs using three adequacy criteria, 16 test case prioritization techniques, and four statistical fault localization techniques. We measure the proportion of code needed to be examined in order to locate a fault as the effectiveness of statistical fault localization techniques. We also investigate the integration of test case prioritization and statistical fault localization with postmortem analysis. Result: The main result shows that on average, it is more effective for a statistical fault localization technique to utilize the execution results of a MC/DC-adequate test suite than those of a branch-adequate test suite, and is in turn more effective to utilize the execution results of a branch-adequate test suite than those of a statement-adequate test suite. On the other hand, we find that none of the fault localization techniques studied can be sufficiently effective in suggesting fault-relevant statements that can fit easily into one debug window of a typical IDE. Conclusion: We find that the adequacy criterion and the percentage of a prioritized test suite utilized are major factors affecting the effectiveness of statistical fault localization techniques. In our experiment, the adoption of a stronger adequacy criterion can lead to more effective integration of testing and debugging. © 2012 Elsevier B.V. All rights reserved.; Context: Testing and debugging consume a significant portion of software development effort. Both processes are usually conducted independently despite their close relationship with each other. Test adequacy is vital for developers to assure that sufficient testing effort has been made, while finding all the faults in a program as soon as possible is equally important. A tight integration between testing and debugging activities is essential. Objective: The paper aims at finding whether three factors, namely, the adequacy criterion to gauge a test suite, the size of a prioritized test suite, and the percentage of such a test suite used in fault localization, have significant impacts on integrating test case prioritization techniques with statistical fault localization techniques. Method: We conduct a controlled experiment to investigate the effectiveness of applying adequate test suites to locate faults in a benchmark suite of seven Siemens programs and four real-life UNIX utility programs using three adequacy criteria, 16 test case prioritization techniques, and four statistical fault localization techniques. We measure the proportion of code needed to be examined in order to locate a fault as the effectiveness of statistical fault localization techniques. We also investigate the integration of test case prioritization and statistical fault localization with postmortem analysis. Result: The main result shows that on average, it is more effective for a statistical fault localization technique to utilize the execution results of a MC/DC-adequate test suite than those of a branch-adequate test suite, and is in turn more effective to utilize the execution results of a branch-adequate test suite than those of a statement-adequate test suite. On the other hand, we find that none of the fault localization techniques studied can be sufficiently effective in suggesting fault-relevant statements that can fit easily into one debug window of a typical IDE. Conclusion: We find that the adequacy criterion and the percentage of a prioritized test suite utilized are major factors affecting the effectiveness of statistical fault localization techniques. In our experiment, the adoption of a stronger adequacy criterion can lead to more effective integration of testing and debugging. © 2012 Elsevier B.V. All rights reserved.
收录类别	EI
关键词	Experiments Integration Statistics
部门归属	(1) School of Computer Science and Engineering Beihang University Beijing China; (2) Department of Computer Science The University of Hong Kong Pokfulam Hong Kong; (3) Department of Computer Science City University of Hong Kong Tat Chee Avenue Hong Kong; (4) State Key Laboratory of Computer Science Institute of Software Chinese Academy of Sciences Beijing China
语种	英语
WOS记录号	WOS:000317327000008
引用统计
内容类型	期刊论文
URI标识	http://ir.iscas.ac.cn/handle/311060/15169
专题	中国科学院软件研究所
推荐引用方式 GB/T 7714	Jiang Bo,Zhai Ke,Chan W.K.,et al. on the adoption of mc/dc and control-flow adequacy for a tight integration of program testing and statistical fault localization[J]. Information and Software Technology,2012:-.
APA	Jiang Bo,Zhai Ke,Chan W.K.,Tse T.H.,&Zhang Zhenyu.(2012).on the adoption of mc/dc and control-flow adequacy for a tight integration of program testing and statistical fault localization.Information and Software Technology,-.
MLA	Jiang Bo,et al."on the adoption of mc/dc and control-flow adequacy for a tight integration of program testing and statistical fault localization".Information and Software Technology (2012):-.