how well does test case prioritization integrate with statistical fault localization?

ISCAS OpenIR

	how well does test case prioritization integrate with statistical fault localization?
	Jiang Bo; Zhang Zhenyu; Chan W.K.; Tse T.H.; Chen Tsong Yueh
	2012
发表期刊	Information and Software Technology
ISSN	9505849
卷号	54 期号:7 页码:739-758
摘要	Context: Effective test case prioritization shortens the time to detect failures, and yet the use of fewer test cases may compromise the effectiveness of subsequent fault localization. Objective: The paper aims at finding whether several previously identified effectiveness factors of test case prioritization techniques, namely strategy, coverage granularity, and time cost, have observable consequences on the effectiveness of statistical fault localization techniques. Method: This paper uses a controlled experiment to examine these factors. The experiment includes 16 test case prioritization techniques and four statistical fault localization techniques using the Siemens suite of programs as well as grep, gzip, sed, and flex as subjects. The experiment studies the effects of the percentage of code examined to locate faults from these benchmark subjects after a given number of failures have been observed. Results: We find that if testers have a budgetary concern on the number of test cases for regression testing, the use of test case prioritization can save up to 40% of test case executions for commit builds without significantly affecting the effectiveness of fault localization. A statistical fault localization technique using a smaller fraction of a prioritized test suite is found to compromise its effectiveness seriously. Despite the presence of some variations, the inclusion of more failed test cases will generally improve the fault localization effectiveness during the integration process. Interestingly, during the variation periods, adding more failed test cases actually deteriorates the fault localization effectiveness. In terms of strategies, Random is found to be the most effective, followed by the ART and Additional strategies, while the Total strategy is the least effective. We do not observe sufficient empirical evidence to conclude that using different coverage granularity levels have different overall effects. Conclusion: The paper empirically identifies that strategy and time-cost of test case prioritization techniques are key factors affecting the effectiveness of statistical fault localization, while coverage granularity is not a significant factor. It also identifies a mid-range deterioration in fault localization effectiveness when adding more test cases to facilitate debugging. © 2012 Elsevier B.V. All rights reserved.
收录类别	ei
部门归属	(1) School of Computer Science and Engineering, Beihang University, Beijing, China; (2) State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, China; (3) Department of Computer Science, City University of Hong Kong, Tat Chee Avenue, Hong Kong; (4) Department of Computer Science, University of Hong Kong, Pokfulam, Hong Kong; (5) Centre for Software Analysis and Testing, Swinburne University of Technology, Melbourne, Australia
语种	英语
WOS记录号	WOS:000304387600006
引用统计
内容类型	期刊论文
URI标识	http://ir.iscas.ac.cn/handle/311060/14726
专题	中国科学院软件研究所
推荐引用方式 GB/T 7714	Jiang Bo,Zhang Zhenyu,Chan W.K.,et al. how well does test case prioritization integrate with statistical fault localization?[J]. Information and Software Technology,2012,54(7):739-758.
APA	Jiang Bo,Zhang Zhenyu,Chan W.K.,Tse T.H.,&Chen Tsong Yueh.(2012).how well does test case prioritization integrate with statistical fault localization?.Information and Software Technology,54(7),739-758.
MLA	Jiang Bo,et al."how well does test case prioritization integrate with statistical fault localization?".Information and Software Technology 54.7(2012):739-758.

条目包含的文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
1-s2.0-S095058491200（1818KB）			开放获取	使用许可	请求全文