IEEE TRANSACTIONS ON SOFTWARE ENGINEERING,VOL.41.NO.4.APRIL 2015 331 Are Slice-Based Cohesion Metrics Actually Useful in Effort-Aware Post-Release Fault-Proneness Prediction?An Empirical Study Yibiao Yang,Yuming Zhou,Hongmin Lu,Lin Chen,Zhenyu Chen,Member,/EEE,Baowen Xu, Hareton Leung,Member,IEEE,and Zhenyu Zhang,Member,IEEE Abstract-Background.Slice-based cohesion metrics leverage program slices with respect to the output variables of a module to quantify the strength of functional relatedness of the elements within the module.Although slice-based cohesion metrics have been proposed for many years,few empirical studies have been conducted to examine their actual usefulness in predicting fault-proneness. Objective.We aim to provide an in-depth understanding of the ability of slice-based cohesion metrics in effort-aware post-release fault-proneness prediction,i.e.their effectiveness in helping practitioners find post-release faults when taking into account the effort needed to test or inspect the code.Method.We use the most commonly used code and process metrics,including size,structural complexity,Halstead's software science,and code churn metrics,as the baseline metrics.First,we employ principal component analysis to analyze the relationships between slice-based cohesion metrics and the baseline metrics.Then,we use univariate prediction models to investigate the correlations between slice-based cohesion metrics and post-release fault-proneness.Finally,we build multivariate prediction models to examine the effectiveness of slice-based cohesion metrics in effort-aware post-release fault-proneness prediction when used alone or used together with the baseline code and process metrics.Results.Based on open-source software systems,our results show that:1)slice-based cohesion metrics are not redundant with respect to the baseline code and process metrics;2)most slice-based cohesion metrics are significantly negatively related to post-release fault-proneness; 3)slice-based cohesion metrics in general do not outperform the baseline metrics when predicting post-release fault-proneness;and 4)when used with the baseline metrics together,however,slice-based cohesion metrics can produce a statistically significant and practically important improvement of the effectiveness in effort-aware post-release fault-proneness prediction.Conclusion.Slice-based cohesion metrics are complementary to the most commonly used code and process metrics and are of practical value in the context of effort-aware post-release fault-proneness prediction Index Terms-Cohesion,metrics,slice-based,fault-proneness,prediction,effort-aware ◆ 1 INTRODUCTION OHESION refers to the relatedness of the elements cohesion is a subjective concept and hence is difficult to use within a module [1],[21.A highly cohesive module is in practice [141.In order to attack this problem,program one in which all elements work together towards a single slicing is applied to develop quantitative cohesion metrics, function.Highly cohesive modules are desirable in a system as it provides a means of accurately quantifying the interac- as they are easier to develop,maintain,and reuse,and tions between the elements within a module [12].In the last hence are less fault-prone [1],[2].For software developers, three decades,many slice-based cohesion metrics have it is expected to automatically identify low cohesive mod- been developed to quantify the degree of cohesion in a ules targeted for software quality enhancement.However, module at the function level of granularity [3],[4],[51,[61, [7],[8],[9],[10].For a given function,the computation of a slice-based cohesion metric consists of the following two .Y.Yang,Y.Zhou,H.Lu,L.Chen,and B.Xu are with the State Key steps.At the first step,a program reduction technology Laboratory for Novel Software Technology,Department of Computer Science and Technology,Nanjing UIniversity,Nanjing 210023,China. called program slicing is employed to obtain the set of pro- E-mail:yangyibiao@smail.nju.edu.cn,(zhouyuming,hmlu,Ichen, gram statements (i.e.program slice)that may affect each bioxufenju.edu.cn. output variable of the function [9],[11].The output varia- Z.Chen is with the State Key Laboratory for Novel Software Technology, bles include the function return value,modified global School of Software,Nanjing UIniversity,Nanjing 210023,China. E-mail:zychen@software.nju.edu.cn. variables,modified reference parameters,and variables H.Leung is with the Department of Computing,Hong Kong Polytechnic printed or other outputs by the function [12].At the second University,Hung Hom,Hong Kong,China. step,cohesion is computed by leveraging the commonality E-mail:cshleung@inet.polyu.edu.hk Z.Zhang is with the State Key Laboratory of Computer Science,Institute among the slices with respect to different output variables. of Software,Chinese Academy of Sciences,Beijing,China. Previous studies showed that slice-based cohesion metrics E-mail:zhangzy@ios.ac.cn. provided an excellent quantitative measure of cohesion [3], Manuscript received 16 Feb.2013;revised 24 Oct.2014;accepted 29 Oct. [13],[14].Hence,there is a reason to believe that they 2014.Date of publication 11 Nov.2014;date of current version 17 Apr.2015. should be useful predictors for fault-proneness.However, Recommended for acceptance by T.Menzies. For information on obtaining reprints of this article,please send e-mail to: few empirical studies have so far been conducted to exam- reprints@ieee.org,and reference the Digital Object Identifier below. ine the actual usefulness of slice-based cohesion metrics Digital Object Identifier no.10.1109/TSE.2014.2370048 for predicting fault-proneness,especially compared with See http://www.ieee.org/publi standards/pAre Slice-Based Cohesion Metrics Actually Useful in Effort-Aware Post-Release Fault-Proneness Prediction? An Empirical Study Yibiao Yang, Yuming Zhou, Hongmin Lu, Lin Chen, Zhenyu Chen, Member, IEEE, Baowen Xu, Hareton Leung, Member, IEEE, and Zhenyu Zhang, Member, IEEE Abstract—Background. Slice-based cohesion metrics leverage program slices with respect to the output variables of a module to quantify the strength of functional relatedness of the elements within the module. Although slice-based cohesion metrics have been proposed for many years, few empirical studies have been conducted to examine their actual usefulness in predicting fault-proneness. Objective. We aim to provide an in-depth understanding of the ability of slice-based cohesion metrics in effort-aware post-release fault-proneness prediction, i.e. their effectiveness in helping practitioners find post-release faults when taking into account the effort needed to test or inspect the code. Method. We use the most commonly used code and process metrics, including size, structural complexity, Halstead’s software science, and code churn metrics, as the baseline metrics. First, we employ principal component analysis to analyze the relationships between slice-based cohesion metrics and the baseline metrics. Then, we use univariate prediction models to investigate the correlations between slice-based cohesion metrics and post-release fault-proneness. Finally, we build multivariate prediction models to examine the effectiveness of slice-based cohesion metrics in effort-aware post-release fault-proneness prediction when used alone or used together with the baseline code and process metrics. Results. Based on open-source software systems, our results show that: 1) slice-based cohesion metrics are not redundant with respect to the baseline code and process metrics; 2) most slice-based cohesion metrics are significantly negatively related to post-release fault-proneness; 3) slice-based cohesion metrics in general do not outperform the baseline metrics when predicting post-release fault-proneness; and 4) when used with the baseline metrics together, however, slice-based cohesion metrics can produce a statistically significant and practically important improvement of the effectiveness in effort-aware post-release fault-proneness prediction. Conclusion. Slice-based cohesion metrics are complementary to the most commonly used code and process metrics and are of practical value in the context of effort-aware post-release fault-proneness prediction. Index Terms—Cohesion, metrics, slice-based, fault-proneness, prediction, effort-aware Ç 1 INTRODUCTION COHESION refers to the relatedness of the elements within a module [1], [2]. A highly cohesive module is one in which all elements work together towards a single function. Highly cohesive modules are desirable in a system as they are easier to develop, maintain, and reuse, and hence are less fault-prone [1], [2]. For software developers, it is expected to automatically identify low cohesive mod￾ules targeted for software quality enhancement. However, cohesion is a subjective concept and hence is difficult to use in practice [14]. In order to attack this problem, program slicing is applied to develop quantitative cohesion metrics, as it provides a means of accurately quantifying the interac￾tions between the elements within a module [12]. In the last three decades, many slice-based cohesion metrics have been developed to quantify the degree of cohesion in a module at the function level of granularity [3], [4], [5], [6], [7], [8], [9], [10]. For a given function, the computation of a slice-based cohesion metric consists of the following two steps. At the first step, a program reduction technology called program slicing is employed to obtain the set of pro￾gram statements (i.e. program slice) that may affect each output variable of the function [9], [11]. The output varia￾bles include the function return value, modified global variables, modified reference parameters, and variables printed or other outputs by the function [12]. At the second step, cohesion is computed by leveraging the commonality among the slices with respect to different output variables. Previous studies showed that slice-based cohesion metrics provided an excellent quantitative measure of cohesion [3], [13], [14]. Hence, there is a reason to believe that they should be useful predictors for fault-proneness. However, few empirical studies have so far been conducted to exam￾ine the actual usefulness of slice-based cohesion metrics for predicting fault-proneness, especially compared with
Y. Yang, Y. Zhou, H. Lu, L. Chen, and B. Xu are with the State Key Laboratory for Novel Software Technology, Department of Computer Science and Technology, Nanjing University, Nanjing 210023, China. E-mail: yangyibiao@smail.nju.edu.cn, {zhouyuming, hmlu, lchen, bwxu}@nju.edu.cn.
Z. Chen is with the State Key Laboratory for Novel Software Technology, School of Software, Nanjing University, Nanjing 210023, China. E-mail: zychen@software.nju.edu.cn.
H. Leung is with the Department of Computing, Hong Kong Polytechnic University, Hung Hom, Hong Kong, China. E-mail: cshleung@inet.polyu.edu.hk.
Z. Zhang is with the State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, China. E-mail: zhangzy@ios.ac.cn. Manuscript received 16 Feb. 2013; revised 24 Oct. 2014; accepted 29 Oct. 2014. Date of publication 11 Nov. 2014; date of current version 17 Apr. 2015. Recommended for acceptance by T. Menzies. For information on obtaining reprints of this article, please send e-mail to: reprints@ieee.org, and reference the Digital Object Identifier below. Digital Object Identifier no. 10.1109/TSE.2014.2370048 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 41, NO. 4, APRIL 2015 331 0098-5589
2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information