J Mending et al./ Information Systems 35(2010)467-482 Average perceived usefulness scores for ambiguous versus unambiguous label types. Unambiguous label, N= 132 Ambiguous label. N= 4 StDev Mean F 4.538 1241 3.238 1.495 3 4 confirm our assump- low familiarity ) Both variables have been described in for hypothesis H3. The Section 3. 1. Appendix A lists all items used in the questionnaire. We obtained the following results at were not listed as ambiguous by the participants (reported average total Application domain knowledge does not showa factor scores are 4.538 in contrast to 3. 238). The ANOVA significant interaction effect on the relationship be- test showed these differences to be statistically significant tween label type and perceived usefulness(F= 1.36 p=0. 245. partial eta square=0.008). Accordingly, hy- Moderating effects: As discussed in the demo pothesis H4a must be refuted section, the participants ranged in terms of their Notation familiarity does not show a significant parity with the EpC notation used in the process mod interaction effect on the relationship betw well as in their knowledge of the chosen application type and perceived usefulness(F= 1.334, p=0.239, domain(complaints handling). More precisely, six part artial eta square=0.006). Accordingly, hypothesis ipants brought to bear experience with complaint H4b must be refuted handling domain, and 17 out of 29 participants were above the median in notation familiarity These results are similar to those reported in [11, 26. Again we first established reliability and validity of the which also did not indicate sig measure "familiarity with the epC notation" Cronbachs ox gnificant moderation effects or the familiarity scale was computed to be 0.914, and of their measures of application domain knowledge or composite reliability was computed to be 0.859. Factor familiarity with the notation on understanding of con- loadings for the three familiarity measures were 0.919 ceptual models-and contrary to those reported in 0.930 and 0.931. all significant at p=0.000. Average [38, 39]. both of which reported some spurious effects on variance extracted of the familiarity construct was a number of the dependent variables they considered In estimated to be 0.927. as described above, AvE also indicate that understanding of textual labels contained in expertise gained familiarity construct. Altogether, these result suggest from previous notation usage or from previous knowledge adequate reliability and validity. Appendix B summarizes of the considered domain. In light of the other results In order to test hypotheses H4a and H4b, we examined presented above, the findings suggest that a labels perceived usefulness of the labels between two sets of two of the labels itsel sd dependent on the grammatical style he differences in the average total factor scores for usefulness is indee groups of participants (high/low application domain knowledge and high/low familiarity with the EPC nota 3.3. Discussion tion). Support for the hypotheses would then exist if the differences in the dependent variables between the The support for our hypotheses strongly suggests that a groups would be significant. We used an analysis of verb-object labeling style is rightfully proposed as covariance(ANCOVA)test implemented in SPSS 16.0 to preferred way of activity labeling. Indeed, our results test the hypotheses. ANCOVA is an appropriate analysis indicate strong and favorable percep towards a technique because it allows to control for potential effects superiority of the verb-object labeling style. Given the of covariates in the examination of dependent variable scores between two treatment groups [55]. ANcoVa perceived usefulness) play in informing actual usage assumptions of equal slopes were tested prior to conduct, behavior [44, 56, 57, we deem this finding instrumental d showed no violation of normality. to explaining, and supporting, process model understand- We used two covariates in the analysis of the effect on ability. However, whilst process modelers tend to favor beling type on perceived usefulness. The first is the verb-object styles, this situation does not necessarily binary variable" Knowledge of the complaints handling reflect actual usage for activity labeling. In fact, our domain", which simply establishes the existence of any exploration of the usage frequency of activity labels in the relevant knowledge in this domain. As a second covariate, SAP reference model indicates that a large proportion of we used the median of the total factor score of the three labels found in practice cannot be interpreted as genuine item"Familiarity " scale, to separate the respondents pool implementations of this style(see Section 2). In contrast, in two groups using a dummy variable(high familiarity/ ur results indicate that there is wide variety in labeling.The results displayed in Table 4 confirm our assump￾tion and lend strong support for hypothesis H3. The average total factor score for perceived usefulness was higher for those label types that were not listed as ambiguous by the participants (reported average total factor scores are 4.538 in contrast to 3.238). The ANOVA test showed these differences to be statistically significant at p ¼ 0:000. Moderating effects: As discussed in the demographics section, the participants ranged in terms of their famil￾iarity with the EPC notation used in the process model, as well as in their knowledge of the chosen application domain (complaints handling). More precisely, six parti￾cipants brought to bear experience with complaints handling domain, and 17 out of 29 participants were above the median in notation familiarity. Again we first established reliability and validity of the measure ‘‘familiarity with the EPC notation’’. Cronbach’s a for the familiarity scale was computed to be 0.914, and composite reliability was computed to be 0.859. Factor loadings for the three familiarity measures were 0.919, 0.930 and 0.931, all significant at p ¼ 0:000. Average variance extracted of the familiarity construct was estimated to be 0.927. As described above, AVE also exceeded the squared correlation between the PU and the familiarity construct. Altogether, these result suggest adequate reliability and validity. Appendix B summarizes factor loadings, communalities, and correlations. In order to test hypotheses H4a and H4b, we examined the differences in the average total factor scores for perceived usefulness of the labels between two sets of two groups of participants (high/low application domain knowledge and high/low familiarity with the EPC nota￾tion). Support for the hypotheses would then exist if the differences in the dependent variables between the groups would be significant. We used an analysis of covariance (ANCOVA) test implemented in SPSS 16.0 to test the hypotheses. ANCOVA is an appropriate analysis technique because it allows to control for potential effects of covariates in the examination of dependent variable scores between two treatment groups [55]. ANCOVA assumptions of equal slopes were tested prior to conduct, and showed no violation of normality. We used two covariates in the analysis of the effect on labeling type on perceived usefulness. The first is the binary variable ‘‘Knowledge of the complaints handling domain’’, which simply establishes the existence of any relevant knowledge in this domain. As a second covariate, we used the median of the total factor score of the three item ‘‘Familiarity’’ scale, to separate the respondents pool in two groups using a dummy variable (high familiarity/ low familiarity). Both variables have been described in Section 3.1. Appendix A lists all items used in the questionnaire. We obtained the following results: Application domain knowledge does not show a significant interaction effect on the relationship be￾tween label type and perceived usefulness (F ¼ 1:363, p ¼ 0:245, partial eta square ¼ 0:008). Accordingly, hy￾pothesis H4a must be refuted. Notation familiarity does not show a significant interaction effect on the relationship between label type and perceived usefulness (F ¼ 1:334, p ¼ 0:239, partial eta square ¼ 0:006). Accordingly, hypothesis H4b must be refuted. These results are similar to those reported in [11,26], which also did not indicate significant moderation effects of their measures of application domain knowledge or familiarity with the notation on understanding of con￾ceptual models—and contrary to those reported in [38,39], both of which reported some spurious effects on a number of the dependent variables they considered. In the context of the study reported in this paper, the results indicate that understanding of textual labels contained in process models is independent from any expertise gained from previous notation usage or from previous knowledge of the considered domain. In light of the other results presented above, the findings suggest that a label’s usefulness is indeed dependent on the grammatical style of the labels itself. 3.3. Discussion The support for our hypotheses strongly suggests that a verb–object labeling style is rightfully proposed as a preferred way of activity labeling. Indeed, our results indicate strong and favorable perceptions towards a superiority of the verb–object labeling style. Given the key role that usage beliefs (such as perceived ambiguity or perceived usefulness) play in informing actual usage behavior [44,56,57], we deem this finding instrumental to explaining, and supporting, process model understand￾ability. However, whilst process modelers tend to favor verb–object styles, this situation does not necessarily reflect actual usage for activity labeling. In fact, our exploration of the usage frequency of activity labels in the SAP reference model indicates that a large proportion of labels found in practice cannot be interpreted as genuine implementations of this style (see Section 2). In contrast, our results indicate that there is wide variety in labeling. ARTICLE IN PRESS Table 4 Average perceived usefulness scores for ambiguous versus unambiguous label types. Unambiguous label, N ¼ 132 Ambiguous label, N ¼ 42 ANOVA Mean StDev Mean StDev F Sig. Perceived usefulness 4.538 1.241 3.238 1.495 31.553 0.000 476 J. Mendling et al. / Information Systems 35 (2010) 467–482